Wednesday, August 27, 2008

Asp include file causing wrong encoding

We faced this interesting problem in while developing our website. There are 2 different pages that show address fields - in Korean characters. Both are read from the same table. Problem is these korean characters that appears properly one of the page and other one it is garbled.

First I thought the encoding is not set properly in the page. So after spending sometime to trace to see if we are not setting the page culture - we were in the same old state where the characters appears inconsistent. Same piece of code on both the asp but even we set the encoding properly the content is garbled in one case. Could that be a virus?

You will be surprised - if you were not - atleast I was surprised to know that an include on top of the asp page was causing issue. If we remove a particular asp include in that page - the page renders properly. Ok thought that include file has some code that corrupts the encoding or character or something. Nothing of that sort.

Ok so what next? I had no idea and neither my friend. Then it struck that the ASP file while saving from Editor itself can have encoding specified and this can affect the encoding. There is something called UTF-8 BOM(Byte Order Mark). Some of the editor when we save the file would add this BOM and usually is not seen in some of the editors - it hides the character. I have noticed this character appear in VSS file version comparer. It often looks like "" - in case you get to see - at the beginning of the file. So if you have this character then it affects ASP encoding and causing above behaviour to appear.

Removing BOM - check here

Quickest thing to do was create a notepad file and copy pasted the content of the original file and overwrite the problem causing file. This fixed the problem.