Wednesday, July 29, 2009

JSON Response Encoding Troubles

On the project I am currently on, we have had some troubles with some of the JSON responses returned from the server. Specifically if the strings contained escaped characters. We are using ExtJs on this project and upon receiving the response on the client, we were doing an Ext.decode(response.responseText); This was throwing exceptions, like error: unterminated string literal.

All the values that caused problems were escaped characters.
\"
\\
\r
\n
\t

It turns out that those escape sequences in the response are treated as part of the JSON source, not the JSON data. If your data contains escaped characters, the '\' needs to be encoded so that it becomes part of the data, and the Ext.decode(or eval) will then remove the extra '\'. The list above becomes:

\" --> \\\"
\\ --> \\\\
\r --> \\r
\n --> \\n
\t --> \\t

\\ is particularly bad if it is the last character in the data and it is not encoded, as it will combine with the field terminating ", and cause the JSON to fail.

\n or \r, cause the JSON source to have a return in the middle of the JSON, thus invalidating it.

\" will cause the field to prematurely terminate, causing the JSON to fail.

\t is really pretty harmless, as it actually tabs the data, but at least it doesn't cause the JSON to fail. Other ones that may be of concern, but haven't investigated.
\b - backspace
\f - formfeed
\v - vertical tab
\' - single quote(haven't noticed this one being a problem)