Fixing JSON Parse Errors by Reading the Message Properly

A JSON parse error looks like a verdict, and it is closer to a bug report. The parser got as far as it could, stopped at one character and told you what it was looking for when it stopped. That is exactly the information needed to find the problem, and most people read only the first words ("Unexpected token") and start guessing. The messages differ by language and version, the positions count from different origins and the same mistake produces a different sentence in each tool. Learning to read them takes an hour and pays back every time a response fails to parse at 2 a.m. This guide explains what the parts of a message mean, shows real messages from Python, Node, PHP and jq for a dozen common faults, gives a method for finding the problem quickly and lists the sources of bad JSON in the order I would suspect them. The detailed cases are covered by the articles linked along the way, including unexpected token at position 0, unexpected end of JSON input and single quotes instead of double quotes.

The anatomy of a message

Every parse error answers three questions: where, what and why. Where is a position, expressed as a character offset, a line and column or both. What is the thing the parser met, such as a token it did not expect. Why is what it was expecting at that point, which is the part most worth reading. Take Python 3.13's message for a trailing comma:

Illegal trailing comma before end of object: line 1 column 46 (char 45)

The text says what is wrong (a trailing comma), where in the structure (before the end of an object) and where in the text (line 1, column 46, offset 45). Three subtleties govern every message.

Offsets count from zero, lines and columns from one. Python's char 45 and column 46 are the same place, the 46th character. Node 24 prints both: at position 8 (line 1 column 9). jq prints a line and a column. PHP prints neither and returns only an error code, with the message Syntax error.

The reported position is where the parser noticed, not where you made the mistake. For a missing comma, {"name": "Ada" "age": 36}, Python says Expecting ',' delimiter at char 15, the opening quote of "age": the parser had finished the value "Ada", expected a comma or a closing brace, and found a quote. The mistake is the comma that should be before that position. For a truncated document, the position is the end of the input. So the habit is to read the text before the reported position, not at it.

Positions count characters in some tools and bytes in others. Python and Node count characters. A tool that reports bytes will drift from the character position once the text has non-ASCII characters, by the extra bytes of each multi-byte character. When a position looks one or two off in a file with accents, that is why. Also, a BOM at the start counts as a character (one, U+FEFF) in some tools and is skipped in others, shifting every position by one.

A table of real messages

Here are the twelve inputs and what four tools said about each:

Read down the Python column and you see how much information a good message carries. Read across and you see the difference between tools. PHP's json_decode returns a code and a short string for nearly everything, so use JSON_THROW_ON_ERROR, log the input and find the position with another tool. jq 1.6's messages talk about numeric literals because its tokenizer tries to read anything that is not punctuation as a number, so Invalid numeric literal means "this word is not a value", whether it is undefined, <html> or a stray quote. Node 24's messages are the most explicit about expectation (Expected ',' or ']' after array element), and older Node and browsers said Unexpected token and a position, with less context.

A method that works

When a parse fails, do these in order, and stop at the first that explains it.

1. Look at the raw text, not the exception. Print or save the exact string that was passed to the parser. Many failures come from a variable that is not what you think it is: a response body that is an HTML error page, an empty string, the word undefined or null stringified, an object that was converted to text with str(). The text will tell you in a second what an hour of staring at code will not. If the data came over HTTP, look at the status code and the Content-Type header too: a 502 with text/html was never going to be JSON.

2. Read the character before the position. Show about 20 characters on each side of the reported offset. A short function does it:

import json

def explain(text):
    try:
        json.loads(text)
        print("valid")
    except json.JSONDecodeError as e:
        start = max(0, e.pos - 20)
        before, after = text[start:e.pos], text[e.pos:e.pos + 20]
        print(f"{e.msg} at line {e.lineno}, column {e.colno} (char {e.pos})")
        print("   ...", repr(before), "<<HERE>>", repr(after))

Run on four inputs, it printed:

Illegal trailing comma before end of object at line 1, column 46 (char 45)
   ... '"a", "b"], "age": 36' <<HERE>> ',}'
Expecting ',' delimiter at line 1, column 16 (char 15)
   ... '{"name": "Ada" ' <<HERE>> '"age": 36}'
Expecting ',' delimiter at line 1, column 34 (char 33)
   ... '", "tags": ["a", "b"' <<HERE>> ''
Expecting value at line 1, column 1 (char 0)
   ... '' <<HERE>> '<html><body>502 Bad Gateway</body></html>'

Each one is diagnosed at a glance. The first has a comma immediately after the marker, followed by a closing brace. The second has a missing comma between "Ada" and "age". The third ends at the marker with nothing after it, which means the text stopped there: truncation. The fourth has nothing before the marker and an HTML tag after it: the input is not JSON. The repr form matters, since it shows quotes, control characters and invisible characters that a plain print hides: a BOM shows as '\ufeff', a tab as '\t'.

3. Decide whether it is JSON at all. Position 0 is the signal. If the first character is <, you have HTML or XML. If it is a letter such as u, N, I, E or t, the text is a word, probably undefined, NaN, Error or a plain-text message. If it is a quote followed by a brace, you have double-encoded JSON (below). If the first thing is nothing, the body is empty. The parser cannot fix any of these, and the cure is upstream: find out why the server sent HTML (a failed request, an auth redirect, a gateway timeout) or why the variable is empty.

4. Check the end. An error at the last character, or Unexpected end of JSON input, means truncation: the text ended inside a structure. Truncation comes from a response cut off by a timeout, a proxy limit, a crashed server, a file written without being finished or a log line shortened by a size limit. Compare the length with what was expected, and look at the Content-Length header, if there is one.

5. Check the middle, then the encoding. If the error is in the middle, look for the usual syntax mistakes at that position: a trailing comma, a missing comma, single quotes, an unquoted key, a comment, a raw newline in a string, an unescaped backslash. If the position is right at the start and the first bytes are EF BB BF, it is a BOM. If accented characters came out wrong earlier, the encoding was mishandled before parsing (fixing garbled text in CSV files for good covers the diagnosis, which applies equally here).

Twelve causes, in the order I suspect them

Most bad JSON comes from a short list. Position and message tell you which.

1. The server returned HTML. An error page, a login redirect, a proxy message. Position 0, first character <. Look at the status code.

2. An empty body. Position 0, Unexpected end of JSON input or Expecting value. A 204 response, a timeout, a client that read the body twice.

3. **The word undefined or null as text.** A variable that was never set, then stringified. See unexpected token at position 0.

4. Truncation. The error is at the end of the text. See unexpected end of JSON input.

5. A trailing comma. The error points at the closing brace, or at the comma in Python 3.13 (trailing commas in JSON).

6. Single quotes. A Python repr, a JavaScript object literal. The error is at position 1. See single quotes instead of double quotes.

7. Unquoted keys. {a: 1}. Same position and message as single quotes.

8. Comments. // or /* */. Error at the first slash, or "extra data" after the value (why JSON has no comments).

9. A raw newline or tab inside a string. The message names a control character. Text built by hand without escaping.

10. A bad escape. \x41, a lone backslash in a Windows path. "C:\Users\ada" contains \U, which is not a JSON escape.

11. A byte order mark. Position 0, first character invisible. Files from Windows tools (the UTF-8 BOM in CSV has the same story).

12. Two documents. Extra data: a stream of several JSON values, a log file with one object per line fed to a single-document parser. Use JSON Lines handling.

Valid JSON that is wrong: double encoding

One failure produces no parse error at all and shows up on the next line. If a service serializes an object to JSON, puts that text in a field and serializes again, the result is a JSON string whose content is JSON. The first parse succeeds and returns a string:

"{\"id\": 7, \"ok\": true}"
str {"id": 7, "ok": true}
dict {'id': 7, 'ok': True}
once['id'] -> string indices must be integers, not 'str'

After one json.loads, the value is a str, and indexing it with "id" raised TypeError: string indices must be integers. Parsing it a second time gives the dict. The telltale is a response that starts with a quote and a brace, "{\", or a value that prints with backslashes in front of every quote. The right fix is to stop encoding twice at the producer, since the second layer doubles the size and hides the structure from schemas; the workaround is to parse again and say why in a comment.

Getting good data to debug with

Debugging is only as good as the evidence, so build the evidence collection into the code. When a parse fails, log the first 200 characters of the input, its length, the content type and status for HTTP and the error position, together with a request or correlation ID. Do not log full bodies where they may hold personal data: truncate, or hash. Keep the raw payload for failed parses in a short-lived store with access control, since reproducing the failure from a log line that says "JSON parse error" is rarely possible. In a client, check response.ok and the content type before calling .json(), so that a 502 produces "server returned 502 with text/html" and not a syntax error about a <.

From the command line, the same tools help: jq . file.json and python3 -m json.tool file.json print an error and a position (for the same file with a trailing comma on line 3, json.tool said Illegal trailing comma before end of object: line 3 column 11 (char 47) and jq 1.6 said Expected another key-value pair at line 4, column 1, pointing at the closing brace), and curl -i shows the headers and the body together. To find the position by hand, head -c 47 file.json | tail -c 20 shows the text before offset 47, and an editor with a "go to offset" feature does the same. For large files, bisect: parse the first half and the second half of each array and narrow down.

Errors in the browser, in servers and in frameworks

The same faults show up wrapped in different code, and a few patterns are worth knowing. In a browser, await response.json() raises when the body is not JSON, and the message is the parser's, so a gateway error page produces Unexpected token '<', "<!DOCTYPE "... is not valid JSON and nothing about a 502. The fix is to look at the response before parsing it:

const res = await fetch(url);
const type = res.headers.get("content-type") || "";
if (!res.ok || !type.includes("application/json")) {
  const body = await res.text();
  throw new Error(`HTTP ${res.status} ${type}: ${body.slice(0, 200)}`);
}
const data = await res.json();

That turns a cryptic syntax error into "HTTP 502 text/html: <html>...", which names the real cause. It also reads the body once, as text, so you can log it; a body can be consumed only once, and a second .json() after a failed first one gives a different, confusing error.

On the server side, framework body parsers reject malformed requests before your code runs, usually with a 400 and a message from the underlying parser. Express's JSON parser, for example, replies with a SyntaxError and the parser's text. Do not return that text unfiltered to the client in production, since it can reveal the library and the input, and do return a stable error body that clients can handle: a fixed code such as invalid_json, a human sentence and, if you want, the position. In PHP frameworks, json_decode with JSON_THROW_ON_ERROR raises a JsonException whose message is Syntax error, so catch it and add the context yourself. In Python web frameworks the exception is JSONDecodeError, whose pos, lineno and colno attributes give you the position to include in a 400 response.

One bad line in a stream of many

Logs, exports and queue messages are often one JSON document per line, and a single bad line should not stop the rest. Parse line by line, catch the error per line and keep a count. On a five-line file with one trailing comma and one line of plain text:

line 2: Illegal trailing comma before end of object (col 9): '{"id": 2,}'
line 4: Expecting value (col 1): 'not json'
3 valid, 2 invalid

jq can do the same in one line: jq -R 'fromjson? | .id' lines.jsonl printed 1 3 5, the values from the valid lines, with the ? suppressing the errors, and a select with try printed the two bad lines themselves. Counting the invalid ones and keeping them in a rejects file is the same discipline as for CSV (logging rejected rows from a CSV import for review), and a stream that suddenly has many invalid lines is a signal that something upstream changed, not a handful of random errors.

When you cannot fix the source

Sometimes the producer is not yours and the JSON is invalid in a consistent way: a vendor with trailing commas, a legacy service that writes single quotes, an export that includes comments. You have three options, in order of preference. Ask them to fix it, with a sample and the validator's message, which costs one email and is permanent. Parse with a tolerant parser that you choose on purpose, JSON5 for the extra syntax, and isolate it at the edge so that the rest of the system sees strict JSON. Or repair the text with a small, tested function for the specific fault, remembering that regular expressions on JSON break strings that contain the characters you are replacing. Whatever you choose, keep the original text, because a repaired document is a different document and you will want to see what was sent when the vendor disagrees.

Writing errors that help the next person

If you produce parse errors, for instance in an API that accepts JSON, make yours as good as the best parser's. Return a 400 status, the content type application/json, and a body with a stable machine-readable code, a sentence in plain words and the position when you have it. Say what was expected: "expected a property name in double quotes at line 3, column 5". Do not echo the whole body. Add a request ID to help support find the log entry. And keep a test for it: send a document with a trailing comma, one with single quotes and an empty body, and assert the status and the shape of the error, so that a framework upgrade that changes the wording does not change your contract.

When the text looks right and fails

The most frustrating failures are the ones where you can see nothing wrong. The document looks like valid JSON on screen, and the parser stops in the middle of it. The cause is a character that looks like something JSON allows and is not. Four come up repeatedly, and I ran each through Python and Node:

nbsp between tokens    Expecting value (char 5) near '":\xa01}'
smart quotes           Expecting property name enclosed in double quotes (char 1) near '{“a”'
zero-width space       Expecting value (char 5) near '":\u200b1}'
ideographic space      Expecting value (char 5) near '":\u30001}'

JSON allows exactly four whitespace characters, space, tab, line feed and carriage return. A non-breaking space (U+00A0), which word processors and web pages insert freely, looks like a space and is rejected: the message says Expecting value at the character where the value should start, and Node prints Unexpected token ' ' with a character that looks like a space inside the quotes. A zero-width space (U+200B) is invisible and fails the same way. Curly quotes copied from a document or a chat window, “a”, look like quotes and are two unrelated characters, so the parser reports that it expected a property name in double quotes at position 1. The repr of the text around the reported position shows each of them as an escape (\xa0, \u200b) where a plain print shows nothing odd.

The cure for a file with these is to replace them with the real characters, and the better cure is to stop copying JSON through tools that rewrite text: a chat message, a word processor, a web page's rendered output. If you must paste JSON from one, paste into a code editor that shows whitespace, and check the first error with the position display above. Search for the offending code points with a regular expression in your editor, [\u00a0\u200b\u3000\u201c\u201d], and fix them in one pass.

A debugging session, from message to cause

Putting the pieces together, here is how the method runs on a failure that was reported as "the API returns bad JSON". The client's log says Unexpected token '<', "<!DOCTYPE "... is not valid JSON. Step one, the raw text: the first characters are an HTML doctype. Step two, the first character is <, which settles the question of whether this is JSON at all: it is not, and nothing in the parser can fix it. Step three, the status and content type from the same log line: 502, text/html. The cause is not a JSON problem, it is a gateway error, and the investigation moves to the load balancer's logs.

A second report: the same client, a different endpoint, Unexpected end of JSON input. The raw text is empty, the status is 200 and the content length is zero. The server answered successfully with no body, probably an endpoint that returns nothing for an empty result and a client that expects []. The fix is on the server to return [], or on the client to handle an empty body, and the parse error was only the visible edge of it.

A third: a batch job reading a file fails with Expecting ',' delimiter: line 1 column 34 (char 33) and the position is the last character of the file. The text ends inside an array. The file was written by a process that was killed, and the fix is to write to a temporary name and rename on completion, so that a reader never sees a half-written file (scheduling CSV reports from SQL with cron shows the pattern for CSV, and it is the same for JSON).

None of these needed a code change in the parser. All three were solved by looking at the text, then at the position and then at where the text came from. The parser's message did its job in each case: it said the input was not what it expected, and said where.

Preventing parse errors

The best fix is to stop producing bad JSON. Generate JSON with a serializer, never with string concatenation. Validate output in tests with a strict parser, including the failure paths: the error response of your own API must be JSON too, with the same content type, so clients can parse it. Put a schema on both sides and validate at the boundary, so that errors say which field is wrong instead of which character. And write a test that sends the awkward inputs from a cheat sheet of JSON edge cases to throw at a parser through the code that parses untrusted JSON, so that the first time you see them is not in production.

When the message is not enough

Sometimes the message is right and still unhelpful: the position points at a place in a 40 MB file, or the file is valid in every tool you try and fails in one. For the first, bisect, or load it with a tolerant parser such as JSON5 and diff the result with a strict one. For the second, suspect the invisible: a BOM, a non-breaking space where a normal one should be (U+00A0 looks like a space and is not whitespace in JSON), a Unicode quote copied from a document (“ is not "), a zero-width character. xxd or the repr of the text near the position shows them.

Also remember what a parse error does not tell you. A document that parses is only syntactically valid: it can still be missing a field, have the wrong type or carry a value out of range, and those are schema questions, answered by a validator and not a parser. The grammar is RFC 8259, Python's exception attributes are described in the json module reference, and V8's error wording is documented with the language in the MDN page on JSON.parse errors.