Python JSONDecodeError Extra Data and Concatenated Objects

You load a file that looks like a perfectly good JSON object, and Python answers json.decoder.JSONDecodeError: Extra data: line 1 column 9 (char 8). That is a different message from the syntax errors, and it has a different meaning. The parser did not find a mistake inside the document. It found a complete document, finished with it and then discovered that the text was not over. This article explains the message, shows the five usual sources, and gives three readers for the cases where the extra text is legitimate data, followed by the cases where it is not.

What the parser is telling you

json.loads expects the whole string to be exactly one JSON value, optionally surrounded by whitespace. When it has read one value and the next non-whitespace character is not the end of the text, it raises Extra data and reports the position of that next character. Here are the common inputs and what Python 3.13 said:

{"a": 1}{"b": 2}              Extra data: line 1 column 9 (char 8)
{"a": 1}\n{"b": 2}\n          Extra data: line 2 column 1 (char 9)
{"a": 1},                     Extra data: line 1 column 9 (char 8)
{"a": 1} // note              Extra data: line 1 column 10 (char 9)
[1,2]]                        Extra data (char 5)

In every case the position is the first character after the first value (plus any whitespace). That is the useful fact: the offset tells you where the first value ended, and everything from there to the end of the text is what the parser could not place. Print the text from that position and you can classify it. text[e.pos:e.pos + 60] is enough.

The five usual sources

Objects glued together. A process appended records to a file without separators, or two JSON responses were written to the same stream. The extra text starts with { or [. This is the first row of the table above.

JSON Lines. The file has one complete object per line, which is a valid and popular format (see what JSON Lines is and why pipelines prefer it), but it is not a single JSON document, so json.load(f) on it fails at the start of line 2. If the file has an extension like .jsonl or .ndjson, or came from a log shipper or a database export, this is the likely cause.

A trailing comma after the document. {"a": 1}, shows up when someone copies an element out of an array and leaves its separator behind. The extra text is a comma.

A comment or a note. JSON has no comments, so // note after the value fails. The parser reports the position of the first slash. See JSON has no comments, so what do you do.

An extra closing bracket. [1,2]] finishes at the first ] and then finds another. It is the mirror image of a missing bracket and is common after hand edits.

Classifying takes a few seconds and saves guessing:

from json import JSONDecodeError, loads
try:
    loads(text)
except JSONDecodeError as e:
    if e.msg == "Extra data":
        print(repr(text[e.pos:e.pos + 60]))

If it prints something starting with '{', you have records. If it prints ',', '//' or ']', you have a stray character.

Reading JSON Lines

When there is one value per line, read the file the way it is written:

import json

records = []
with open("events.jsonl", encoding="utf-8") as f:
    for number, line in enumerate(f, start=1):
        line = line.strip()
        if not line:
            continue
        try:
            records.append(json.loads(line))
        except json.JSONDecodeError as e:
            raise ValueError(f"line {number}: {e.msg}") from e

On the three-line sample, [{"a": 1}, {"b": 2}, {"c": 3}] came back. Three things are worth the extra lines. Skipping blank lines tolerates a trailing newline. Reporting the line number points to the bad record without making the reader count. And this approach reads one record at a time, so memory stays flat for a file of any size, which the single json.load cannot do.

Do not "fix" such a file by splitting the whole text on }{ or on a newline followed by a brace. A string value can contain }{ or an escaped \n, and a regular expression has no idea where strings begin. Splitting by lines works because JSON Lines guarantees that a newline never occurs inside a record: a newline inside a string must be written as \n.

Reading glued values with raw_decode

If the values are concatenated with no newline, like {"a":1}{"b":2} {"c":3}, there are no lines to split on. The standard library has a decoder method for this: raw_decode(text, idx) parses one value starting at idx and returns it along with the index where it stopped.

import json

def values(text):
    decoder = json.JSONDecoder()
    pos = 0
    while True:
        while pos < len(text) and text[pos].isspace():
            pos += 1
        if pos >= len(text):
            return
        value, pos = decoder.raw_decode(text, pos)
        yield value

print(list(values('{"a": 1}{"b": 2}  {"c": 3}')))

The output was [{'a': 1}, {'b': 2}, {'c': 3}]. The loop skips whitespace by hand because raw_decode does not, and it stops when the text is used up. The decoder understands strings, so braces inside strings are no problem. For very large inputs, reading everything into one string first is a memory risk, and a streaming parser is the better tool (see handling large JSON files in Python with ijson).

Fixing a stray character

If the file was meant to be a single document, and the extra text is a comma, a comment or a bracket, the repair is to remove it, and the question is where the character came from. A script that edits the file should do it with the parser and write the result back with json.dump, not by deleting text. If the producer wrote it, fix the producer. Search for code that opens a file in append mode ("a") to write a document that should have replaced it. That one-character difference between "a" and "w" was the cause in more of these files than any other.

Telling the sources apart in a real file

If you are not sure what you have, three quick checks narrow it down without opening an editor. wc -l file tells you the number of lines, and a document with more than one line that begins with { on each is probably JSON Lines. head -c 200 file shows the start and tail -c 200 file the end, and a pretty-printed single document starts with a brace and indents what follows, while JSON Lines puts a whole object on every line. jq -c . file on JSON Lines prints each record and succeeds, because jq reads a stream of values, while it fails on a stray comma. That difference in jq behavior is also a quick way to confirm that a file is a stream; see selecting fields and filtering arrays with jq.

Why it matters for pipelines

A reader that raises on extra data is doing you a favor. json.loads that quietly stopped after the first object would hand back one record from a file of a thousand, and the pipeline would run to completion with 0.1 percent of the data. Libraries that offer a lenient "read the first value" mode exist, and using one on an input you did not verify is how silent loss happens. Count the records you read and compare with the count you expect, for example with wc -l for JSON Lines.

When you should not parse it as one document

Reading glued or line-separated values works when the data is meant to be a sequence. It is the wrong response in three cases.

The file is a truncated or merged copy of one document. Two exports concatenated by cat a.json b.json > all.json give {...}{...}, which is not the merge of the two documents, and reading it as two records may not be what you want. Merge them semantically (arrays concatenated, objects combined) and decide what happens on conflicts.

The consumer expects a single document. Wrapping the records into an array with json.dump(records, out) gives a valid single document, but the size now has to fit in memory for whoever reads it. For large data keep JSON Lines and say so in the file name.

The extra text means the producer is broken. If an API response is {"ok": true} followed by a stack trace, the stack trace is the story. Log it, fail the call, and report it to the owner instead of parsing around it.

For more on the grammar that defines what one complete value is, read RFC 8259, and for the decoder's methods the Python json module documentation. JSON Lines has its own short specification at jsonlines.org.