Comparing and Patching JSON Documents Without Surprises

Sooner or later two JSON documents have to be reconciled. A deploy changes an API response and someone asks what is different. A config is edited in two places and the two copies must be combined. A client sends a partial update, and the server has to apply it without clobbering the rest. A history of a document is needed for an audit. The tools for each of these look similar, and the differences between them are the kind that do not show on a small example: an array that loses its order, a null that means "delete", an operation that fails halfway, a diff that is a thousand times bigger than the change.

This guide is the path I take through those problems, in order: make the two documents comparable, decide what to do about arrays, choose what the output of a comparison should be, then patch, merge and test. Each section has measured results and links to the shorter articles with the details.

Three different jobs

"Compare" hides three jobs, and the right tool differs.

1. Are they the same? A yes or no, for a test, a cache check or a change detector. Parsed values compared with == answer it.

2. What is different? A report for a person: which paths were added, removed or changed, and from what to what.

3. How do I turn one into the other? A replayable description, a patch, that a program can apply to the first document to get the second.

Most of the confusion in this area is using the tool for one job on another. A text diff is a poor answer to the first (it fails on key order) and a worse answer to the third. A JSON Patch is a good answer to the third and a hard thing to read for the second. A path report is the best answer to the second and cannot be applied. Decide which job you have before you pick a library.

Step 1: make the documents comparable

JSON allows many spellings of the same data: any key order, any whitespace, é or é, 1.0 or 1. A comparison that reads the text sees all of them as differences. Two config files with the same content but different layout, key order and escaping, and a single changed value, gave this:

diff a.json b.json                          12 changed lines
diff <(jq -S . a.json) <(jq -S . b.json)    1 changed line   (TIMEOUT_MS: 3000 -> 3500)

Sorting the keys and fixing the layout is the first step for any text comparison (diffing two JSON files ignoring key order and whitespace). For a yes or no, skip the text and compare parsed values: Python's == on two dictionaries, jq's ==, util.isDeepStrictEqual in Node. All three ignore key order and layout, and all three say 1.0 equals 1.

Know the traps at this stage:

• jq 1.6 rewrites numbers. It converted 1.0 to 1 and 100000000000000000000 to 1e+20, which silently changes a value while only meaning to reformat. jq 1.8.2 keeps the literals, and then its text diff reports 1.0 against 1 as a change. Pin the version, and for value comparison use ==.

• Sort order differs by language. Python and jq sort keys by code point, JavaScript by UTF-16 code unit, which put an emoji before U+FF5E where the others put it after. One formatter per repository (sorting JSON keys for clean version control diffs).

• Redirecting a file onto itself empties it. jq -S . f > f left 0 bytes. Write to a temporary file and move it.

• Duplicate keys vanish. Every parser keeps the last. If duplicates are possible, validate before comparing.

If the files live in git and you want readable history without rewriting them, git config diff.json.textconv 'jq -S .' plus *.json diff=json in .gitattributes normalizes the display only.

Step 2: decide what an array is

Objects are simple to compare because their members have names. Arrays are not, and every diff and patch tool has to make a choice about them. Three meanings, and the tool has to be told which:

• A sequence. Position is the data: steps, coordinates, a ranking. Compare in order.

• A set (or multiset). Position is an accident: tags, permissions, rows from a query without an ORDER BY. Compare ignoring order.

• A collection with identities. Each element has a key (sku, id) and elements should be matched by it. Compare by key.

For sets, compare canonical text: sort each element's JSON with sorted keys, then sort the list or count it with a Counter. On 200,000 records that took 1.06 s sorted and 1.32 s counted. A Counter (a multiset) notices a repeated record that a set hides, which is the safer default, and nested arrays need their own rule or they stay ordered (comparing JSON arrays where order does not matter). jq's sort orders any JSON values and ignores object key order; in JavaScript JSON.stringify keeps key order and needs a replacer that sorts.

For collections, the cheapest correct move is to key the array before comparing: turn [{"sku": "A", ...}, ...] into {"A": {...}, ...}. Paths then name the element (order.items.B-220.price) instead of its position, and a reordered list stops looking like a change. The cost of not doing it is the next section.

What position-based diffs cost

I built a document of 20,000 order lines (1.83 MB), then edited it: 40 quantities changed, 5 lines removed from the middle, 5 appended, one status changed. Fifty-one edits. Then compared before and after with each library:

tool                                      time        output
jsonpatch.make_patch (Python)             0.30 s      51 operations, 3,682 bytes
fast-json-patch compare (Node)              40 ms     68,662 operations, 4,479,840 bytes
DeepDiff, default (Python)                5.75 s      68,662 entries
jsondiffpatch.diff (Node)                  561 ms     delta of 1,646,604 bytes
DeepDiff, lines keyed by sku              3.24 s      5 added, 5 removed, 41 changed
DeepDiff, iterable_compare_func by sku   14.07 s      41 changed, 5 added, 5 removed
jsondiffpatch with objectHash            12.76 s      delta of 1,961 bytes
JSON Merge Patch (create_patch)           <0.01 s     1,834,055 bytes: the whole array

The difference is not speed, it is the size of the answer. Removing five lines from the middle shifts every later element up by up to five positions. A diff that compares position against position sees 20,000 elements with different neighbors and reports 68,662 changes. A library that aligns the arrays first (the Python jsonpatch one did) sees five removals. The keyed and hashed approaches see them too, and cost more time for it. Merge Patch cannot do anything but send the array again: its output was the size of the document.

Two consequences. If you store or transmit diffs, generate them with a tool that aligns arrays, or key the arrays; a 4.5 MB "patch" for a 3.7 KB change is a bug in the choice of tool. And do not assume the numbers transfer: check the output size on your own shape of data before you build a pipeline on a diff library.

Step 3: choose the output

Once the documents are comparable, pick the form of the answer.

A path report, for people. For two API responses, the useful output is a short list of added, removed and changed paths with old and new values. In jq it is a flatten-and-compare of leaf paths, with arrays keyed by identity; in Python DeepDiff with the volatile fields excluded. Two traps: jq's paths(scalars) silently skips false and null leaves (use tostream | select(length == 2)), and DeepDiff with ignore_order=True reports an element's index in the first document, so after a reorder the path can point to the wrong row (finding what changed between two API responses). On the test pair, nine paths of 33 noisy diff lines were the real changes, and removing request ids and timestamps before comparing is what reached them.

A patch, for machines. If the difference has to be replayed, stored, sent or undone, it should be data. Two standards exist and they are not interchangeable.

Step 4: JSON Patch and JSON Merge Patch

JSON Patch (RFC 6902) is an array of operations on JSON Pointer paths: add, remove, replace, move, copy, test.

[{ "op": "test",    "path": "/replicas", "value": 3 },
 { "op": "replace", "path": "/replicas", "value": 4 },
 { "op": "add",     "path": "/ports/-",  "value": 9090 }]

The details that bite are in how operations behave (what JSON Patch is and how RFC 6902 works). add on an array inserts and shifts; - appends; an index past the end is an error; add on an existing member replaces it; replace and remove on a missing path fail. A key containing / is written ~1 and one containing ~ is ~0, so /env/a~1b reaches the key a/b and /env/a/b raises an error. A test that fails stops the patch, which makes it a precondition and a way to detect a concurrent change without a version column. It compares values: key order ignored, [1, 2] not equal to [2, 1], 1 equal to 1.0, true not equal to 1.

JSON Merge Patch (RFC 7396) is a JSON object merged into the target. It is what you would write by hand:

{ "replicas": 4, "env": { "REGION": null } }

Nested objects merge; any other value replaces; null deletes. That makes it short (68 bytes against 164 for the equivalent JSON Patch on a four-field change) and limited in three ways that do not raise errors. A value cannot be set to null, because null means delete: a generated patch for "note": "hello" to "note": null produced {"note": null} and applying it removed the key instead. Arrays are replaced whole, so appending an element means sending the list; and a patch that is not an object replaces the document (JSON Merge Patch versus JSON Patch).

How to choose, short version: Merge Patch for partial updates to an object of scalar fields where null and absent mean the same; JSON Patch for arrays, for real nulls, for preconditions and for audit logs. Different libraries produce different but equivalent JSON Patches for the same pair (Python removed /ports/1, Node replaced /ports/1 and removed /ports/2), so compare patches by applying them, not by text.

Step 5: apply a patch safely

Applying is one call; the care is in what surrounds it (applying a JSON Patch in Python and JavaScript).

Atomicity. The RFC requires a patch to apply entirely or not at all, and libraries honor that only in their copying mode. Python's apply_patch copies the document first, so a failed patch leaves the input unchanged; with in_place=True the same patch (replace /replicas 4, then remove /missing) left replicas at 4. fast-json-patch mutates by default and did the same. The copy has a price on large documents: 295 ms to copy a 100,000-item document against 5 ms for 1,000 in-place replacements, so copy once and apply in place to the copy, then swap on success.

Validation. fast-json-patch's validate reports OPERATION_PATH_UNRESOLVABLE, OPERATION_OP_INVALID or OPERATION_VALUE_REQUIRED without applying anything. In Python, a malformed patch raises InvalidJsonPatch, an inapplicable operation JsonPatchConflict, a failed test JsonPatchTestFailed, and a path through a missing parent raises jsonpointer.JsonPointerException, which is not a subclass of the others, so catch both families.

Status codes. For an HTTP endpoint: 400 for a body that is not a valid patch, 409 (or 412 with If-Match) when a test fails, 422 when a valid patch cannot apply to this document. The stored document was unchanged in all four failure cases I tried.

Authorization. A client who can send any path can write any field. Check every operation's path and from against an allowlist of editable fields before applying.

Step 6: merging is not patching

A merge combines two documents that were edited independently, with no record of what changed. It is a different operation from applying a patch, and it needs four decisions (deep merging two JSON objects safely):

• Depth. {**a, **b}, Object.assign and jq's + are shallow: the second env replaces the first, and REGION is lost. jq's * and a ten-line recursive function merge nested objects.

• Arrays. On [8080, 8443, 9000] and [9090], the tools gave four answers: replace ([9090]), merge by index (lodash, [9090, 8443, 9000]), concatenate (deepmerge, four elements), or append by schema (jsonmerge). Choose on purpose.

• Null. In every tool, null in the override is a value that wins, not a delete.

• Aliasing and safety. A shallow copy shares nested dictionaries with the original, so a later edit changes both. In JavaScript, a recursive merge over parsed JSON that copies a __proto__ key set a property on Object.prototype for the whole process (({}).polluted read "yes"). lodash and deepmerge skip the key, a hand-written version must, and a version using Object.entries still gave the result an inherited isAdmin: true that did not show in JSON.stringify. Skip __proto__, constructor and prototype explicitly.

If you can say what the change is, apply it as a patch; merge only when you have two independent versions and no better information.

Step 7: keep history as patches

A patch is a record of a change, so a revision history is a base document plus a list of patches. I generated 101 revisions of an 18.5 KB service config, three edits each, and stored them both ways:

101 full snapshots                      1,923,898 bytes     19,033 gzipped
base + 100 JSON Patches                    31,694 bytes      3,469 gzipped
rebuild revision 100 by replaying all patches    54 ms
undo the last 10 revisions with inverse patches  restored revision 90 exactly

The history shrank to 1.6% of the snapshots, and gzip compressed the snapshots well (they repeat) but still left them five times larger. Rebuilding any revision is a replay from the base or from a periodic snapshot; keep a full snapshot every N revisions so that replay stays short. Inverse patches come from diffing the two revisions in the other direction (make_patch(new, old)), and applying them in reverse order undid ten revisions exactly. Store the inverse at write time if undo matters, because computing it later needs both documents.

Pick the library by its array behavior (Step 2), store the library version with the history, and treat the array alignment as part of the format: a patch generated by one tool applies correctly with any, but two tools will not give the same patch for the same change.

Step 8: test what you compare

The assertions in a test suite are comparisons, and they fail in two directions: too strict (flaky on a timestamp, a float, an order) or too loose (a subset check that cannot see an extra field). Three layers work (writing tests that compare JSON output):

• Exact equality for stable fields, matchers for volatile ones (pytest.approx, IsStr(regex=...), Jest's expect.any, expect.closeTo).

• A golden file with sorted keys, volatile fields replaced by a placeholder and floats rounded, regenerated only by an explicit UPDATE_GOLDEN=1 and reviewed like code. Jest's --ci mode refuses to write a missing snapshot, which is the behavior you want on a build server.

• Shape checks against a schema, separately from value checks.

Beware expect.arrayContaining: it passed [1, 2, 3] for [3, 1], so it ignores extras unless you also check the length.

Equal in meaning, different in text

Normalizing the layout does not make two documents comparable if the values themselves have several spellings. Five cases came up in the tests, and each fails in a different direction.

Numbers. 1.0 and 1 are the same number in JSON and different Python types. {"a": 1} == {"a": 1.0} is True, but DeepDiff reported a type_changes entry for it, and jsonpatch.make_patch on my two config files produced a second operation, replace /ratio 1, for a change that is only a different spelling. Float arithmetic adds noise: 0.1 + 0.2 is not 0.3, and DeepDiff's default reported it while significant_digits=6 made it equal. Decide on a tolerance and apply it consistently.

Booleans and integers. Python treats True as 1: {"a": True} == {"a": 1} is True, so an equality-based test would pass if an API changed a flag into a number. DeepDiff caught it as a type change, and jsonpatch's test operation rejected true against 1. When types matter, use a tool that compares them or check them separately.

Strings. The letter é can be one code point (U+00E9, length 4 for "café") or an e plus a combining accent (length 5). They render identically and compare unequal. If data comes from several systems, normalize to NFC before comparing (unicodedata.normalize("NFC", s)).

Timestamps. 2026-10-02T01:01:07Z, 2026-10-02 01:01:07+00 and 2026-10-02T03:01:07+02:00 are one instant. As text they are three values; parsed with fromisoformat they compared equal. Another tool in these tests rewrote the first form as the second while re-serializing, which would have shown up as a change in a text diff. For a field you know is a timestamp, parse both sides.

Null and absent. {"note": null} and {} are different documents and the same to a Merge Patch, which can neither set nor preserve a null. Decide which your system means.

The practical answer is to write a small normalization step for your documents, separate from the diff library: it drops the volatile fields, parses the timestamps, rounds the floats, sorts the sets and keys the collections, and then a plain equality or a generic diff does the rest. That step is where the knowledge of your data lives, and a test for it (a fixture with each kind of noise) is worth more than any option on the diff tool.

A worked cycle: bring one document in line with another

Here is the whole path on the two config files from Step 1, a.json (the old version) and b.json (the new one, with the keys shuffled and one real change).

1. Compare. jq -S . a.json against jq -S . b.json showed TIMEOUT_MS: 3000 -> 3500 and, with jq 1.8, the 1.0 against 1 line.

2. Generate a patch. jsonpatch.make_patch(a, b) gave [{"op": "replace", "path": "/env/TIMEOUT_MS", "value": 3500}, {"op": "replace", "path": "/ratio", "value": 1}]. The second operation is the spelling change from above; filter it out if your consumers treat the numbers as equal.

3. Apply to a copy. patch.apply(a) == b was True, and a was unchanged afterwards because the default mode copies.

4. Keep the inverse. make_patch(b, a) gave the two operations in reverse, with /ratio set back to 1.0 and TIMEOUT_MS to 3000, and applying it to the patched document returned exactly a.

5. Verify. Compare parsed values with == (or jq -e) and fail the job if the result is not equal to the target.

That cycle is the same for a 1 KB config and a 2 MB document: the only things that change are the time (89 ms to apply the 51-operation patch to the large document, 0.30 s to compute it) and the care needed with arrays.

Choosing by size and speed

For documents of a few kilobytes, every tool here is instant and the choice is about semantics. At 1.8 MB with 20,000 array elements, the choice starts to cost. fast-json-patch computed its answer in 40 ms and jsonpatch in 0.30 s, but the first returned a 4.5 MB patch and the second 3.7 KB; the fast one is only useful if the size of the result does not matter. DeepDiff is the most informative and the slowest, at 5.75 s for the positional comparison and 14 s with a matching function, which is acceptable in a test or a one-off report and not in a request handler. jsondiffpatch with an object hash made a delta of 1,961 bytes and took 12.8 s.

Memory follows the same pattern as any whole-document tool: both documents are held in memory, and a diff library builds more structure on top. At ten times the size, expect ten times the time for the linear tools and a hundred for the quadratic ones. When documents reach hundreds of megabytes, stop diffing as documents. Convert to rows and let a database compare them, or compare per record with a hash of the canonical text and report only the records whose hashes differ.

Reading a patch before you apply it

A generated patch is a program for your data, and it deserves a look before it runs on anything that matters. Print it, count the operations and compare the count with what you expected: fifty-one operations for fifty-one edits is right, and 68,662 for the same edits means the array alignment failed and the patch will rewrite most of the document. Look at the paths. A path under a field the change should not touch (an id, a created timestamp) means the diff saw something you did not mean to change, such as a field the formatter rewrote. Look for replace operations whose old and new values are equal in meaning, like the 1.0 to 1 one above, and filter them out. Then apply the patch to a copy, compare the result with the target, and only then to the real document.

Keeping the result reproducible

Two machines running different versions of a diff library can produce different, equally valid patches for the same pair, and different versions of jq can format the same file differently. Write the versions down where the output lives: the patch file's header, the commit message, the CI log. Pin the formatter in the repository's tooling, and keep a fixture directory with a few awkward pairs (reordered keys, a shuffled array, a 1.0 against 1, a false turning into null) and the expected report for each. When a library upgrade changes a result, the fixtures say so in the pull request that upgrades it, not in production a month later.

A decision table

the question                                      use
same or not                                       parsed ==, jq -e, isDeepStrictEqual
what changed, for a reader                        keyed path report (jq or DeepDiff)
show me the diff in git                           textconv with jq -S
what changed in an API between deploys            path report, volatile fields removed
partial update of an object of scalars            JSON Merge Patch
update with arrays, real nulls, a precondition    JSON Patch with a test
history of a document                             base + patches, snapshots every N
combine two independent edits                     deep merge, explicit array rule
tags or rows in any order                         canonical strings in a Counter

When a diff and patch is more machinery than the job needs

The document is small and replaced as a whole. Sending 2 KB is cheaper than the code to diff it. Patching pays when documents are large, when changes are small, or when you need the record of change.

You need to edit one field in a database. PostgreSQL's jsonb_set and MongoDB's update operators change a path in place without reading the document into the application, and avoid overwriting concurrent changes.

Elements have identities and change independently. A list of comments or order lines belongs in its own table or collection with ids, not in an array that is patched by position.

People edit the same text. For collaborative editing, a diff of JSON is the wrong layer; use a structure built for concurrent edits.

Large files. All the tools above read whole documents. For multi-gigabyte inputs, convert to JSON Lines and compare sorted lines or load both into a database (handling JSON files too large to parse).

The standards are RFC 6902 (JSON Patch), RFC 7396 (JSON Merge Patch) and RFC 6901 (JSON Pointer). The libraries are documented at python-json-patch.readthedocs.io, zepworks.com/deepdiff and the jq manual. To look at a document before and after, use the JSON formatter.