Canonical XML and Why Your Diffs Need It Before Comparing

You have two XML files that should be the same, and a byte comparison says they are not. Or you have a checksum stored last week and a file today, and the checksum no longer matches although a person reading both sees the same order. XML allows a lot of freedom in how a document is written, and most tools use that freedom differently. Canonical XML is the standard answer: one algorithm that turns any document into one agreed byte form, so that two documents with the same content end up with the same bytes.

The problem in three files

Here is one order written three ways. The first is what a person types, the second is what a Python library emitted, the third is what a template engine produced:

<?xml version="1.0" encoding="UTF-8"?>
<order id="7" status="paid" currency="EUR">
  <customer>Ana</customer>
  <total>19.90</total>
</order>

<?xml version='1.0' encoding='utf-8'?>
<order currency="EUR" status="paid" id="7"><customer>Ana</customer><total>19.90</total></order>

<order   id="7"   status='paid'   currency="EUR" >
<customer>Ana</customer>
<total>19.90</total></order>

They differ in the declaration (present with double quotes, present with single quotes, absent), in the order of the three attributes, in the quote character around status, in the spaces inside the start tag, and in the whitespace between elements. A reader sees one order. Tools see three files:

md5sum a1.xml a2.xml a3.xml

7fc43ff0bd86e0460a4bcdfea92b9de2  a1.xml
b397c1adc4b011cc55fd592b2130d94f  a2.xml
31bf2ecb8bea4246a2b68ad7a7e1691b  a3.xml

Three sums. A plain diff between the first two reports the whole file as changed, because the second has no line breaks. None of this is a mistake in any of the producers. The XML specification says attribute order carries no meaning, either quote character is allowed, and an encoding declaration may be written in any case. The freedom is deliberate, and it is exactly what makes bytes a poor basis for comparison.

What Canonical XML does

Canonical XML 1.0, usually called C14N, is a W3C Recommendation that defines a physical form for the logical content of a document. The rules that matter for comparison are these:

• The XML declaration and the document type declaration are removed.

• Attributes are sorted, namespace declarations first and then by name.

• Attribute values always use double quotes, and special characters inside them are escaped one fixed way.

• Empty elements are written as a start tag and an end tag, never as <a/>.

• Line endings are normalized to a single line feed, and the text is UTF-8.

• Whitespace inside a start tag is reduced to one space between attributes.

• CDATA sections are replaced by their character content, escaped.

• Comments are dropped in the default form and kept in the "with comments" form.

xmllint implements it with --c14n. Run it on the three files:

for f in a1 a2 a3; do xmllint --c14n $f.xml; echo; done

<order currency="EUR" id="7" status="paid">
  <customer>Ana</customer>
  <total>19.90</total>
</order>
<order currency="EUR" id="7" status="paid"><customer>Ana</customer><total>19.90</total></order>
<order currency="EUR" id="7" status="paid">
<customer>Ana</customer>
<total>19.90</total></order>

The attributes are now in one order, with double quotes, with no declaration. That fixed three of the five differences. The remaining one is visible in the output: the first file has indentation, the second has none, the third has line breaks without indentation. C14N did not touch them, and the checksums of the canonical forms still differ:

for f in a1 a2 a3; do xmllint --c14n $f.xml | md5sum; done

6b8fbe43d260336b65277b8ad9159237  -
fd25fb673877e64b33d9ef9f9f4e7714  -
55110c981c8b8cff53e63532d2d2cdb2  -

Why whitespace survives

In XML, whitespace between elements is part of the document. The parser reports it as text nodes, because the format cannot know whether the whitespace in <p>one <b>two</b> three</p> matters. In that paragraph it clearly does. The canonical form therefore keeps every text node exactly. Removing indentation is a decision about your data, not a property of XML, and C14N does not make that decision for you (when whitespace in XML matters covers the cases).

For data files, where the elements hold records and the indentation is decoration, you make the decision when you parse. With lxml, remove_blank_text=True drops text nodes that contain only whitespace between elements:

from lxml import etree
import hashlib

def canon(path):
    parser = etree.XMLParser(remove_blank_text=True)
    return etree.tostring(etree.parse(path, parser), method="c14n")

for f in ("a1.xml", "a2.xml", "a3.xml"):
    print(f, hashlib.sha256(canon(f)).hexdigest()[:16])
print(canon("a3.xml").decode())

a1.xml fdbd5b13d2bcc604
a2.xml fdbd5b13d2bcc604
a3.xml fdbd5b13d2bcc604
<order currency="EUR" id="7" status="paid"><customer>Ana</customer><total>19.90</total></order>

All three files give the same digest. This is the recipe for a content fingerprint of a data document: parse, drop blank text, canonicalize, hash. The same bytes can be diffed with an ordinary text tool, because both sides are now in one form. Pretty-printing the canonical bytes gives a readable diff, which is the subject of diffing XML while ignoring attribute order.

Where canonical XML is used

C14N was designed for XML Digital Signature, where a signature has to be computed over bytes that survive being parsed and re-serialized on the way. It is why signing libraries ask you to pick a canonicalization method, and why a signature breaks when a gateway reformats the signed part (XML signature wrapping explained covers a family of attacks that live in that gap). The same property makes it useful for change detection, cache keys and test fixtures: anywhere you need to ask whether two documents are "the same" without a person looking.

There are variants. Inclusive C14N 1.0 carries in-scope namespace declarations from ancestors onto the output element, which matters when you canonicalize a fragment. Exclusive C14N (--exc-c14n) keeps only the namespaces a fragment actually uses, which is why signatures use it for parts of a larger document. C14N 1.1 adjusts how xml:id and xml:base are handled. For comparing two whole data files, the plain form is the right one.

Related reading

The cluster hub is comparing and versioning XML. Before a diff, a pretty-printed form helps (pretty printing XML for cleaner Git history), and the command-line formatter is covered in formatting XML with xmllint. The specification is Canonical XML 1.0, and lxml documents its C14N support. To reformat a file in the browser, use the XML formatter.

When canonical XML does not give you equality

Element order. C14N never reorders elements. Two feeds with the same orders in a different sequence have different canonical forms. If order is not meaningful, sort first (sorting XML elements for stable output).

Equal values written differently. 19.9 and 19.90, true and 1, and 2026-01-05 and 2026-01-05T00:00:00 are different text. Normalizing values needs the schema's types, not the canonicalizer.

Mixed content. If the file is a document, such as XHTML or DocBook, with text between inline elements, removing blank text changes its meaning. Canonicalize without stripping and accept that formatting differences will show.

Comments and processing instructions. The default form drops comments. If a comment matters, use the with-comments form, and if it does not, check that the producer is not carrying data in comments.

Different namespace prefixes. Documents using different prefixes for one namespace canonicalize to different bytes, since the prefix is part of the serialization. The logical content is equal and the bytes are not. Convert both to the same prefixes before comparing, or compare by parsed structure instead of by bytes.