Disabling DTDs and External Entities in Java and Python

Two languages account for a large share of the XXE advisories I have read: Java, because its XML parsers are old and permissive by default, and PHP, covered separately. Python is mostly safe now, with a few switches that undo it. This article gives the settings for the Java parsers, the Python equivalents, and the result of each setting on the same attack file, so you can see which ones work.

Java: the default is the vulnerable one

var f = DocumentBuilderFactory.newInstance();
var doc = f.newDocumentBuilder().parse(new File("xxe.xml"));
System.out.println(doc.getElementsByTagName("v").item(0).getTextContent().trim());

default factory            -> 56ff69440bc5

With no configuration at all, the parser read /etc/hostname and returned its content, the container's host name. Any Java code that parses XML from a request with the default factory has this behavior. The same holds for SAXParserFactory, XMLInputFactory (StAX, with a different property), TransformerFactory and SchemaFactory, each of which has its own set of switches. Here is what I tried on the DOM factory, with the result of each:

The strictest: refuse the DOCTYPE

var f = DocumentBuilderFactory.newInstance();
f.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);

This is the setting I recommend as the default for any service that reads data, because it removes the whole mechanism. A document with a DOCTYPE is rejected before any entity is defined, so external entities, parameter entities and the entity expansion attacks all fail with one error. It is the setting the OWASP cheat sheet lists first. The price is that legitimate documents with a DTD are refused, which for APIs, feeds and uploads means none. Some configuration formats and legacy XML do carry a harmless DOCTYPE (an Apple property list does, for example); if you must accept them, use the next group of settings instead.

The softer ones: restrict access

f.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
f.setAttribute(XMLConstants.ACCESS_EXTERNAL_DTD, "");
f.setAttribute(XMLConstants.ACCESS_EXTERNAL_SCHEMA, "");

FEATURE_SECURE_PROCESSING turns on a bundle of limits in the JDK parsers, among them the restriction on accessing external resources and the entity expansion limits. In my run, enabling it alone blocked the read, and the error names the property that applied (accessExternalDTD). Setting ACCESS_EXTERNAL_DTD and ACCESS_EXTERNAL_SCHEMA to the empty string says that no protocol is allowed for fetching a DTD or schema, with the same effect. These are good settings for a defense in depth and for code that cannot reject a DOCTYPE. They leave the DOCTYPE itself and the internal entities working, so a document can still define and use an internal entity, within the expansion limits.

The old advice, the set of three features turning off external-general-entities, external-parameter-entities and load-external-dtd, also works, and its result is different in kind:

features off + no expand   ->

The parse succeeded and the value was empty. Nothing was read, which is safe, and nothing was reported, which is a trap for debugging: the document silently lost data. If you choose this route, log when a document contains a DOCTYPE, so the loss is visible. I prefer the explicit rejection because it fails loudly.

Which should you pick? For a service that accepts XML from outside: disallow-doctype-decl set to true, plus FEATURE_SECURE_PROCESSING, plus a size limit on the input. Apply the same recipe to every factory you create, and put it in one helper method so that no call site builds a parser on its own. A code search for newInstance() on the XML factories finds the places that need it. For JAXB, remember that unmarshalling reads from a source you must build yourself: pass a configured XMLStreamReader or SAXSource, not a File or InputStream, or JAXB makes its own default parser.

Python

The Python standard library and lxml are in better shape. I ran the same file:

lxml default                       XMLSyntaxError: Entity 'x' not defined, line 3, column 10
lxml resolve_entities=True         '4cc1864fc1ac\n'
lxml resolve_entities='internal'   XMLSyntaxError: Entity 'x' not defined, line 3, column 10
lxml resolve_entities=False        ''
lxml load_dtd + resolve True       '4cc1864fc1ac\n'
ElementTree                        ParseError: undefined entity &x;, line 3, column 6
defusedxml.ElementTree             EntitiesForbidden(name='x', system_id='file:///etc/hostname', ...)

Both lxml and ElementTree refuse the external entity by default. That is the current state, in the versions I ran, and it is a property of the library version, not of the language. The two lxml rows that read the file show how it breaks: resolve_entities=True turns the protection off, and combining load_dtd=True with resolve_entities=True and no_network=False does too. The audit is a search for those strings in the codebase. The settings I use as an explicit baseline for untrusted input:

parser = etree.XMLParser(resolve_entities=False, no_network=True, load_dtd=False,
                         dtd_validation=False, huge_tree=False)
root = etree.fromstring(data, parser)

resolve_entities=False leaves entity references unexpanded (the value was empty in the row above, like the Java route with the features off); no_network=True is the default and worth stating; huge_tree=False keeps the depth and size limits (limiting XML size and depth for uploads). If you want rejection and not silence, defusedxml is the tool:

import defusedxml.ElementTree as DET
DET.fromstring(data)          # raises EntitiesForbidden for any entity declaration

It raises EntitiesForbidden with the entity name and its system id, and also guards against the expansion attacks and external DTD retrieval. For a Python service that handles XML from outside, defusedxml is cheap and explicit, and a failing test makes sure it stays on. More in safe XML parsing in Python with defusedxml.

Python's other XML libraries are less consistent. xml.dom.minidom, in my run, parsed the file without reading it and returned None for the text node, and the documentation for the standard xml package lists which modules are vulnerable to which attacks; read that table for the Python version you run, since the answers change between releases.

One helper, one test

The setting is only as good as its coverage. Both languages follow the same pattern: a single factory function that returns a configured parser, a lint rule or code review habit that forbids creating parsers elsewhere, and a test that runs the three-line attack file through each entry point and asserts that it fails or returns nothing:

def test_rejects_external_entity():
    with pytest.raises(etree.XMLSyntaxError):
        parse_untrusted(open("xxe.xml", "rb").read())

Run the same test against the billion laughs file (stopping the billion laughs attack in your parser). Library upgrades change defaults, and a test notices.

Related reading

For the overview see XML security problems to close before production, and for the other languages safe XML handling in PHP and Node. The Java settings are listed in the OWASP XXE Prevention Cheat Sheet, the JDK's limits in the JAXP processing limits documentation, and the Python situation in the xml modules security notes. Test a path against a sample in the XPath tester.

When disabling DTDs breaks a legitimate document

The format uses a DOCTYPE. Apple property lists, some XHTML and older publishing formats carry a DOCTYPE with a public identifier. Rejecting every DTD refuses them. Allow the DOCTYPE with external access disabled, and keep the expansion limits.

The document needs entities defined in a DTD. Typesetting and legal XML sometimes defines named entities in an external DTD. Resolve them from a local, trusted copy through a catalog or an entity resolver, never from the network.

Validation needs the DTD. Validate with a schema (XSD) instead, or load a vetted local DTD with network access off.

You control the producer. If you also write the documents, remove the DOCTYPE at the source. It is cheaper than carrying an exception.