Reading Financial and Government XML Datasets With Python

Governments and central banks publish a lot of data as XML: exchange rates, statistics, company filings, legislation, open data lists. The files tend to share a few habits. They use namespaces, because they come from standards bodies. They put values in attributes. They are published at a stable URL and updated on a schedule. And the documentation is a PDF that is slightly behind the file.

I will read one real example from start to finish, the European Central Bank's daily euro foreign exchange reference rates, because it is public, small, stable and typical of the genre.

What the file looks like

The top of the daily file, as downloaded:

<?xml version="1.0" encoding="UTF-8"?>
<gesmes:Envelope xmlns:gesmes="http://www.gesmes.org/xml/2002-08-01" xmlns="http://www.ecb.int/vocabulary/2002-08-01/eurofxref">
	<gesmes:subject>Reference rates</gesmes:subject>
	<gesmes:Sender>
		<gesmes:name>European Central Bank</gesmes:name>
	</gesmes:Sender>
	<Cube>
		<Cube time='2026-10-26'>
			<Cube currency='USD' rate='1.1188'/>
			<Cube currency='JPY' rate='176.15'/>
			<Cube currency='CZK' rate='24.369'/>
			...

Look at the structure before you write any code. There are two namespaces. The wrapper (Envelope, subject, Sender, name) belongs to the gesmes namespace, from a messaging standard. The data (Cube) is in the default namespace, http://www.ecb.int/vocabulary/2002-08-01/eurofxref. There are **three levels of Cube**, with different roles: the outer one is a container, the middle one carries the date as a time attribute, and the inner ones carry a currency and a rate as attributes and have no content. Every number is an attribute value, so it comes out as text.

Reading it

from decimal import Decimal
from lxml import etree

NS = {"e": "http://www.ecb.int/vocabulary/2002-08-01/eurofxref",
      "g": "http://www.gesmes.org/xml/2002-08-01"}
r = etree.parse("ecb.xml").getroot()
day = r.find("e:Cube/e:Cube", NS)
print(day.get("time"), len(day), "currencies")
rates = {c.get("currency"): c.get("rate") for c in day}
print({k: rates[k] for k in list(rates)[:4]})
print("no ns:", len(r.findall("Cube/Cube/Cube")))
print("subject:", r.findtext("g:subject", namespaces=NS), "| sender:", r.findtext("g:Sender/g:name", namespaces=NS))

2026-10-26 29 currencies
{'USD': '1.1188', 'JPY': '176.15', 'CZK': '24.369', 'DKK': '7.4747'}
no ns: 0
subject: Reference rates | sender: European Central Bank

The date is 26 October 2026, the file had 29 currencies and the four I printed are the first four. The line no ns: 0 is the lesson of the namespace once more: the unprefixed path Cube/Cube/Cube found nothing at all, while the namespaced one found the data. The prefixes in my dictionary are mine (e and g); the document's own are gesmes and the empty default, and only the URIs count. The same data from the shell, without a prefix map, is possible with local-name():

xmllint --xpath 'count(//*[local-name()="Cube"][@currency])' ecb.xml
xmllint --xpath 'string(//*[local-name()="Cube"][@currency="JPY"]/@rate)' ecb.xml

29
176.15

29 currencies and a yen rate of 176.15, the same as Python read. The local-name() form ignores the namespace and is fine for a one-off, but it would match a Cube in any namespace, so I would not use it in code that must be exact.

Reading the rate correctly

A rate file is easy to read and easy to misread. The first check is the direction of the quote. In this file, the rate is the amount of the currency per one euro: USD 1.1188 means one euro buys 1.1188 US dollars. So 100 dollars are 100 divided by 1.1188 euros:

rates = {c.get("currency"): Decimal(c.get("rate")) for c in day}
print("100 USD in EUR", (Decimal(100) / rates["USD"]).quantize(Decimal("0.01")))

100 USD in EUR 89.38

Multiplying instead of dividing would give 111.88, a sign error that is invisible in the output because both numbers look like plausible amounts. Always do a sanity check against a number you know: the euro is worth more than a dollar in this file's date, so 100 dollars should be fewer than 100 euros. 89.38 passes that check.

The second check is precision. The rates have four to five significant digits and come as text, so parse them as Decimal, as above, and keep as many digits as the source gave: 0.83820 for sterling has a trailing zero that a float would drop. Round at the end, for display, and say how (the ECB does not state a rounding rule for conversions in the file; its published conventions are on its site).

The third check is what the number is. These are reference rates, published once per business day at a fixed time of day, not tradable prices. They are suitable for statistics, reporting and approximate conversion. Whether they are suitable for an invoice or a ledger depends on your accounting rules.

The history file

The ECB also publishes the last 90 days in one file with the same structure, one dated Cube per business day. The same code reads it:

import collections, statistics
days = r.findall("e:Cube/e:Cube", NS)
print(len(days), "days", days[-1].get("time"), "to", days[0].get("time"), "| cubes/day", collections.Counter(len(d) for d in days).most_common(3))
usd = [(d.get("time"), d.find('e:Cube[@currency="USD"]', NS).get("rate")) for d in days]
print(usd[:3], usd[-1])
v = [float(x[1]) for x in usd]
print("min", min(v), "max", max(v), "mean", round(statistics.mean(v), 4))

64 days 2026-07-29 to 2026-10-26 | cubes/day [(29, 64)]
[('2026-10-26', '1.1188'), ('2026-10-23', '1.1193'), ('2026-10-22', '1.1205')] ('2026-07-29', '1.138')
min 1.1177 max 1.1699 mean 1.1452

64 dated entries for a window of 90 calendar days, because the ECB publishes on business days (about 64 of 90). Every day had exactly 29 currencies in this window, so a table of date, currency and rate has 64 × 29 = 1,856 rows, with no gaps. The newest entry comes first in the file, so "the last element" is the oldest day. If you join these rates to your transactions by date, decide what a Saturday transaction uses (the previous Friday is the usual rule) and write it down. Currencies do get added and dropped over time, and a longer history shows days with fewer currencies, so check the per-day count before you assume a rectangle.

Other public data in XML

The same habits turn up elsewhere, and so do the same fixes. Company filings and statistical releases may use XBRL or SDMX, vocabularies with many namespaces, contexts and units where the number's meaning sits in other elements (a unitRef for the currency, a contextRef for the period). Government open data portals often serve lists of data sets as RSS, Atom or custom XML (reading government open data in XML). Legislation is typically document-like XML with deep nesting, where the analysis is about text and structure and not about rows. For each, I do the same steps: fetch the file once and keep it; read the top of it by eye; list its namespaces; profile the path counts (counting elements and attributes in an unknown XML file); then write the reader.

A habit worth keeping: save the downloaded file with the date in its name, so a figure you quoted last month can be reproduced. The URL for "today's" file serves a different file tomorrow.

Citing and refreshing

When the output of this reading goes into a report, state the source file, its date and the quote direction. When it goes into a pipeline, fetch with a timeout, check the HTTP status, parse, check that the date is the expected business day and the currency count is the expected number, and only then replace the table. A fetch that returns an HTML error page, or a file with 27 currencies instead of 29, should stop the job and leave the old table in place. Fetch politely: once a day is the publication rate, and a script that hits the URL every minute adds nothing.

Related reading

For the namespace mechanics see how XML namespaces work and selecting elements in a namespace with XPath. The feed side of public data is in working with XML feeds and APIs, the incremental side in incremental XML feed imports by last modified, and the cluster overview in analysis on XML data when nobody has converted it yet. The rates and their conventions are documented on the ECB euro foreign exchange reference rates page, and the file format follows the Namespaces in XML rules. Try the paths in the XPath tester.

When the dataset has a better format

A CSV or an API is offered. Many publishers offer the same data as CSV or JSON, and a flat file is easier when you only need a table. Use the XML when it is the complete version, or the only one that has the metadata.

The data set is large and structured as a time series. Statistical agencies often offer a data API that returns exactly the series you ask for. Download one series, not the archive.

You need an authoritative value. For legal or accounting use, take the figure from the publisher's official document, with its date, and not from a parse of a file that may be revised.

The vocabulary is a heavy standard. Reading XBRL with a general XML parser is possible and painful. Use a library built for the standard.