Topic

XML

Older than half the tools that still emit it. Namespaces, XPath and getting rows out of feeds and exports.

XML

Well Formed Versus Valid XML and the Difference in Practice

Well formed means the syntax parses, valid means it follows a DTD or XSD. Ten broken files and two schemas run through xmllint show what each catches.

6 min read

XMLGuide

What XML Is Good At and Why It Still Shows Up in Your Data

A tested beginner guide to XML: where you will meet it in data work, what it does better than JSON, how it is structured, and the first checks to run.

17 min read

XML

Reading Publisher Metadata in JATS XML for One Article

Pull the title with inline markup, authors with ORCID and affiliations, group authors, editors and the several dates out of one JATS article front matter.

7 min read

XML

Flattening a Product Catalog Feed With Repeating Fields

A shopping feed with a namespace, prices written as 12.50 EUR, optional sale prices and repeated shipping and image fields, flattened into three clean tables.

7 min read

XML

Joining XML Derived Tables to Your Own Data Safely

A join between parsed XML orders and a customer CSV silently dropped 1,280 lines and 3.2 million in revenue. Find the unmatched keys and normalize them.

6 min read

XML

Storing Raw XML in a Data Lake and Parsing It Later

Land source XML unchanged, compressed, partitioned by date and described by a manifest with a checksum, so parsing can be rerun when the rules change.

6 min read

XML

Using DuckDB to Read XML Through Python and Arrow Tables

Stream XML into Arrow batches with lxml, query them in DuckDB at once, or try the community webbed extension. 1.2 million lines in 7.9 s with a 53 ms query.

6 min read

XML

When XML Is Still the Right Format and When It Is Not

Documents with inline markup, schemas that partners must honor, signatures and long-lived standards keep XML in use. A checklist of five situations.

6 min read

XML

XML Versus Protocol Buffers for Services and Messages

50,000 orders in four formats: protobuf was 5.1 MB, JSON 10.8 MB, XML 14.2 MB, and it parsed in 9 ms against 161 for XML. What that buys and costs.

5 min read

XML

Compressing XML Archives and Reading Them In Place

A 127 MB XML file shrank to 14.8 MB with gzip, 15.4 MB with zstd and 10.8 MB with xz. Costs in seconds, and how to parse a gzip file without unpacking.

6 min read

XML

Parallel Processing of Split XML Files With Python Pool

Split a 127 MB XML file into four parts in 3.4 seconds, then process them in parallel for a 3.8 times speedup. Code, results and the limits.

6 min read

PostgreSQL

Reading XML With xmltable in a Real Import in PostgreSQL

Import XML into PostgreSQL with xmltable: XPath row and column paths, namespaces, attributes, nested elements, large files and validation of a feed.

6 min read

Get new articles by email

We send one email when a new article is published. No spam.