Library

Everything you need to level up your coding skills.

XML

Reading Publisher Metadata in JATS XML for One Article

Pull the title with inline markup, authors with ORCID and affiliations, group authors, editors and the several dates out of one JATS article front matter.

7 min read

XML

Flattening a Product Catalog Feed With Repeating Fields

A shopping feed with a namespace, prices written as 12.50 EUR, optional sale prices and repeated shipping and image fields, flattened into three clean tables.

7 min read

XML

Joining XML Derived Tables to Your Own Data Safely

A join between parsed XML orders and a customer CSV silently dropped 1,280 lines and 3.2 million in revenue. Find the unmatched keys and normalize them.

6 min read

XML

Storing Raw XML in a Data Lake and Parsing It Later

Land source XML unchanged, compressed, partitioned by date and described by a manifest with a checksum, so parsing can be rerun when the rules change.

6 min read

XML

Using DuckDB to Read XML Through Python and Arrow Tables

Stream XML into Arrow batches with lxml, query them in DuckDB at once, or try the community webbed extension. 1.2 million lines in 7.9 s with a 53 ms query.

6 min read

XML

When XML Is Still the Right Format and When It Is Not

Documents with inline markup, schemas that partners must honor, signatures and long-lived standards keep XML in use. A checklist of five situations.

6 min read

XML

XML Versus Protocol Buffers for Services and Messages

50,000 orders in four formats: protobuf was 5.1 MB, JSON 10.8 MB, XML 14.2 MB, and it parsed in 9 ms against 161 for XML. What that buys and costs.

5 min read

XML

Compressing XML Archives and Reading Them In Place

A 127 MB XML file shrank to 14.8 MB with gzip, 15.4 MB with zstd and 10.8 MB with xz. Costs in seconds, and how to parse a gzip file without unpacking.

6 min read

XML

Parallel Processing of Split XML Files With Python Pool

Split a 127 MB XML file into four parts in 3.4 seconds, then process them in parallel for a 3.8 times speedup. Code, results and the limits.

6 min read

JSON

Backfilling Historic Events After a Schema Change Safely

Three ways to rewrite a renamed JSON property across 1.5 million events in PostgreSQL: one UPDATE, batches and a table swap, with WAL, bloat and leftovers.

7 min read

JSON

Handling Properties That Change Type in JSON Event Data

Fix invalid input syntax for type numeric on a JSON property that became a string, null or object, with jsonb_typeof, pg_input_is_valid and a function.

7 min read

JSON

Indexing Hot JSON Properties for Dashboards in PostgreSQL

Expression, partial and GIN indexes on a 2.25 million row jsonb event table: a selective lookup fell from 65 ms to 5 ms, a 4 percent filter got slower.

6 min read

Get new articles by email

We send one email when a new article is published. No spam.