XML
Parsing a Large Wikipedia Style XML Dump From bz2 Directly
Stream a compressed MediaWiki style export with iterparse, skip redirects, count links and text, and keep memory near 24 MB while reading 176 MB of XML.
6 min read
Tag
how-to
590 posts
CSV
Opening a CSV File That Is Too Large for Excel or Sheets
A worksheet holds 1,048,576 rows. When a CSV has more, filter, aggregate or split it first with DuckDB, split or csvkit, and hand Excel a file that fits.
6 min read
CSV
Converting CSV to Markdown Tables for Documentation
pandas, Miller and csvlook turn a CSV into a Markdown table, and each breaks on a pipe or a newline. pandas wrote 12.50 as 12.5 and csvlook 02134 as 2,134.
9 min read
JSON
Using JSON Schema Enums Formats and Patterns Correctly
Where Python and Node disagree on JSON Schema formats and patterns: email, uri, hostname, digits, trailing newlines, plus exact enum and const behavior.
6 min read
XML
Parsing Tax Style E Invoices and Checking the Totals
Read a UBL style invoice with namespaced lines, recompute each line, sum the net amounts and compare the tax rounded per line with the tax per rate group.
6 min read
Excel
Date Formulas for Month Ends and Business Days in Excel
Excel date formulas for month starts and ends, adding months, business days with holidays, fiscal years and age, with the edge cases that break them.
7 min read
CSV
Converting CSV to SQL INSERT Statements the Safe Way
Generate INSERT statements from a CSV without breaking on O'Neil or running a DROP TABLE hidden in a value. Batches of 1,000 loaded 193,708 rows in 1.6 s.
13 min read
Excel
LET and LAMBDA for Readable Formulas in Excel Workbooks
Use LET to name values and LAMBDA to define reusable functions so long Excel formulas become readable, faster and easier to review, with portability notes.
6 min read
CSV
Converting JSON to CSV When the Data Is Nested With Arrays
Flatten nested JSON to CSV with pandas, Python or DuckDB: dotted columns, a row per array item, the order that vanishes and arrays written as Python text.
10 min read
XML
Reading a Medical Style HL7 CDA Document Safely With lxml
Extract the patient id and measurements from a synthetic CDA clinical document with a namespace map, handle nullFlavor and disable entity resolution.
6 min read
JSON
Validating Arrays and Nested Objects With JSON Schema
Validate arrays of objects, tuples and deep nesting with JSON Schema: items, prefixItems, contains, uniqueItems and error paths, tested in Python.
6 min read
PostgreSQL
Tuning PostgreSQL for SSDs With random_page_cost and More
Tune PostgreSQL for SSDs: lower random_page_cost, set effective_io_concurrency, check changed plans, use per tablespace values and keep scans that win.
7 min read