XMLGuide
Choosing XML Over the Alternatives or Not for a New Project
A decision guide for XML versus JSON, YAML, CSV and Protocol Buffers, with measured sizes and parse times, six scenarios and a migration path.
16 min read
Topic
XML
Older than half the tools that still emit it. Namespaces, XPath and getting rows out of feeds and exports.
Excel
Converting Google Sheets to CSV or Excel With a Script
Export a Google Sheet to CSV or xlsx from a script using the export URL, the Drive API or gspread, choose the sheet by gid and handle sharing and auth safely.
6 min read
XML
Indexing XML Columns in PostgreSQL Without Wrecking Inserts
An expression index on xpath turned a 245 ms scan into 1 ms on 50,000 documents and made loading 2.8 times slower. A generated column does the same.
6 min read
Excel
Converting Excel to Markdown and HTML Tables for Docs
Turn a worksheet into a Markdown or HTML table for documentation with pandas, escape pipes and line breaks, align numbers and keep wide tables readable.
6 min read
XML
Streaming XML Into PostgreSQL Without a DOM Using COPY
Parse a 127 MB XML file with iterparse and feed two tables through COPY. 400,000 orders loaded in 6.5 seconds and 44 MB, against 38 seconds for row inserts.
6 min read
Excel
Converting Excel to SQL Insert Statements Safely and Fast
Turn a worksheet into SQL INSERT statements that quote text, handle NULLs and dates, and load in batches, or skip the script and use COPY or a driver.
6 min read
XML
Benchmarking ElementTree lxml and iterparse on 127 MB
Four ways to sum 1.2 million order lines from a 127 MB XML file, with time and peak memory, plus the clearing pattern that keeps streaming flat.
6 min read
Excel
Converting Excel to Parquet for Faster Analytics Loads
Convert an Excel workbook to Parquet with pandas or DuckDB, fix column types first, write one file per sheet and check counts so analytics loads run fast.
7 min read
Excel
Converting Excel to YAML Config Files Without Surprises
Turn an Excel sheet of settings or tables into YAML in Python: shape the data, avoid the Norway problem and number traps, dump safely and validate.
7 min read
XML
How Much Memory a DOM Parser Needs for a Large XML File
Measured peak memory of ElementTree and lxml on 32 MB and 127 MB order files shows 11 to 15 times the file size, and how streaming cut it to 22 MB.
6 min read
Excel
Converting Excel to XML With a Python Script Safely
Convert an Excel sheet to XML in Python: valid element names, escaping, elements versus attributes, encoding, large files and validation against a schema.
7 min read
XMLGuide
Parsing Big XML Files Fast Without Running Out of Memory
A measured guide to large XML files: memory multipliers, streaming, library choice, COPY into PostgreSQL, parallel splits, compression and indexes.
17 min read
Get new articles by email
We send one email when a new article is published. No spam.