YAML
Environment Variable Substitution in YAML Config Files
Substitute environment variables into YAML config after parsing, not before. Expanding text first broke on a colon and hash; a typed walk kept the value.
6 min read
Library
Everything you need to level up your coding skills.
YAML
Reading YAML Front Matter From Markdown Files in Python
Split a Markdown file into YAML metadata and body with one regex, then handle dates, CRLF, a BOM, a missing closing line and body text with dashes.
6 min read
YAML
Merging Two YAML Files in Python With a Deep Merge
dict update is shallow and replaces whole blocks. A 10-line deep merge, the list policy choice and what null means, tested on a base and an override file.
6 min read
CSV
Round Tripping Exports Back Into Imports Without Drift
A CSV export should re-import to the same data. A test where identical files hid a NULL turned into an empty string, and how to test round trips on rows.
7 min read
YAML
Validating YAML Configuration With pydantic Models
Load YAML, validate it with pydantic models and get every error with its path. The Norway problem caught, extra keys rejected and ranges enforced.
7 min read
YAML
Loading Multiple Documents From One YAML File in Python
A YAML file can hold several documents separated by three dashes. safe_load fails on them, safe_load_all reads them and safe_dump_all writes them.
6 min read
YAML
Preserving Comments and Order With ruamel yaml Round Trips
Edit a YAML file from Python and keep its comments, key order, quotes and block strings with ruamel.yaml round-trip mode, plus settings and caveats.
7 min read
XML
Parallel Processing of Split XML Files With Python Pool
Split a 127 MB XML file into four parts in 3.4 seconds, then process them in parallel for a 3.8 times speedup. Code, results and the limits.
6 min read
CSV
Choosing Column Names Users Recognize in CSV Imports
Name CSV columns in the user's words, keep a stable internal key and accept aliases on import. A mapping function tested on three realistic headers.
7 min read
CSV
Background Jobs for Large CSV Exports That Do Not Time Out
Move big CSV exports into a job: chunk by key, write to storage, send an expiring link, track progress and clean up. Keyset chunks beat OFFSET.
8 min read
CSV
Streaming a CSV Export From a Web App Without Memory Spikes
Stream a CSV download instead of building it in memory: one million rows took 336 MB as an array and 2 MB streamed. Headers, buffering and limits.
7 min read
CSV
Reporting Row Level Errors Back to the User of a CSV Import
Report CSV import errors users can act on: line, column, value and rule, all problems per row, and a file of rejected rows to fix and re-upload.
8 min read
Get new articles by email
We send one email when a new article is published. No spam.