JSON
Extracting Properties Into Typed Columns Nightly From JSON
A nightly job that turns jsonb event properties into a typed facts table in PostgreSQL: safe casts, a watermark, idempotent inserts and a 47x speedup.
7 min read
Tag
automation
40 posts
CSV
Background Jobs for Large CSV Exports That Do Not Time Out
Move big CSV exports into a job: chunk by key, write to storage, send an expiring link, track progress and clean up. Keyset chunks beat OFFSET.
8 min read
CSV
Previewing a CSV Upload Before Committing Any Changes
Preview a CSV import safely: inspect the file, then run the real import in a transaction that rolls back to count creates, updates and errors.
7 min read
CSV
Designing a CSV Import Screen Users Can Succeed With
What a CSV import screen needs so users succeed: a template, clear limits, column mapping, a preview, plain-language errors and a count before commit.
7 min read
CSVGuide
CSV Import and Export Features in Applications Done Right
Build CSV import and export into an application: staged imports with dry runs, streamed and background exports, column names, round trips and safety.
19 min read
XML
Detecting Structure Changes in an XML Feed Over Time
A feed renamed price to cost and the loader kept running with empty columns. A short script compares element path snapshots and reports the change.
6 min read
XML
Checking Feed Health With a Daily Script and Clear Alerts
A feed health check that tests the fetch, status, content type, well-formedness, item count and freshness, run on eight scenarios with its output.
6 min read
Excel
Testing That a Generated Excel Report Is Actually Correct
Test a generated Excel report with pytest: structure, control totals, formulas, formats and edge cases, plus a fixture of bad input and a golden file.
8 min read
JSON
Using JSON in Bash Scripts Safely With jq Arguments
Build and read JSON in Bash without breaking: jq --arg instead of string concatenation, newlines in values, exit codes, null as text and filter injection.
7 min read
Excel
Protecting a Generated Workbook From Accidental Edits
Lock the cells, sheets and structure of a workbook your script generates so readers cannot break formulas, with openpyxl and XlsxWriter, and know the limits.
7 min read
Excel
Watching a Folder for New Excel Files and Importing Them
Watch a folder for new Excel files and import each once: polling or watchdog, waiting for the copy to finish, moving files, idempotent loads and failures.
7 min read
Excel
Batch Converting XLS Files to XLSX With LibreOffice
Convert a folder of old XLS files to XLSX with LibreOffice headless, Python or Excel automation, and check what formatting and formulas survive.
6 min read