JSON
Converting YAML to JSON for Tools That Need It Safely
Convert YAML to JSON with PyYAML and ruamel.yaml, and handle dates, yes and no, octal numbers, non-string keys, anchors, multiple documents and floats.
8 min read
Tag
how-to
590 posts
JSON
Converting JSON to YAML on the Command Line Safely
Convert JSON to YAML with yq, Python's yq and a PyYAML one-liner, and see which strings get corrupted by YAML 1.1 readers: NO, yes, 12:30, 0123 and 1e3.
7 min read
CSV
Choosing Column Names Users Recognize in CSV Imports
Name CSV columns in the user's words, keep a stable internal key and accept aliases on import. A mapping function tested on three realistic headers.
7 min read
CSV
Background Jobs for Large CSV Exports That Do Not Time Out
Move big CSV exports into a job: chunk by key, write to storage, send an expiring link, track progress and clean up. Keyset chunks beat OFFSET.
8 min read
CSV
Streaming a CSV Export From a Web App Without Memory Spikes
Stream a CSV download instead of building it in memory: one million rows took 336 MB as an array and 2 MB streamed. Headers, buffering and limits.
7 min read
CSV
Reporting Row Level Errors Back to the User of a CSV Import
Report CSV import errors users can act on: line, column, value and rule, all problems per row, and a file of rejected rows to fix and re-upload.
8 min read
CSV
Previewing a CSV Upload Before Committing Any Changes
Preview a CSV import safely: inspect the file, then run the real import in a transaction that rolls back to count creates, updates and errors.
7 min read
CSV
Designing a CSV Import Screen Users Can Succeed With
What a CSV import screen needs so users succeed: a template, clear limits, column mapping, a preview, plain-language errors and a count before commit.
7 min read
CSVGuide
CSV Import and Export Features in Applications Done Right
Build CSV import and export into an application: staged imports with dry runs, streamed and background exports, column names, round trips and safety.
19 min read
CSV
A Survey Export With Hundreds of Columns Made Usable
A survey CSV has one column per question, 300 wide. Unpivot it to long form with DuckDB, drop blanks, name the questions and check the answer count.
6 min read
CSV
A Government Dataset With Footnotes in the Data Cells
Statistical CSVs put flags in value cells: 1,204,300 (p), 5.3 e, and dots for missing. Split number from flag, keep both and check against the stated total.
7 min read
CSV
A Log Export With Unescaped Quotes and How to Read It
A log export has quotes inside quoted fields without doubling them. See what Python and DuckDB return, which setting reads them right and how to prove it.
7 min read