CSV
Converting CSV to JSON Lines for Streaming Pipelines
Convert CSV to JSON Lines, one object per line: a 13 MB streaming script, DuckDB in 0.3 s, files that split cleanly and a cut-off file that shows itself.
9 min read
Library
Everything you need to level up your coding skills.
CSV
Converting CSV to JSON With Python Without Losing Types
Convert CSV to JSON with the csv module, then decide on types: zip stays text, money stays exact. pandas gave 2134.0 for 02134; converters took 0.4 to 36 s.
9 min read
JSONGuide
Validating JSON With JSON Schema in Real Team Projects
A working guide to JSON Schema: write a schema, run it in Python and Node, handle formats, refs and versions, and check JSON stored in a database.
19 min read
CSVGuide
Converting CSV to JSON XML YAML and Excel in Practice
Convert one awkward CSV to JSON, JSON Lines, XML, YAML, Excel, Parquet, Markdown and SQL, read each back and see the cost on 193,708 rows.
22 min read
JSON
Reading JSON Lines With pandas Without the Usual Surprises
Read JSON Lines into a pandas DataFrame with read_json(lines=True): dtypes, chunking, nested fields, big integers, bad lines and the pyarrow engine.
6 min read
JSON
Compressing JSON Lines for Storage and Reading Them Back
Compare gzip, zstd and xz on a 36 MB JSON Lines file for size and speed, and read the result back with jq, DuckDB, pandas and Python without unpacking.
6 min read
JSON
Splitting a JSON Lines File Into Chunks Without Breakage
Split a JSON Lines file by line count, size or key with split and awk, keep every record whole, gzip chunks on the fly and process them in parallel.
7 min read
JSON
JSON Lines Versus CSV for Event Data and Log Exports
JSON Lines or CSV for event exports? Measured file size, compressed size and read time on 200,000 events, plus nesting, sparse fields and null handling.
6 min read
JSON
Finding One Bad Line in a Huge JSON Lines File Fast
Locate the malformed record in a large JSON Lines file with jq, Python, awk and DuckDB, why parser line numbers drift, and how to split good from bad.
7 min read
JSON
Streaming JSON Lines Into PostgreSQL Using COPY and psql
Load a JSON Lines file into a PostgreSQL jsonb column with COPY. The backslash trap in text format, the csv-format fix, bad-line errors and timings.
8 min read
JSONGuide
Using jq to Slice Up JSON in the Terminal Reliably
A tested guide to jq 1.8 for slicing JSON in the terminal: filters, streams, CSV, grouping, logs, safe variables, in-place edits and number quirks.
17 min read
JSON
JSON parse and stringify Options You Probably Missed
The extra arguments of JSON.stringify and JSON.parse, tested in Node 24: toJSON, replacer arrays, indent limits, call order and exact numbers.
9 min read
Get new articles by email
We send one email when a new article is published. No spam.