JSONGuide
Using jq to Slice Up JSON in the Terminal Reliably
A tested guide to jq 1.8 for slicing JSON in the terminal: filters, streams, CSV, grouping, logs, safe variables, in-place edits and number quirks.
17 min read
Library
Everything you need to level up your coding skills.
JSON
Pretty Printing JSON in Python and From the Terminal
Pretty print JSON with json.dumps(indent), python -m json.tool and pprint, keep non-ASCII readable, sort keys and collapse short arrays.
6 min read
XMLGuide
Comparing and Versioning XML Files in Git and in Pipelines
A tested guide to comparing XML, from canonical form and keyed record diffs to Git diffs, merges, feed shape checks and schema compatibility.
18 min read
CSV
Splitting a CSV by Column Value Into Separate Files
Split a CSV into one file per column value with awk, qsv, Miller or DuckDB, keep the header in every file, check the counts and avoid unsafe file names.
9 min read
CSV
Removing Duplicate Rows From a CSV With Shell Tools
Remove duplicate CSV rows with sort -u, awk, qsv and Miller. See what each does to the header and the order, which row it keeps, and why awk drops a good row.
9 min read
CSV
Sorting a CSV File by Column in the Terminal Correctly
Sort a CSV by column from the shell without losing the header or reading text as numbers. Compare sort, qsv, Miller, csvsort and DuckDB on 2,000,000 rows.
9 min read
CSV
Counting Rows and Unique Values in a CSV From the Shell
Count records and distinct values in a CSV from the shell with wc, sort, uniq, qsv and Miller, and see why wc -l and messy spellings give wrong counts.
7 min read
CSV
Using qsv for Fast CSV Statistics and Slicing of Big Files
qsv counts, profiles, searches, slices and sorts CSV files in fractions of a second. See count, stats, frequency, slice and index run on 2,000,000 rows.
9 min read
CSV
Filtering CSV Rows From the Command Line With Miller
Filter, sort, compute and summarize CSV from the shell with Miller's filter, put and then-chains, and see how it treats empty cells, leading zeros and sums.
8 min read
CSV
Getting Started With csvkit for Everyday CSV Tasks
csvkit gives you csvcut, csvgrep, csvlook, csvstat, csvsort and csvsql with real CSV parsing. See the everyday commands, inference traps and speed cost.
8 min read
CSV
Using awk on CSV Files and Where It Gets Dangerous
awk -F, sums and groups a CSV in one pass when nothing is quoted. See the cases where it silently returns wrong fields, and what FPAT does and does not fix.
8 min read
CSV
Selecting CSV Columns With cut and Why Quotes Break It
cut -d, -f picks CSV columns until a quoted comma shifts every field after it. See the failure, a guard that detects it, and CSV-aware replacements.
8 min read
Get new articles by email
We send one email when a new article is published. No spam.