CSV
Using awk on CSV Files and Where It Gets Dangerous
awk -F, sums and groups a CSV in one pass when nothing is quoted. See the cases where it silently returns wrong fields, and what FPAT does and does not fix.
8 min read
Tag
how-to
590 posts
XML
Debugging an XSLT Transform That Outputs Nothing or Too Much
Five stylesheets that output nothing or the whole file as text, with the cause of each: a namespace, a built-in rule, an absolute path, and a literal id.
7 min read
YAML
Loading YAML Seed Data Into PostgreSQL With Python
Load a YAML list into PostgreSQL 18 with psycopg: adapt dates, arrays and jsonb, rerun safely with ON CONFLICT and let constraints reject bad rows.
6 min read
Excel
Converting a Huge Excel File to CSV or Parquet Safely
Convert a very large Excel workbook to CSV or Parquet with a streaming script, DuckDB or LibreOffice, keeping types, dates, encoding and every sheet right.
8 min read
JSON
The SQLite Arrow Operators for JSON Explained With Examples
SQLite's -> and ->> operators read JSON like PostgreSQL and MySQL, with their own rules for bare keys, negative indexes, precedence and the types they return.
7 min read
CSV
Previewing a CSV in the Terminal With column and less
Look at a CSV in the terminal with head, column -t, less -S, csvlook, qsv table and Miller, and learn where each one misleads you on quotes and encodings.
8 min read
XMLGuide
XSLT and XQuery for Real Transformations Step by Step
A working guide to XSLT 1.0, 3.0 and XQuery on the same files: templates, grouping, enrichment, JSON output, parameters, debugging, speed and which to pick.
19 min read
PostgreSQL
How to Read a PostgreSQL Query Plan Step by Step Guide
Read a PostgreSQL query plan line by line: tree shape, estimate and actual groups, loops, own time per node, and a worked plan with one slow step.
7 min read
Editor's pick
CSVGuide
Working With CSV Files in the Shell Without Opening Excel
A practical guide to CSV in the shell: coreutils, awk, csvkit, Miller, qsv and DuckDB compared on 2,000,000 rows, with the quoting rule and a triage script.
22 min read
JSON
Querying JSON Text With json_extract in SQLite Step by Step
What json_extract returns for each JSON type in SQLite, how paths and multiple paths behave, the number and text traps, and what repeated calls cost.
7 min read
CSV
Python CSV Newlines and the Blank Line Bug on Windows
Why a CSV written by Python shows a blank line after every row on Windows, how newline='' prevents it, and how to check the bytes. Reproduced without Windows.
8 min read
YAMLGuide
Using YAML With Databases for Seeds Exports and Queries
Five jobs YAML does next to a database: seed data, exports, stored documents, test fixtures and queries, with tested code for PostgreSQL, SQLite and DuckDB.
19 min read