
Doing Proper Analysis on Excel Data With SQL in Practice
A guide to analyzing Excel data with SQL: query workbooks in DuckDB, join, summarize, compare years, find gaps, build cohorts, reconcile and replace models.
Tag
102 posts

A guide to analyzing Excel data with SQL: query workbooks in DuckDB, join, summarize, compare years, find gaps, build cohorts, reconcile and replace models.

A working guide to XSLT 1.0, 3.0 and XQuery on the same files: templates, grouping, enrichment, JSON output, parameters, debugging, speed and which to pick.

A practical guide to CSV in the shell: coreutils, awk, csvkit, Miller, qsv and DuckDB compared on 2,000,000 rows, with the quoting rule and a triage script.

One XML file and one task in eight languages, with each output, the defaults that change your data, and how each treats absent, empty and bad values.

Five jobs YAML does next to a database: seed data, exports, stored documents, test fixtures and queries, with tested code for PostgreSQL, SQLite and DuckDB.

Diagnose an Excel workbook that is huge, slow to calculate or too large to read in code, then fix it by size, formulas, the Data Model or a different format.

The command-line tools that cover daily XML work: xmllint, xmlstarlet, xsltproc, xq and yq, with tested commands, timings, exit codes and the traps of each.

A tested guide to SQLite's JSON functions: reading, arrows, editing, json_each, aggregates, JSONB, expression indexes and the type traps, on 200,000 rows.

How to move recurring Excel work into scripts: choosing VBA, Python, Office Scripts or Power Query, scheduling, merging, splitting, protecting and testing.

What survives when YAML becomes JSON, CSV, XML or Excel and back, with a tested loss matrix, the type traps in each direction and a conversion checklist.

Read EXPLAIN ANALYZE output in PostgreSQL: plan tree, cost, rows and loops, scan and join nodes, BUFFERS, sorts, estimates and a way to find the slow step.

A working guide to CSV in Python: csv module, DictReader, pandas, polars and DuckDB measured on 2,000,000 rows, plus the encoding, quoting and type rules.