YAML
Reading YAML Anchors in CI Files and Merge Key Gotchas
How to expand anchors and merge keys in CI files to see the effective job, and the yq 4.54.1 merge-key order bug that gave retries 2 where the spec says 5.
6 min read
Tag
how-to
590 posts
CSV
Writing a Schema for a CSV With Frictionless Data in Python
Infer a Table Schema, fix it by hand, add unique, enum and pattern constraints and validate with Frictionless: 10 errors for 8 faults in 193,708 rows.
11 min read
Excel
Converting Google Sheets to CSV or Excel With a Script
Export a Google Sheet to CSV or xlsx from a script using the export URL, the Drive API or gspread, choose the sheet by gid and handle sharing and auth safely.
6 min read
CSV
Checking Row and Column Counts of a CSV Before Loading
How many rows does this CSV have? wc -l, awk, csv, pandas and DuckDB gave 7, 8, 7, an error, 4 and 8 on one small file. Count records and field widths.
10 min read
Excel
Converting Excel to Markdown and HTML Tables for Docs
Turn a worksheet into a Markdown or HTML table for documentation with pandas, escape pipes and line breaks, align numbers and keep wide tables readable.
6 min read
CSVGuide
How to Validate a CSV File Before You Trust the Data
Validate a CSV delivery in five layers, bytes, structure, schema, values and keys, with line numbers and exit codes: 193,708 rows checked in about a second.
21 min read
XML
Streaming XML Into PostgreSQL Without a DOM Using COPY
Parse a 127 MB XML file with iterparse and feed two tables through COPY. 400,000 orders loaded in 6.5 seconds and 44 MB, against 38 seconds for row inserts.
6 min read
YAML
Helm Values Files Without Copy Paste Using Layered Values
Layer Helm values so production overrides only what differs: maps merge, lists replace, --set wins. Rendered with helm template and checked with kubeconform.
7 min read
Excel
Converting Excel to SQL Insert Statements Safely and Fast
Turn a worksheet into SQL INSERT statements that quote text, handle NULLs and dates, and load in batches, or skip the script and use COPY or a driver.
6 min read
CSV
Finding Rows That Do Not Match the Expected Pattern in a CSV
Check every column of a CSV against a regex and report failing rows with file line numbers. fullmatch beats match, and 2025-13-45 passes a date pattern.
9 min read
Excel
Converting Excel to Parquet for Faster Analytics Loads
Convert an Excel workbook to Parquet with pandas or DuckDB, fix column types first, write one file per sheet and check counts so analytics loads run fast.
7 min read
JSON
Loading JSON Into a pandas DataFrame the Right Way
pandas read_json turned the zip "01234" into 1234. How orient, lines, dtype and the pyarrow backend change the result, and what changed in pandas 3.
6 min read