Library

Everything you need to level up your coding skills.

JSON

Cleaning a Twitter Style Archive File That Is Not Quite JSON

Clean a social media archive that is JavaScript, not JSON: strip the assignment, join parts, fix string numbers and dates, unescape HTML and write Parquet.

6 min read

JSON

Flattening a Shopify Style Order Export Into Tables

Flatten a Shopify style export of 20,000 orders into orders and line items with pandas and DuckDB: money as strings, null customers, tags and zones.

7 min read

Flat illustration of a database beside the table it holds, with the DuckDB and GitHub logos.

JSON

Analyzing a GitHub Events Dump With DuckDB and SQL

Query a GitHub style events dump of 300,000 JSON lines in DuckDB: one struct for ten event types, truncated commit lists and three-state booleans.

7 min read

JSON

Normalizing a Stripe Style Webhook Payload in DuckDB

Turn a Stripe style webhook log into tables with DuckDB: dedupe retries, zero-decimal currencies, invoice lines and the metadata map that hides gaps.

7 min read

JSON

Fixtures in JSON for Database Tests With PostgreSQL

Load JSON fixtures into PostgreSQL for tests with jsonb_populate_recordset: 25,000 rows in 0.21 s, bad fixtures caught by constraints, one typo missed.

6 min read

JSON

Contract Testing JSON APIs With an OpenAPI Schema Tool

Test a JSON API against its OpenAPI contract with Schemathesis: 1,903 generated requests found a 500 and undocumented statuses, and missed one bug.

6 min read

JSON

Mock Data Generators for JSON Tests Demos and Load

Generate JSON test data with Faker, Hypothesis, SQL and jq: seeds for repeatable runs, collisions, and a property test that found real bugs.

6 min read

JSON

Comparing JSON Ignoring Volatile Fields in Python Tests

Compare JSON documents in tests while ignoring ids and timestamps: strip keys, DeepDiff exclusions, placeholders that keep relations, jq del, float tolerance.

6 min read

JSON

Snapshot Testing JSON Output Without Flaky Failures

Snapshot tests for JSON in pytest with syrupy and Node's test runner: what a snapshot stores, why random ids break it, and matchers and scrubbing that fix it.

6 min read

PostgreSQL

Exporting JSON Lines From a Query in PostgreSQL Safely

Export JSON Lines from a PostgreSQL query with row_to_json or to_jsonb and COPY TO STDOUT, handle quoting, types, nulls, large results and compression.

6 min read

YAML

Round Tripping YAML Without Losing Comments or Order

Edit a YAML file in code and keep its comments, anchors and quotes. PyYAML drops them, ruamel.yaml and yq keep them, and each changes some whitespace.

6 min read

JSON

Using JSON in Bash Scripts Safely With jq Arguments

Build and read JSON in Bash without breaking: jq --arg instead of string concatenation, newlines in values, exit codes, null as text and filter injection.

7 min read

Get new articles by email

We send one email when a new article is published. No spam.