JSON
Handling a Weather API With Nested Arrays of Measurements
Weather APIs return columns as parallel arrays. Turn them into rows with pandas and DuckDB, handle nulls and unequal lengths, and keep units and zones.
6 min read
Library
Everything you need to level up your coding skills.
JSON
Cleaning a Twitter Style Archive File That Is Not Quite JSON
Clean a social media archive that is JavaScript, not JSON: strip the assignment, join parts, fix string numbers and dates, unescape HTML and write Parquet.
6 min read
JSON
Flattening a Shopify Style Order Export Into Tables
Flatten a Shopify style export of 20,000 orders into orders and line items with pandas and DuckDB: money as strings, null customers, tags and zones.
7 min read
JSON
Analyzing a GitHub Events Dump With DuckDB and SQL
Query a GitHub style events dump of 300,000 JSON lines in DuckDB: one struct for ten event types, truncated commit lists and three-state booleans.
7 min read
JSON
Normalizing a Stripe Style Webhook Payload in DuckDB
Turn a Stripe style webhook log into tables with DuckDB: dedupe retries, zero-decimal currencies, invoice lines and the metadata map that hides gaps.
7 min read
CSVGuide
Worked CSV Problems and the Patterns Behind Them Explained
Eight messy CSV files solved end to end with a check each: mixed dates, unquoted commas, multi-line notes, display formats, headers, bad quotes, footnotes.
18 min read
CSV
Loading CSV Into MongoDB With mongoimport and Types
Load a CSV into MongoDB with mongoimport: what its type guessing does to zip codes, how to declare types, skip blanks and upsert on a key, with real runs.
7 min read
CSV
Bulk Loading a CSV File Into DynamoDB Tables Properly
Two ways to load a CSV into DynamoDB: the S3 import that makes a table of string attributes, and a boto3 batch writer with Decimal numbers.
7 min read
CSV
Loading CSV Into Oracle With SQL Loader Control Files
Load a CSV into Oracle with SQL*Loader: a control file for quotes and dates, the bad and log files, exit codes, direct path and external tables.
7 min read
CSV
Loading CSV Into SQL Server With BULK INSERT Correctly
Load a CSV into SQL Server with BULK INSERT: FORMAT CSV for quoted fields, FIRSTROW, UTF-8, error files, server paths and permissions.
7 min read
CSV
Loading CSV Into ClickHouse From the Command Line Fast
Load CSV into ClickHouse with clickhouse-client and FORMAT CSVWithNames, tune delimiter, errors and dates, and check a file first with clickhouse-local.
7 min read
CSV
Loading CSV Into Snowflake and Handling Errors in COPY
Load CSV into Snowflake with a file format, a stage and COPY INTO, then read the errors: ON_ERROR, VALIDATION_MODE, VALIDATE and the 64 day skip rule.
7 min read
Get new articles by email
We send one email when a new article is published. No spam.