YAML
Converting YAML to CSV for Review in a Spreadsheet
Flatten a YAML list of service records into a CSV a reviewer can sort and filter, handling nested keys, list values, missing fields and Excel encoding.
6 min read
Topic
CSV
The format everyone exports and nobody specifies. Delimiters, quoting, encodings and loading it into a database intact.
CSV
Scheduling CSV Reports From SQL With cron Reliably
A cron job that writes a daily CSV report from SQL: wrapper script, atomic output, a lock against overlap and the environment traps that bite at 2 a.m.
7 min read
CSV
Logging Rejected Rows From a CSV Import for Review
Capture every row an import refuses, with its line, column and reason, in a form a person can fix and replay. A DuckDB example with real reject tables.
6 min read
CSV
Handling a CSV Provider Who Changes Columns Without Warning
A provider adds, renames or reorders columns and your load breaks or, worse, succeeds. A header check that names the change, aliases and a policy per case.
7 min read
CSV
Seeding a Database From CSV Files With dbt Seeds Safely
Load small reference CSVs with dbt seeds from version control, set column types so zip codes keep their zeros, and learn when seeds are the wrong tool.
6 min read
CSV
Loading CSV Files in Airflow With a Plain Python Task
Load a daily CSV in Airflow with one plain Python task instead of a chain of operators: idempotent by date, safely retried and testable outside Airflow.
7 min read
CSV
Watching a Folder for New CSV Files and Importing Them
Poll a folder for new CSV files and import each only once it has stopped changing. Why file events fire too early, a stable-file watcher and inotify on Linux.
7 min read
CSV
Archiving Processed CSV Files Automatically After Import
Move each CSV out of the inbox only after its load commits, compress and verify it into dated folders, and expire old files with a clear retention rule.
7 min read
CSV
Building Idempotent CSV Imports That Are Safe to Run Twice
Four ways to make a CSV import safe to rerun, tested on one three-row file run twice: upsert on a key, row hash, replace by batch and a content-hash ledger.
7 min read
CSV
Loading a CSV From S3 Into a Database Without a Local Copy
Four ways to load a CSV stored in S3 into a database: a streamed pipe, the RDS aws_s3 extension, DuckDB reading s3:// directly, and a download-then-load job.
7 min read
CSV
A Python Script That Loads a Daily CSV Into PostgreSQL
A complete psycopg 3 script that stages a daily CSV with COPY, merges it with an upsert, records a content hash, and is safe to run again after a failure.
8 min read
CSVGuide
Automating CSV Imports Into a Database Without Surprises
Build a CSV import job that runs unattended: idempotent loads, one transaction, reject files, an archive, a ledger, locking, alerts and a safe backfill.
21 min read
Get new articles by email
We send one email when a new article is published. No spam.