CSV
Building Idempotent CSV Imports That Are Safe to Run Twice
Four ways to make a CSV import safe to rerun, tested on one three-row file run twice: upsert on a key, row hash, replace by batch and a content-hash ledger.
7 min read
Tag
how-to
590 posts
CSV
Loading a CSV From S3 Into a Database Without a Local Copy
Four ways to load a CSV stored in S3 into a database: a streamed pipe, the RDS aws_s3 extension, DuckDB reading s3:// directly, and a download-then-load job.
7 min read
CSV
A Python Script That Loads a Daily CSV Into PostgreSQL
A complete psycopg 3 script that stages a daily CSV with COPY, merges it with an upsert, records a content hash, and is safe to run again after a failure.
8 min read
CSVGuide
Automating CSV Imports Into a Database Without Surprises
Build a CSV import job that runs unattended: idempotent loads, one transaction, reject files, an archive, a ledger, locking, alerts and a safe backfill.
21 min read
CSV
Detecting Encoded Payloads Hidden in CSV Fields Fast
A short Python scanner that flags base64 blobs, percent-encoded text, script tags, formula starts and oversized fields in a CSV before you import it.
6 min read
CSV
Sending a CSV by Email Safely and the Better Options
Email is a poor way to move a CSV of real data. Ranked alternatives, from not sending it to expiring links, plus how to encrypt a file you must attach.
6 min read
CSV
Handling CSV Files Full of Customer Data Under GDPR
The GDPR duties that land on a CSV of customer data, from minimization and retention to access requests and deletion, turned into practical engineering steps.
7 min read
CSV
Hashing or Masking Columns in a CSV Safely Without Leaks
A plain SHA-256 of an email is reversible with a guess list. Use a keyed HMAC for tokens and partial masking for display, with a script that does both.
7 min read
Excel
Exporting Invoices to Excel With Totals That Add Up
Export invoices and lines to Excel so totals add up: one row per line, tax by a stated rule, document totals checked, no gaps in numbering and exact decimals.
7 min read
XML
Reading XML in Java With JAXB Annotations and Its Defaults
Map XML to Java classes with Jakarta XML Binding, read attributes, lists and optional elements, and see a bad boolean and a bad number accepted silently.
6 min read
CSV
Stripping Personal Data From a CSV Before Sharing It
Remove, generalize and pseudonymize the personal columns of a CSV with DuckDB before it leaves your team, then check that no one can still be singled out.
7 min read
JSONGuide
JSONPath JMESPath and SQL Path Expressions Compared
JSONPath, JMESPath, jq and the path dialects of PostgreSQL, MySQL and SQLite tested on one document: syntax, null handling, speed, indexes, portability.
19 min read