Editor's pick
CSVGuide
Handling CSV Files Too Big to Open in Excel or Python
A decision guide for CSV files too big to open: look, count, sample, query in place, chunk, split, compress or convert, with numbers from a 1.3 GB file.
17 min read
Topic
CSV
The format everyone exports and nobody specifies. Delimiters, quoting, encodings and loading it into a database intact.
CSV
Converting a CSV to Parquet and Why You Might Want To
One DuckDB statement turned a 130 MB CSV into a 34 MB Parquet file; typed columns, dictionary encoding and min/max statistics explain the faster queries.
10 min read
CSV
Round Tripping Exports Back Into Imports Without Drift
A CSV export should re-import to the same data. A test where identical files hid a NULL turned into an empty string, and how to test round trips on rows.
7 min read
CSV
Choosing Column Names Users Recognize in CSV Imports
Name CSV columns in the user's words, keep a stable internal key and accept aliases on import. A mapping function tested on three realistic headers.
7 min read
CSV
Background Jobs for Large CSV Exports That Do Not Time Out
Move big CSV exports into a job: chunk by key, write to storage, send an expiring link, track progress and clean up. Keyset chunks beat OFFSET.
8 min read
CSV
Streaming a CSV Export From a Web App Without Memory Spikes
Stream a CSV download instead of building it in memory: one million rows took 336 MB as an array and 2 MB streamed. Headers, buffering and limits.
7 min read
CSV
Reporting Row Level Errors Back to the User of a CSV Import
Report CSV import errors users can act on: line, column, value and rule, all problems per row, and a file of rejected rows to fix and re-upload.
8 min read
CSV
Previewing a CSV Upload Before Committing Any Changes
Preview a CSV import safely: inspect the file, then run the real import in a transaction that rolls back to count creates, updates and errors.
7 min read
CSV
Designing a CSV Import Screen Users Can Succeed With
What a CSV import screen needs so users succeed: a template, clear limits, column mapping, a preview, plain-language errors and a count before commit.
7 min read
CSVGuide
CSV Import and Export Features in Applications Done Right
Build CSV import and export into an application: staged imports with dry runs, streamed and background exports, column names, round trips and safety.
19 min read
CSV
A Survey Export With Hundreds of Columns Made Usable
A survey CSV has one column per question, 300 wide. Unpivot it to long form with DuckDB, drop blanks, name the questions and check the answer count.
6 min read
CSV
A Government Dataset With Footnotes in the Data Cells
Statistical CSVs put flags in value cells: 1,204,300 (p), 5.3 e, and dots for missing. Split number from flag, keep both and check against the stated total.
7 min read
Get new articles by email
We send one email when a new article is published. No spam.