CSV
A Survey Export With Hundreds of Columns Made Usable
A survey CSV has one column per question, 300 wide. Unpivot it to long form with DuckDB, drop blanks, name the questions and check the answer count.
6 min read
Tag
cleaning
23 posts
CSV
A Government Dataset With Footnotes in the Data Cells
Statistical CSVs put flags in value cells: 1,204,300 (p), 5.3 e, and dots for missing. Split number from flag, keep both and check against the stated total.
7 min read
CSV
A Log Export With Unescaped Quotes and How to Read It
A log export has quotes inside quoted fields without doubling them. See what Python and DuckDB return, which setting reads them right and how to prove it.
7 min read
CSV
A Supplier Price List With Merged Header Rows Fixed
A price list CSV has title lines, a header on row 4, category names written once per group and a footer. Skip, forward fill, filter and check the item count.
7 min read
CSV
An Analytics Export With Percent Signs and Dashes Cleaned
An analytics CSV writes numbers as text: 12,430, 42.5%, $1,204.50, (4.1%) and dashes. Convert them with one macro and prove nothing is left unparsed.
6 min read
CSV
A CRM Export With Multi Line Notes and How to Parse It
A CRM export has notes with line breaks inside quoted fields, so line counts and grep mislead. Parse by records, then flatten the breaks or keep them.
6 min read
CSV
An Ecommerce Export With Commas Inside Product Names
A product export writes names with commas and no quotes, so rows have 6 fields, not 4. Rejoin the middle fields from the good ends and verify every row.
6 min read
CSV
A Bank CSV With Dates in Two Formats and How to Fix It
A statement mixes 2026-03-04 and 05/03/2026 in one column. Classify each row, prove the slash order from the data, convert, and check the dates are in order.
6 min read
JSON
Cleaning a Twitter Style Archive File That Is Not Quite JSON
Clean a social media archive that is JavaScript, not JSON: strip the assignment, join parts, fix string numbers and dates, unescape HTML and write Parquet.
6 min read
CSVGuide
Worked CSV Problems and the Patterns Behind Them Explained
Eight messy CSV files solved end to end with a check each: mixed dates, unquoted commas, multi-line notes, display formats, headers, bad quotes, footnotes.
18 min read
CSV
Handling CSV With Comment Lines and Metadata Headers
Some CSV files carry lines starting with a hash that are notes, not data. Skip them safely, because a comment option can also cut values like Suite #4.
6 min read
CSV
Detecting and Handling Embedded Headers Repeated Mid File
A CSV that repeats its header after every page or section breaks type conversion. Find the repeated lines, drop them safely and check nothing real went too.
6 min read