CSV
Removing Blank Rows and Stray Footer Lines From a CSV
Report-style exports carry titles, blank lines, repeated headers and Total footers. Find the header, drop what is not data and check the footer's claims.
8 min read
Topic
Data Wrangling
Take messy CSV, JSON and spreadsheets and land them in tables without losing a row or a character on the way.
CSV
Normalizing Capitalization and Spelling in a CSV Column
Collapse USA, usa and United States with a mapping table, use casefold, and treat fuzzy matching as a suggestion: it ranked Australia above Austria.
8 min read
CSV
Splitting One CSV Column Into Several Columns With Python
Split names, emails and delimited cells into columns or rows in pandas, handle Last, First, and see why a row-by-row apply ran 40 times slower.
10 min read
CSV
Standardizing Phone Numbers in a CSV File With Python
Normalize phone numbers to E.164 with a regex and the phonenumbers library; the regex accepted 43.8% more numbers than the library calls valid.
8 min read
CSV
Handling Missing Values in a CSV File Without Guessing
A CSV has no NULL, so missing arrives as empty, a dash, N/A or null. Inventory the tokens, convert to real NA, keep the reason and see what fillna(0) does.
8 min read
CSV
Turning Currency Strings Into Numbers in a CSV Exactly
Parse $1,234.50, (12.50), 1.234,50 € and 7,00 into exact Decimals, find the cases where stripping symbols is wrong, and spot a column that mixes currencies.
8 min read
CSV
Fixing Mixed Date Formats in a CSV Column Without Guessing
Four formats in one date column, 959 values that are not dates and 15,642 that could be read two ways. Parse with explicit formats and report what you cannot.
9 min read
CSV
Removing Duplicate Rows From a CSV and Choosing the Keeper
Deduplicating a CSV is easy; deciding which duplicate to keep is not. Compare first, last, latest-signup and highest-balance keepers on 200,000 rows.
9 min read
CSV
Trimming Whitespace in Every Field of a CSV Safely
Trim padded CSV fields with pandas, the csv module, DuckDB and qsv, and see why strip() misses the zero-width space that still breaks an email match.
9 min read
Editor's pick
CSVGuide
A Repeatable Way to Clean a Messy CSV File With Python
A cleaning pipeline that never edits the raw file, counts rows at every step and keeps rejects: 200,000 rows in 3.85 s, identical output twice.
25 min read
CSV
Python CSV Newlines and the Blank Line Bug on Windows
Why a CSV written by Python shows a blank line after every row on Windows, how newline='' prevents it, and how to check the bytes. Reproduced without Windows.
8 min read
CSV
Python CSV Error Field Larger Than Field Limit and the Fix
Fix _csv.Error field larger than field limit (131072). Tell a genuinely large cell from an unclosed quote, raise the limit safely and find the bad record.
8 min read
Get new articles by email
We send one email when a new article is published. No spam.