CSV
Python CSV Newlines and the Blank Line Bug on Windows
Why a CSV written by Python shows a blank line after every row on Windows, how newline='' prevents it, and how to check the bytes. Reproduced without Windows.
8 min read
Topic
Scripting
Shell one-liners and small scripts that turn your repeated database chores into a single command.
CSV
Python CSV Error Field Larger Than Field Limit and the Fix
Fix _csv.Error field larger than field limit (131072). Tell a genuinely large cell from an unclosed quote, raise the limit safely and find the bad record.
8 min read
CSV
Reading a CSV With Inconsistent Column Counts in Python
Find the rows in a CSV whose field count differs from the header, learn why they happen, and quarantine or repair them in Python without losing data.
8 min read
ExcelGuide
A Practical Guide to Automating Excel Work Without Clicking
How to move recurring Excel work into scripts: choosing VBA, Python, Office Scripts or Power Query, scheduling, merging, splitting, protecting and testing.
17 min read
CSV
Why pandas Turns ZIP Codes Into Numbers and How to Stop It
pandas read_csv infers a column of digits as integers, so 02134 becomes 2134. See the cause, the 190,451 broken codes in 2M rows, and the fixes that work.
7 min read
CSV
Parsing Dates While Reading a CSV With pandas Correctly
Use parse_dates and date_format in pandas read_csv, then handle mixed formats and day-first files, where format="mixed" guessed two conventions in one column.
8 min read
CSV
Reading Only Some Columns of a Large CSV With pandas
Use usecols, dtype, nrows and chunksize to load a large CSV into pandas with a fraction of the memory. Measured on 2,000,000 rows, with the float32 trap.
8 min read
CSV
pandas read_csv Dtype Warnings and How to Fix Them
What DtypeWarning mixed types means in pandas read_csv, how it splits one column into ints and strings, and three ways to fix it, with counts.
7 min read
CSV
Using pandas read_csv for Files That Do Not Behave
Make pandas read_csv cope with title lines, comments, European numbers, odd NA markers, short or long rows and the wrong encoding, one option at a time.
9 min read
CSV
Writing a CSV File With Python and Keeping Quotes Correct
Write CSV in Python with csv.writer and DictWriter: newline='', the quoting modes, how None, floats, dates and bytes are written, and what Excel needs.
9 min read
CSV
Reading a CSV Into Dictionaries With csv DictReader
csv.DictReader turns each CSV row into a dict keyed by the header. See what it does with missing and extra fields, BOMs, duplicate headers and blank lines.
9 min read
CSV
Using the Python csv Module for Reading Files Line by Line
Read CSV in Python with csv.reader: open with newline='', handle quoted commas and multi-line fields, set the dialect, and stream millions of rows.
8 min read
Get new articles by email
We send one email when a new article is published. No spam.