Topic

CSV

The format everyone exports and nobody specifies. Delimiters, quoting, encodings and loading it into a database intact.

CSV

Reading Only Some Columns of a Large CSV With pandas

Use usecols, dtype, nrows and chunksize to load a large CSV into pandas with a fraction of the memory. Measured on 2,000,000 rows, with the float32 trap.

8 min read

CSV

pandas read_csv Dtype Warnings and How to Fix Them

What DtypeWarning mixed types means in pandas read_csv, how it splits one column into ints and strings, and three ways to fix it, with counts.

7 min read

CSV

Using pandas read_csv for Files That Do Not Behave

Make pandas read_csv cope with title lines, comments, European numbers, odd NA markers, short or long rows and the wrong encoding, one option at a time.

9 min read

CSV

Writing a CSV File With Python and Keeping Quotes Correct

Write CSV in Python with csv.writer and DictWriter: newline='', the quoting modes, how None, floats, dates and bytes are written, and what Excel needs.

9 min read

CSV

Reading a CSV Into Dictionaries With csv DictReader

csv.DictReader turns each CSV row into a dict keyed by the header. See what it does with missing and extra fields, BOMs, duplicate headers and blank lines.

9 min read

CSV

Using the Python csv Module for Reading Files Line by Line

Read CSV in Python with csv.reader: open with newline='', handle quoted commas and multi-line fields, set the dialect, and stream millions of rows.

8 min read

Reading and Writing CSV Files in Python the Right Way

A working guide to CSV in Python: csv module, DictReader, pandas, polars and DuckDB measured on 2,000,000 rows, plus the encoding, quoting and type rules.

22 min read

CSV

Writing Query Results Back Out to CSV With DuckDB COPY

Export a DuckDB query to CSV with COPY. Control the delimiter, NULL text, quoting, date format and compression, and split output into one file per value.

9 min read

CSV

Joining Two CSV Files With SQL in DuckDB Without a Database

Join two CSV files on a key with plain SQL in DuckDB. Covers a 2M by 250k row join, keys that differ by case and spaces, left and anti joins, and type errors.

8 min read

CSV

Converting a CSV to Parquet With DuckDB in One Line

One COPY statement turns a CSV into Parquet in DuckDB. See the schema it writes, fix zip and money columns, compare compression codecs, and verify the result.

8 min read

CSV

DuckDB Skipping Bad Rows While Reading a Messy CSV

Use ignore_errors and store_rejects in DuckDB to read past bad CSV rows, then query reject_errors to see which lines were dropped and why.

9 min read

CSV

Reading a CSV With Mixed Date Formats in DuckDB Safely

A date column with five formats makes DuckDB fall back to text. Parse it with try_strptime and coalesce, list the failures, and avoid the day-first trap.

7 min read

Get new articles by email

We send one email when a new article is published. No spam.