YAML
Exporting a Table to YAML From PostgreSQL With Python
Export a PostgreSQL table to YAML with psycopg and PyYAML. Fix the UUID error, keep numerics exact as strings and compare with a to_jsonb pipeline.
7 min read
Tag
python
134 posts
YAML
Loading YAML Seed Data Into PostgreSQL With Python
Load a YAML list into PostgreSQL 18 with psycopg: adapt dates, arrays and jsonb, rerun safely with ON CONFLICT and let constraints reject bad rows.
6 min read
CSV
Python CSV Newlines and the Blank Line Bug on Windows
Why a CSV written by Python shows a blank line after every row on Windows, how newline='' prevents it, and how to check the bytes. Reproduced without Windows.
8 min read
CSV
Python CSV Error Field Larger Than Field Limit and the Fix
Fix _csv.Error field larger than field limit (131072). Tell a genuinely large cell from an unclosed quote, raise the limit safely and find the bad record.
8 min read
CSV
Reading a CSV With Inconsistent Column Counts in Python
Find the rows in a CSV whose field count differs from the header, learn why they happen, and quarantine or repair them in Python without losing data.
8 min read
CSV
Why pandas Turns ZIP Codes Into Numbers and How to Stop It
pandas read_csv infers a column of digits as integers, so 02134 becomes 2134. See the cause, the 190,451 broken codes in 2M rows, and the fixes that work.
7 min read
Excel
Fixing Duplicate Column Names When Reading Excel Data
Handle duplicate column headers when reading Excel in pandas, Power Query and SQL loads: automatic suffixes, safe renaming and what duplicates mean.
7 min read
CSV
Parsing Dates While Reading a CSV With pandas Correctly
Use parse_dates and date_format in pandas read_csv, then handle mixed formats and day-first files, where format="mixed" guessed two conventions in one column.
8 min read
CSV
Reading Only Some Columns of a Large CSV With pandas
Use usecols, dtype, nrows and chunksize to load a large CSV into pandas with a fraction of the memory. Measured on 2,000,000 rows, with the float32 trap.
8 min read
CSV
pandas read_csv Dtype Warnings and How to Fix Them
What DtypeWarning mixed types means in pandas read_csv, how it splits one column into ints and strings, and three ways to fix it, with counts.
7 min read
CSV
Using pandas read_csv for Files That Do Not Behave
Make pandas read_csv cope with title lines, comments, European numbers, odd NA markers, short or long rows and the wrong encoding, one option at a time.
9 min read
CSV
Writing a CSV File With Python and Keeping Quotes Correct
Write CSV in Python with csv.writer and DictWriter: newline='', the quoting modes, how None, floats, dates and bytes are written, and what Excel needs.
9 min read