CSV
Fixing Numbers With Thousands Separators in a CSV File
A comma inside a number breaks int(), shifts columns when unquoted, and turns European 1.200 into 1.2. Parse strictly, set thousands= and decimal=, and test.
8 min read
Library
Everything you need to level up your coding skills.
JSON
Loading JSON Into a pandas DataFrame the Right Way
pandas read_json turned the zip "01234" into 1234. How orient, lines, dtype and the pyarrow backend change the result, and what changed in pandas 3.
6 min read
CSV
Removing Blank Rows and Stray Footer Lines From a CSV
Report-style exports carry titles, blank lines, repeated headers and Total footers. Find the header, drop what is not data and check the footer's claims.
8 min read
YAML
Limiting Alias Expansion in a YAML Parser Across Languages
Where each parser sets its alias limit: SnakeYAML options, Go's built-in check, Ruby's aliases flag, DuckDB's node budget and a PyYAML guard, all tested.
7 min read
Excel
Converting Excel to YAML Config Files Without Surprises
Turn an Excel sheet of settings or tables into YAML in Python: shape the data, avoid the Norway problem and number traps, dump safely and validate.
7 min read
XML
How Much Memory a DOM Parser Needs for a Large XML File
Measured peak memory of ElementTree and lxml on 32 MB and 127 MB order files shows 11 to 15 times the file size, and how streaming cut it to 22 MB.
6 min read
CSV
Normalizing Capitalization and Spelling in a CSV Column
Collapse USA, usa and United States with a mapping table, use casefold, and treat fuzzy matching as a suggestion: it ranked Australia above Austria.
8 min read
Excel
Converting Excel to XML With a Python Script Safely
Convert an Excel sheet to XML in Python: valid element names, escaping, elements versus attributes, encoding, large files and validation against a schema.
7 min read
JSON
Pretty Printing JSON in Python and From the Terminal
Pretty print JSON with json.dumps(indent), python -m json.tool and pprint, keep non-ASCII readable, sort keys and collapse short arrays.
6 min read
CSV
Splitting One CSV Column Into Several Columns With Python
Split names, emails and delimited cells into columns or rows in pandas, handle Last, First, and see why a row-by-row apply ran 40 times slower.
10 min read
XMLGuide
Parsing Big XML Files Fast Without Running Out of Memory
A measured guide to large XML files: memory multipliers, streaming, library choice, COPY into PostgreSQL, parallel splits, compression and indexes.
17 min read
Excel
Converting Excel to JSON With Python for Clean Output
Convert an Excel sheet to JSON in Python: records, nested groups and JSON Lines, with typed values, ISO dates, nulls and a check that nothing was lost.
6 min read
Get new articles by email
We send one email when a new article is published. No spam.