CSVGuide
How to Validate a CSV File Before You Trust the Data
Validate a CSV delivery in five layers, bytes, structure, schema, values and keys, with line numbers and exit codes: 193,708 rows checked in about a second.
21 min read
Tag
python
134 posts
JSON
Faster JSON in Python With orjson and What Changes
orjson wrote a 35 MB document 6 times faster than json and parsed it 1.7 times faster. What it returns, what it rejects and what to check before migrating.
6 min read
CSV
Finding Rows That Do Not Match the Expected Pattern in a CSV
Check every column of a CSV against a regex and report failing rows with file line numbers. fullmatch beats match, and 2025-13-45 passes a date pattern.
9 min read
XML
Benchmarking ElementTree lxml and iterparse on 127 MB
Four ways to sum 1.2 million order lines from a 127 MB XML file, with time and peak memory, plus the clearing pattern that keeps streaming flat.
6 min read
CSV
Fixing Numbers With Thousands Separators in a CSV File
A comma inside a number breaks int(), shifts columns when unquoted, and turns European 1.200 into 1.2. Parse strictly, set thousands= and decimal=, and test.
8 min read
JSON
Loading JSON Into a pandas DataFrame the Right Way
pandas read_json turned the zip "01234" into 1234. How orient, lines, dtype and the pyarrow backend change the result, and what changed in pandas 3.
6 min read
CSV
Removing Blank Rows and Stray Footer Lines From a CSV
Report-style exports carry titles, blank lines, repeated headers and Total footers. Find the header, drop what is not data and check the footer's claims.
8 min read
Excel
Converting Excel to YAML Config Files Without Surprises
Turn an Excel sheet of settings or tables into YAML in Python: shape the data, avoid the Norway problem and number traps, dump safely and validate.
7 min read
CSV
Normalizing Capitalization and Spelling in a CSV Column
Collapse USA, usa and United States with a mapping table, use casefold, and treat fuzzy matching as a suggestion: it ranked Australia above Austria.
8 min read
Excel
Converting Excel to XML With a Python Script Safely
Convert an Excel sheet to XML in Python: valid element names, escaping, elements versus attributes, encoding, large files and validation against a schema.
7 min read
JSON
Pretty Printing JSON in Python and From the Terminal
Pretty print JSON with json.dumps(indent), python -m json.tool and pprint, keep non-ASCII readable, sort keys and collapse short arrays.
6 min read
CSV
Splitting One CSV Column Into Several Columns With Python
Split names, emails and delimited cells into columns or rows in pandas, handle Last, First, and see why a row-by-row apply ran 40 times slower.
10 min read