XML
Extracting Time Series From XML Sensor Files With Pandas
Turn XML sensor readings into a pandas time series, resample to hourly means, find a 50 minute gap and a silent switch from Celsius to Fahrenheit.
6 min read
Tag
python
134 posts
CSV
Converting CSV to JSON With Python Without Losing Types
Convert CSV to JSON with the csv module, then decide on types: zip stays text, money stays exact. pandas gave 2134.0 for 02134; converters took 0.4 to 36 s.
9 min read
JSON
Reading JSON Lines With pandas Without the Usual Surprises
Read JSON Lines into a pandas DataFrame with read_json(lines=True): dtypes, chunking, nested fields, big integers, bad lines and the pyarrow engine.
6 min read
YAML
Finding Inconsistent Versions Across YAML Configs in a Repo
Find image versions that disagree across Compose, CI, Helm and Kubernetes files in one pass: postgres in five tags, redis in three and latest in three files.
6 min read
YAML
Counting Services Across Many Compose Files With DuckDB
Turn 24 Compose files into a table of services with Python and DuckDB and answer which images, versions and missing healthchecks they share, in a few queries.
6 min read
CSV
Making a CSV Contract With Your Data Provider in Writing
What to agree with whoever sends you CSV files, as a document plus a manifest your code can check: six deliveries, six different rejections, 193,708 rows.
12 min read
CSV
Testing a CSV Export for Truncated Last Lines in Python
A cut-off CSV loaded in pandas without a warning, last row 200000,Ana,Smith,...,2. Check the final byte, the last record's width, quotes, row count and gzip.
11 min read
CSV
Detecting Columns That Changed Type Between CSV Deliveries
April's load failed on qty 1.5, but six columns had drifted. Compare per-column kinds of value between two CSV deliveries, not just inferred types.
11 min read
CSV
Checking That Every Date in a CSV File Parses Correctly
Check every date in a CSV with a strict parse and a round trip, then ranges. strptime accepted 2025-3-4, DuckDB cast kept the date of 2025-03-14 10:30.
10 min read
CSV
Validating CSV Columns With Python and pydantic Models
Validate each CSV row against a pydantic model with types, literals, patterns and a cross-field rule: 193,708 rows in 0.95 s, with file line numbers.
10 min read
CSV
Writing a Schema for a CSV With Frictionless Data in Python
Infer a Table Schema, fix it by hand, add unique, enum and pattern constraints and validate with Frictionless: 10 errors for 8 faults in 193,708 rows.
11 min read
JSON
Reading JSON From an API With requests and Handling Errors
response.json() raised JSONDecodeError for an empty body, a 204, an HTML 502 and a BOM. Check status first, set a timeout and retry with a Session, as tested.
8 min read