Topic

Data Analysis

Answers from raw rows: aggregates, cohorts, funnels, time series, plus the string and text wrangling that precedes every report.

XML

Analyzing Publication Metadata From XML Archives in Python

Read thousands of journal article XML files straight from a tar.gz, count by year, journal and author count, rank keywords, and avoid unpacking the archive.

6 min read

YAML

Summarizing dbt Project Metadata From YAML in One Report

Read the dbt manifest built from your properties YAML and report documentation coverage, untested models, owners, tags and source freshness in one script.

7 min read

XML

Extracting Time Series From XML Sensor Files With Pandas

Turn XML sensor readings into a pandas time series, resample to hourly means, find a 50 minute gap and a silent switch from Celsius to Fahrenheit.

6 min read

XML

Counting Records by Attribute in a Large XML Export

Count 400,000 orders by status in a 132 MB XML file with grep in 0.4 s, lxml iterparse in 2.0 s or xmlstarlet in 5.1 s, and know when each can be trusted.

6 min read

YAML

Auditing Kubernetes Manifests With SQL and a Findings Score

Flatten Kubernetes Deployments into one row per container and audit limits, probes, latest tags and privileged mode with SQL, then rank workloads by findings.

7 min read

JSON

Reading JSON Lines With pandas Without the Usual Surprises

Read JSON Lines into a pandas DataFrame with read_json(lines=True): dtypes, chunking, nested fields, big integers, bad lines and the pyarrow engine.

6 min read

XML

Summing Values Across Thousands of XML Invoices by Currency

Total 5,000 XML invoice files in under a second with Python, xmlstarlet or grep, group by currency, and catch malformed files and missing totals.

6 min read

XMLGuide

Answering Questions From XML Data With Counts and Joins

A method for turning an XML file into answers: inspect its shape, extract the fields once, check counts and totals, join carefully and record the result.

17 min read

YAML

Finding Inconsistent Versions Across YAML Configs in a Repo

Find image versions that disagree across Compose, CI, Helm and Kubernetes files in one pass: postgres in five tags, redis in three and latest in three files.

6 min read

XML

Choosing Parquet After You Parse XML for Fast Analysis

A 132 MB XML file became 12.1 MB of Parquet. A grouped query ran in 19 ms against seconds of reparsing. How to write it, and when CSV or a database wins.

6 min read

YAML

Counting Services Across Many Compose Files With DuckDB

Turn 24 Compose files into a table of services with Python and DuckDB and answer which images, versions and missing healthchecks they share, in a few queries.

6 min read

XML

Flattening XML in Spark and Databricks With Native Reader

Spark 4.0 reads XML natively with rowTag. Tested on 100,000 orders: schema inference, explode for lines, the double precision trap and from_xml for strings.

7 min read

Get new articles by email

We send one email when a new article is published. No spam.