XML
Extracting Time Series From XML Sensor Files With Pandas
Turn XML sensor readings into a pandas time series, resample to hourly means, find a 50 minute gap and a silent switch from Celsius to Fahrenheit.
6 min read
Library
Everything you need to level up your coding skills.
CSVGuide
Converting CSV to JSON XML YAML and Excel in Practice
Convert one awkward CSV to JSON, JSON Lines, XML, YAML, Excel, Parquet, Markdown and SQL, read each back and see the cost on 193,708 rows.
22 min read
XML
Counting Records by Attribute in a Large XML Export
Count 400,000 orders by status in a 132 MB XML file with grep in 0.4 s, lxml iterparse in 2.0 s or xmlstarlet in 5.1 s, and know when each can be trusted.
6 min read
XML
Summing Values Across Thousands of XML Invoices by Currency
Total 5,000 XML invoice files in under a second with Python, xmlstarlet or grep, group by currency, and catch malformed files and missing totals.
6 min read
XMLGuide
Answering Questions From XML Data With Counts and Joins
A method for turning an XML file into answers: inspect its shape, extract the fields once, check counts and totals, join carefully and record the result.
17 min read
XML
Choosing Parquet After You Parse XML for Fast Analysis
A 132 MB XML file became 12.1 MB of Parquet. A grouped query ran in 19 ms against seconds of reparsing. How to write it, and when CSV or a database wins.
6 min read
XML
Flattening XML in Spark and Databricks With Native Reader
Spark 4.0 reads XML natively with rowTag. Tested on 100,000 orders: schema inference, explode for lines, the double precision trap and from_xml for strings.
7 min read
XML
Parsing XML in BigQuery With a JavaScript UDF or Upstream
BigQuery has no XML loader or functions. Two routes: a JavaScript UDF with fast-xml-parser, tested here in Node, or parsing before the load to Parquet.
6 min read
XML
Loading XML Into Snowflake With Native Support and FLATTEN
Snowflake loads XML into VARIANT with a file format of TYPE XML, and XMLGET plus FLATTEN turn it into rows. The statements, the data shape and the limits.
6 min read
XMLGuide
Handling XML in Warehouses and Lakes the Practical Way
How to get XML into Snowflake, BigQuery, Spark, DuckDB and a lake: keep raw files, parse once with a typed schema, store Parquet and reconcile counts.
16 min read
XML
XML Versus CSV for Exchanging Tabular Data Between Systems
A partner asks for XML for a flat table. Measured on 100,000 rows, XML was 2.7 times the CSV size, though gzip nearly erased the gap. When each is right.
6 min read
XML
XML Versus JSON Versus YAML for Configuration Files
One small config written three ways: size, comments, types and the YAML country code that parses as False. A decision guide for choosing between them.
5 min read
Get new articles by email
We send one email when a new article is published. No spam.