
Choosing Parquet After You Parse XML for Fast Analysis
A 132 MB XML file became 12.1 MB of Parquet. A grouped query ran in 19 ms against seconds of reparsing. How to write it, and when CSV or a database wins.
Tag
58 posts

A 132 MB XML file became 12.1 MB of Parquet. A grouped query ran in 19 ms against seconds of reparsing. How to write it, and when CSV or a database wins.

JSON Lines or CSV for event exports? Measured file size, compressed size and read time on 200,000 events, plus nesting, sparse fields and null handling.

Pick a config format by who edits it and what reads it. One config tested in YAML, TOML, JSON, INI and more: types, traps, errors, size and switching cost.

A partner asks for XML for a flat table. Measured on 100,000 rows, XML was 2.7 times the CSV size, though gzip nearly erased the gap. When each is right.

One small config written three ways: size, comments, types and the YAML country code that parses as False. A decision guide for choosing between them.

A decision guide for XML versus JSON, YAML, CSV and Protocol Buffers, with measured sizes and parse times, six scenarios and a migration path.

When a JSON column in PostgreSQL, MySQL or SQLite is enough, and when a document database such as MongoDB earns its place, with measured numbers.

Deduplicating a CSV is easy; deciding which duplicate to keep is not. Compare first, last, latest-signup and highest-balance keepers on 200,000 rows.

Compare xlsx, xlsm, xls and xlsb for data work, and set a policy that blocks macros by default while allowing reviewed exceptions with a real owner.

XML names are case sensitive and follow strict character rules. Compare camelCase, snake_case and kebab-case for elements, with the rules that decide.

Nested loop, hash join and merge join compared in PostgreSQL: how each works, cost shape, memory and batches, plan clues and fixes when the choice is wrong.

How SQL Server, Oracle, Snowflake and BigQuery store, read, unpack, build and index JSON, side by side, with the null, type and size traps in each.