
Reading Financial and Government XML Datasets With Python
Read the ECB daily reference rates XML: two namespaces, rates as attributes, 29 currencies, a 64-day history, and Decimal for the arithmetic.
Tag
92 posts

Read the ECB daily reference rates XML: two namespaces, rates as attributes, 29 currencies, a 64-day history, and Decimal for the arithmetic.

Map each part of an Excel pivot table to SQL: rows to GROUP BY, values to aggregates, columns to CASE, filters to WHERE and totals to ROLLUP.

Total amounts by currency and top SKUs over 20,000 invoices in one streaming pass with Decimal, 21 MB of memory, and the same USD total as a tree-based check.

Group dates in a pivot table by month, quarter or year, fix the Cannot group that selection error, handle fiscal quarters, and do the same grouping in SQL.

A sheet holds 1,048,576 rows, but a pivot can read more. Use the Data Model, pre-aggregate in SQL or DuckDB, or sample, and know where each approach stops.

Profile an XML file you have never seen in 0.2 seconds and 21 MB of memory, by counting every element and attribute path with iterparse, and read the profile.

Connect Excel to a database, load a SQL query as a connection and build a pivot table on it that refreshes. Covers the Data Model, parameters and traps.

A method for questions about an XML file nobody has converted: look, profile, sample, pick a tool, answer, then prove the numbers with a second method.

For people who know Excel pivot tables and need the SQL behind them: GROUP BY, conditional aggregates, ROLLUP, window functions, unpivoting and checks.

Why spreadsheet formulas slow down as data grows and where SQL pulls ahead: lookup cost, recalculation, memory, repeatability and auditability.

Translate Excel date functions to SQL: YEAR, EOMONTH, EDATE, DATEDIF, NETWORKDAYS, WEEKDAY, WEEKNUM and TEXT, with worked results and week numbering traps.

Map Excel text functions to SQL string functions: LEFT, MID, LEN, TRIM, SUBSTITUTE, FIND, TEXTJOIN, PROPER and TEXT, with the edge cases for spaces and nulls.