PostgreSQL
Seq Scan Versus Index Scan Versus Bitmap Scan Explained
Seq scan versus index scan versus bitmap scan in PostgreSQL: how each works, when the planner picks it, index only scans and why a seq scan can be right.
7 min read
Library
Everything you need to level up your coding skills.
Excel
How Many Rows Can Pandas Read From Excel Before It Struggles
Estimate how many rows pandas can read from an Excel file for your memory, why read_excel loads everything, and how usecols and dtype shrink the load.
7 min read
Excel
Using the Excel Data Model for Larger Datasets in Practice
Load millions of rows into the Excel Data Model with Power Query, relate tables, build pivots on it, keep it small and know the limits of memory and bitness.
7 min read
Excel
Benchmarking Excel Readers in Python With a Fair Test
Compare openpyxl, calamine, pandas engines and DuckDB for reading xlsx: build test files, time runs, record peak memory and avoid common benchmark mistakes.
7 min read
Excel
Reducing Workbook Size by Removing Unused Styles and Data
Find why an xlsx is large by looking inside the zip, then remove unused styles, empty formatted rows, pivot caches, images and links to shrink the file.
8 min read
Excel
Analyzing More Than a Million Rows Without Excel Limits
Analyze data beyond Excel's 1,048,576 row limit with DuckDB, SQLite, pandas in chunks or the Data Model, and bring only the summary back to Excel.
6 min read
Excel
Converting a Huge Excel File to CSV or Parquet Safely
Convert a very large Excel workbook to CSV or Parquet with a streaming script, DuckDB or LibreOffice, keeping types, dates, encoding and every sheet right.
8 min read
Excel
Why Excel Is Slow With Formulas Over Many Rows Explained
Why Excel slows down with many formulas: the calculation chain, volatile functions, whole column references and scanning lookups, and how to find the cost.
8 min read
PostgreSQL
How to Read a PostgreSQL Query Plan Step by Step Guide
Read a PostgreSQL query plan line by line: tree shape, estimate and actual groups, loops, own time per node, and a worked plan with one slow step.
7 min read
Excel
Reading a 500 MB Excel File Without Running Out of Memory
Read a very large xlsx in Python without exhausting memory: why xlsx expands, openpyxl read only mode, calamine and DuckDB, and how to measure peak memory.
7 min read
ExcelGuide
A Guide to Excel Files That Are Too Big or Too Slow
Diagnose an Excel workbook that is huge, slow to calculate or too large to read in code, then fix it by size, formulas, the Data Model or a different format.
17 min read
PostgreSQLGuide
Reading EXPLAIN ANALYZE Output in PostgreSQL Step by Step
Read EXPLAIN ANALYZE output in PostgreSQL: plan tree, cost, rows and loops, scan and join nodes, BUFFERS, sorts, estimates and a way to find the slow step.
18 min read
Get new articles by email
We send one email when a new article is published. No spam.