Benchmarking Excel Readers in Python With a Fair Test

Which Python library reads Excel files fastest? The honest answer is that it depends on the file, the machine and what you measure, and blog posts that give a single number are measuring one file on one laptop. This post does not give numbers. I did not run the harness, and I would rather show you how to produce results you can trust than repeat figures I cannot stand behind. It lays out a method that avoids the usual mistakes, a script to run it, and how to read the result. It is a spoke of Excel files that are too big or too slow.

What to compare

The readers worth testing in Python:

Decide what the benchmark is for before choosing: reading everything into a DataFrame, streaming rows to a database, reading a few columns, or reading many small files. Each favors different readers.

Make test files that look like yours

Reader speed depends on the content. A file of one million numeric cells and a file of the same size with repeated text parse differently. Generate files with the properties of your data:

import random
import xlsxwriter

def make(path, rows, str_cols=4, num_cols=4, distinct=500, seed=1):
    rnd = random.Random(seed)
    words = [f"customer_{i:05d}" for i in range(distinct)]
    wb = xlsxwriter.Workbook(path, {"constant_memory": True})
    ws = wb.add_worksheet("Data")
    ws.write_row(0, 0, [f"s{i}" for i in range(str_cols)] + [f"n{i}" for i in range(num_cols)])
    for r in range(1, rows + 1):
        ws.write_row(r, 0, [rnd.choice(words) for _ in range(str_cols)]
                          + [round(rnd.random() * 1000, 2) for _ in range(num_cols)])
    wb.close()

for n in (10_000, 100_000, 500_000):
    make(f"bench_{n}.xlsx", n)

A fixed seed makes the files reproducible. constant_memory lets XlsxWriter write large files without holding them. Vary what matters for you: the number of rows, the share of text columns, the number of distinct strings (shared strings table size), dates, formulas with cached values, and styles. If your real files come from a particular system, test with a sample of those instead, with sensitive values replaced.

Keep the files out of the cache question: after generating, run each reader once untimed (a warm-up) or drop the operating system's file cache between runs, and say which you did. A cold read of a file from a slow disk measures the disk.

Time and memory in a fresh process

Run each reader in its own process so that one reader's allocations do not affect another's and so that peak memory is a clean number. A runner script takes the reader name and the file, and an outer driver launches it several times:

# read_one.py
import sys, time

reader, path = sys.argv[1], sys.argv[2]

def run():
    if reader == "openpyxl":
        from openpyxl import load_workbook
        wb = load_workbook(path, data_only=True)
        n = sum(1 for _ in wb["Data"].iter_rows(values_only=True))
    elif reader == "openpyxl_ro":
        from openpyxl import load_workbook
        wb = load_workbook(path, read_only=True, data_only=True)
        n = sum(1 for _ in wb["Data"].iter_rows(values_only=True))
        wb.close()
    elif reader == "pandas_openpyxl":
        import pandas as pd
        n = len(pd.read_excel(path, engine="openpyxl"))
    elif reader == "pandas_calamine":
        import pandas as pd
        n = len(pd.read_excel(path, engine="calamine"))
    elif reader == "calamine":
        from python_calamine import CalamineWorkbook
        n = len(CalamineWorkbook.from_path(path).get_sheet_by_name("Data").to_python())
    else:
        raise SystemExit("unknown reader")
    return n

t0 = time.perf_counter()
n = run()
print(f"{reader},{path},{n},{time.perf_counter() - t0:.3f}")

Peak memory comes from the operating system, not from Python's own tracking, which misses native allocations. On Linux, wrap the command: /usr/bin/time -v python read_one.py openpyxl bench_100000.xlsx and read "Maximum resident set size". On macOS, /usr/bin/time -l prints "maximum resident set size" (in bytes). From Python, resource.getrusage(resource.RUSAGE_SELF).ru_maxrss gives the same number at the end of the process (kilobytes on Linux, bytes on macOS). Record it in the same line as the time.

Repeat each combination five times, in an interleaved order (reader A, B, C, A, B, C) so drift from thermal throttling or background tasks does not favor one reader. Report the median and the spread. A single fast run is an accident.

Note the timer covers the whole run() including imports done inside it, which for pandas is substantial. Decide what you want: for a long-running job, import cost is irrelevant, so import before the timer; for a command-line tool that starts fresh each time, it counts. State the choice.

Check that they read the same thing

A reader that silently drops empty rows, truncates at a limit or reads only the first sheet looks fast. Every run must assert equal results. A cheap check is the row count and a checksum of a numeric column:

total = sum(r[4] for r in rows if isinstance(r[4], (int, float)))

Compare n and total across readers for each file, and stop the benchmark when they differ. Also compare types: one reader may return dates as datetimes and another as serial numbers, or numbers as floats and others as ints. If the downstream code needs a certain type, the cost of converting belongs in the measurement.

Report it properly

A benchmark result without context is not transferable. Keep a table with:

• the hardware (CPU model, cores, memory, disk type), the operating system, the Python version and the version of every library;

• the file properties (rows, columns, text share, distinct strings, size on disk);

• cold or warm cache, and the repetition count;

• time (median, minimum and maximum) and peak memory for each reader.

Plot time against rows for each reader on a log scale to see how it grows. Linear growth is the goal; a reader whose memory grows faster than its file size will fail at some size you can extrapolate. Share the script and the generated files' recipe, so someone else can reproduce your numbers and disagree with them.

Reading the result

Expect to find, and verify on your own machine, a pattern like this. Pure Python readers in full-load mode use a large multiple of the file size in memory and take the longest. Streaming modes keep memory low and run at a similar or slower speed. Rust-based parsers are faster and use less memory per cell, with differences in how they type cells. A pandas call adds the DataFrame construction cost on top of the reader. The relative order can change with file shape: a file with many distinct strings and a file with a few change the picture. Treat this as hypotheses to test, not as results.

The decision usually follows from a constraint rather than from rank. If memory is capped, pick the lowest-memory reader that is fast enough. If the job runs nightly, pick the simplest one that finishes in the window. If you need formulas or styles, you are limited to the readers that return them. If you read each file once, convert to Parquet and stop paying the cost again (converting a huge Excel file to CSV or Parquet).

Common benchmark mistakes

• Timing one run. Variance between runs on a laptop can exceed the difference between readers.

• Warm versus cold cache unstated, or mixed.

• Different work. One reader returns lists and another a DataFrame with type inference.

• Not checking correctness. A fast wrong answer wins.

• Tiny files. At a few thousand rows, process start and imports dominate and the ranking is meaningless.

• **Trusting tracemalloc.** It only tracks Python allocations, and misses the native memory in compiled readers.

• Benchmarking on the wrong machine. Results from a laptop do not carry to a server with a different disk and CPU.

• Optimizing the wrong thing. If the time is spent downstream (cleaning, writing to a database), the reader's speed hardly matters.

When a benchmark will not answer your question

Your files are not like the test files. Use a sample of the real ones.

The bottleneck is not reading. Profile the whole job first (why Excel is slow with formulas over many rows covers the Excel side, and a Python profiler the other).

You need correctness more than speed. For formulas, merged cells or styles, pick the reader that supports them and accept its speed.

The files are small. Under a few thousand rows, any reader is instant. Choose on convenience.

See the openpyxl notes on performance and the pandas reference for read_excel, which lists the supported engines. To check a small extract as JSON, use the CSV to JSON converter.