Why a Hostile CSV Upload Can Hurt Your Application Server

Nobody has to exploit a bug to hurt a service that accepts CSV uploads. The format is simple and the parsers are obliging, which is exactly the problem: a parser will faithfully read whatever is put in front of it, and a service that reads faithfully with no limits can be made to use all of its memory, all of its CPU or all of its disk by a file that looks innocent in a directory listing. This is the upload side of CSV security risks that catch teams out. The attacks below are not exotic. Most of them are accidents that happen to honest users every month, and a hostile user just does them on purpose and at scale.

The compressed bomb

Services that accept .csv.gz or .zip save bandwidth and disk. They also let a very small request describe a very large file. DEFLATE, the algorithm inside gzip and zip, handles repetition so well that a long run of one byte compresses by a factor near 1,000. Building the proof takes four lines:

import gzip
with gzip.open("bomb.csv.gz", "wb") as f:
    f.write(b"0" * 200_000_000)

The resulting file is 194,431 bytes on disk and decompresses to 200,000,000. If your code calls gzip.open(...).read() or hands the file to a library that decompresses fully before parsing, a request under 200 KB allocates 200 MB, and ten concurrent requests allocate 2 GB. Nothing in the upload size check tripped, because the check looked at the compressed size.

The defense is to count the decompressed bytes as you read and stop at a limit. The loop is short:

import gzip, sys

LIMIT = 50 * 1024 * 1024  # decompressed bytes we are willing to read

def read_capped(path):
    total = 0
    with gzip.open(path, "rb") as f:
        while chunk := f.read(1024 * 1024):
            total += len(chunk)
            if total > LIMIT:
                raise ValueError(f"decompressed size is over {LIMIT} bytes, stopping at {total}")
            yield chunk

try:
    for _ in read_capped(sys.argv[1]):
        pass
except ValueError as e:
    print("rejected:", e)

rejected: decompressed size is over 52428800 bytes, stopping at 53477376

It reads one megabyte at a time, so it stops a little past the limit (53,477,376 bytes here, a few chunks over) and never holds more than a chunk in memory. The point is that the cap is on what comes out. A limit on what goes in tells you nothing about a compressed file.

One field that never ends

The next failure is a quoting problem. A field that starts with a double quote ends at the next unescaped double quote. If the closing quote never appears, a lenient parser keeps reading, newlines and all, until the end of the file. Python's csv module does this by default:

import csv, io
rows = list(csv.reader(io.StringIO('a,"b\nc,d\ne,f\n')))
print(len(rows), rows)

1 [['a', 'b\nc,d\ne,f\n']]

Three lines of input produced one record whose second field is everything after the opening quote. On a 500 MB file with a quote near the top, that field is 500 MB, held in one Python string. It is also what happens to honest data when a producer forgets to escape a quote inside a name, as in how CSV quoting works, so the same defense also catches broken files.

Python's reader has a limit for exactly this, csv.field_size_limit, which defaults to 131,072 characters. A field above it raises an error:

_csv.Error: field larger than field limit (131072)

I measured this with a field of 200,000 characters and the message appeared at once. Keep the limit. It is common advice to raise it to sys.maxsize to make the error go away, and for an upload endpoint that is the wrong response: you have disabled the guard that protects you (fixing the Python field larger than limit error explains when raising it is right, which is for a file you control). For untrusted input, set it lower than the default if your fields are short. Other parsers differ. Some have no limit, and some count bytes where others count characters, so read the documentation of the one you use and test it with a deliberately broken file.

Many rows, many columns

Two shape attacks use ordinary bytes. A file of ten million lines containing one comma each is about 20 MB and takes a loop ten million iterations long. Parsing is quick, and whatever you do per row, such as validation, a lookup or an insert, is the cost. At the validation cost of a database round trip per row, 10 million rows is hours of work for a worker, which makes a queue unavailable to everyone else.

A file with a header of 100,000 columns breaks code that builds a table or a dataframe from the header. A CREATE TABLE with 100,000 columns fails in most databases (PostgreSQL's limit is 1,600 columns), pandas allocates a block per column, and a loader that generates SQL from the header builds a statement of megabytes. The cure is a cap on the column count before you look at anything else, and a header that must match a known list.

The row cap and the column cap are separate numbers, and so are the field cap and the decompressed byte cap. A file can pass three and break on the fourth. Limiting file size and row count for CSV uploads puts figures on each of them.

CPU-heavy values

A hostile value can cost more than its length. A regular expression with nested quantifiers can take exponential time on a crafted string, and a validation routine that applies such a pattern to every field turns a 1 MB file into minutes of CPU. A date parser that tries ten formats per cell does ten times the work on a file of garbage. Use patterns that cannot backtrack catastrophically, cap the length of any field before running a pattern on it, and measure the worst case with a test file of junk, not only with a valid one.

The filename and the path

An uploaded file arrives with a name the client chose. ../../etc/cron.d/job.csv is a path traversal attempt, and a name with spaces, a null byte or unusual Unicode can break scripts that pass it to a shell. Never use the client's name as a path. Generate your own, such as a random identifier, store the original name as a label and escape it wherever you display it. Store uploads outside the web root and never serve them back under a content type a browser will execute. The OWASP file upload cheat sheet lists these controls in one place.

A safe shape for the endpoint

Put the controls together and the endpoint looks like this. Accept only the types you expect, by reading the file and not by trusting the extension. Enforce a body size at the web server, so an oversized request is rejected before it reaches your code. Stream the file to disk or to the parser a chunk at a time with the decompressed cap. Parse with a field size limit, a row cap and a column cap, and fail with a message that names the line. Load into a staging table under a role that can insert into that table and nothing else (loading a CSV into a staging table, then merging), validate there, and only then move rows to the real tables. A large file belongs in a background job with its own memory limit, and streaming a huge CSV into a database shows the loader.

None of this needs a security product. It needs five numbers (compressed size, decompressed size, rows, columns, field length), chosen from what real files look like, and a test that sends a file past each.

When the uploader is someone you trust

A nightly file from your own warehouse, produced by code you control, does not need the same defense. A cap still helps, because a bug in your producer can generate a file that looks like an attack, and a limit turns that incident into an error message. But you can set the numbers from your real maximum with generous headroom, and skip filename and type controls.

The cases where the numbers differ are worth stating. An internal analyst loading a 4 GB file on purpose needs a higher cap, not an exemption, and a per-role limit does it. A partner feed is untrusted in content but not in volume, and its cap can be set from the contract. Only a public endpoint needs the whole set, and a public endpoint that takes CSV is a feature you should ask whether you need. For checking that the file itself is well formed, validating a CSV file comes after these limits, and the Python behavior described here is in the csv module documentation and the gzip module documentation.