Using csv parse in Node for Large Files Without Surprises

csv-parse is the parser in the csv family of packages for Node, and the one I would pick for a pipeline that has to cope with hostile input: it is strict by default, its errors carry a code and a line number, and nearly every behavior is an option. That is also its weakness, because the options have costs, and some of them change the data. This article goes through the ones that matter for large files, with timings on 2,000,000 rows and the output of each option on a small sample. The streaming basics are in the streams article, and the whole picture in the guide. The format it implements is RFC 4180.

What each option costs

I parsed the whole file with a handful of configurations and counted the records. The script runs each variant in its own process, for a reason explained at the end:

import fs from "node:fs";
import { parse } from "csv-parse";

const path = "/batches/csv-0001-0200/work/orders-2m.csv";
const variants = {
  "arrays":                  { from_line: 2 },
  "columns: true":           { columns: true },
  "columns + cast: true":    { columns: true, cast: true },
  "columns + cast_date":     { columns: true, cast: true, cast_date: true },
  "columns + on_record":     { columns: true, on_record: (r) => (r.status === "refunded" ? r : null) },
  "columns + skip_empty":    { columns: true, skip_empty_lines: true, trim: true },
  "arrays + raw + info":     { from_line: 2, info: true, raw: true },
};
const only = process.argv[2];
for (const [name, opts] of Object.entries(variants)) {
  if (only && name !== only) continue;
  const t = performance.now();
  let n = 0;
  for await (const _ of fs.createReadStream(path).pipe(parse(opts))) n++;
  console.log(`${name.padEnd(24)} ${String(n).padStart(8)} records ${((performance.now() - t) / 1000).toFixed(2)}s`);
}

for v in "arrays" "columns: true" "columns + cast: true" "columns + cast_date" "columns + on_record" "columns + skip_empty" "arrays + raw + info"; do node --max-old-space-size=4096 options_bench.mjs "$v"; done

arrays                    2000000 records 4.04s
columns: true             2000000 records 6.02s
columns + cast: true      2000000 records 7.80s
columns + cast_date       2000000 records 9.75s
columns + on_record        400135 records 6.14s
columns + skip_empty      2000000 records 6.55s
arrays + raw + info       2000000 records 4.73s

Three conclusions. Turning each row into an object (columns: true) costs about 50% more than leaving it as an array, 6.02 s against 4.04 s, so for a hot path where you only need a few fields by position, arrays are cheaper. Each conversion option adds work for every field of every record, which is why cast and date casting stack up. And on_record is the cheap way to filter: it runs inside the parser, so records you return null for are never emitted to your code, and only 400,135 of 2,000,000 crossed into the stream. Do the filtering there and not in a Transform later.

cast: convenient and risky

cast: true converts any field that looks like a number to a Number. Here is a small file with a ZIP column, a number column, a boolean-looking column and a date:

import { parse } from "csv-parse/sync";

const zips = "zip,n,ok,when\n02134,1,true,2025-03-14\n90210,2.50,false,2025-03-15\n00501,3,TRUE,not a date\n";
console.log("cast:true      ", JSON.stringify(parse(zips, { columns: true, cast: true })));
console.log("cast function  ", JSON.stringify(parse(zips, {
  columns: true,
  cast: (value, ctx) => (ctx.header ? value : ctx.column === "n" ? Number(value) : value),
})));
console.log("cast_date      ", JSON.stringify(parse(zips, { columns: true, cast: (v, c) => (c.column === "when" ? new Date(v) : v) }).map((r) => String(r.when))));
try { parse("a,a\n1,2\n", { columns: true }); console.log("dup header ok"); } catch (e) { console.log("duplicate:", e.code, e.message.slice(0, 70)); }
console.log("columns trim   ", JSON.stringify(parse(" id , name \n1,Ana\n", { columns: (h) => h.map((x) => x.trim()) })));
console.log("quote:false    ", JSON.stringify(parse('a,"b,c"\n', { quote: false, relax_column_count: true })));
console.log("relax_quotes   ", JSON.stringify(parse('a,say "hi" now,c\n', { relax_quotes: true })));
try { parse('a,say "hi" now,c\n'); } catch (e) { console.log("strict         ", e.code); }
console.log("delimiter ;    ", JSON.stringify(parse("a;b\n1;2\n", { delimiter: ";", columns: true })));
console.log("max_record_size", (() => { try { parse("a," + "x".repeat(50), { max_record_size: 20 }); } catch (e) { return e.code; } })());
console.log("to_line 3      ", JSON.stringify(parse("a\nb\nc\nd\n", { to_line: 3 })));
console.log("from_line/skip ", JSON.stringify(parse("Title\n\nid,n\n1,2\n", { from_line: 3, columns: true })));

node options_demo.mjs

cast:true       [{"zip":2134,"n":1,"ok":"true","when":"2025-03-14"},{"zip":90210,"n":2.5,"ok":"false","when":"2025-03-15"},{"zip":501,"n":3,"ok":"TRUE","when":"not a date"}]
cast function   [{"zip":"02134","n":1,"ok":"true","when":"2025-03-14"},{"zip":"90210","n":2.5,"ok":"false","when":"2025-03-15"},{"zip":"00501","n":3,"ok":"TRUE","when":"not a date"}]
cast_date       ["Fri Mar 14 2025 00:00:00 GMT+0000 (Coordinated Universal Time)","Sat Mar 15 2025 00:00:00 GMT+0000 (Coordinated Universal Time)","Invalid Date"]
dup header ok
columns trim    [{"id":"1","name":"Ana"}]
quote:false     [["a","\"b","c\""]]
relax_quotes    [["a","say \"hi\" now","c"]]
strict          INVALID_OPENING_QUOTE
delimiter ;     [{"a":"1","b":"2"}]
max_record_size CSV_MAX_RECORD_SIZE
to_line 3       [["a"],["b"],["c"]]
from_line/skip  [{"id":"1","n":"2"}]

The cast:true line is the trap: {"zip":2134,...} and {"zip":501,...}. The leading zeros are gone, because Number("02134") is 2134, and the damage is permanent for anyone downstream. The ok column stayed a string, "true", so cast: true converts numbers and not booleans. The cast function line is the fix: a function that receives the value and a context (with ctx.column and ctx.header) and converts only the column you name. The ZIPs stayed "02134", "90210" and "00501", while n became the numbers 1, 2.5 and 3. A cast that is explicit about its columns is a schema, and it documents the contract with the file. That is where I stop treating cast: true as a convenience.

The date line shows another quiet failure. A cast function that builds new Date(value) gave a real date for 2025-03-14 and 15 and Invalid Date for "not a date", with no error. Invalid Date is a valid object, so nothing downstream complains until you call toISOString() and it throws. Validate in the cast function and throw or collect the value yourself.

Quotes, delimiters and limits

The same demo shows the strictness options:

• **relax_quotes: true** accepted a,say "hi" now,c, returning the middle field with its inner quotes intact. Without it, the parser raised INVALID_OPENING_QUOTE. Strict is the right default, and relax is for a producer that writes quotes carelessly; know that it is not a repair and count what it accepted.

• **quote: false** turned off quote handling altogether. For a,"b,c" it returned ["a","\"b","c\""], three fields. That is the split behavior, so use it only for a format that never quotes.

• **delimiter: ";"** read a semicolon file. The delimiter can be a string or an array of candidates.

• **max_record_size** limits the size of a single record and raised CSV_MAX_RECORD_SIZE when a record exceeded 20 bytes in the test. It is the Node counterpart of Python's field size limit, and the right defense against an unclosed quote swallowing a file: set it to a number that is generous for your data.

• **from_line and to_line**: to_line: 3 returned the first three records and stopped; from_line: 3 skipped a title and a blank line to start at the header. These limit the work done, so a preview of the first 100 lines of a large file costs almost nothing.

• **columns** accepts a function over the header, which is the place to clean names. A header of " id , name " with columns: h => h.map(x => x.trim()) gave keys id and name.

Errors, with codes

Every parse failure carries a code, which you can switch on. The ones seen in these tests were CSV_RECORD_INCONSISTENT_COLUMNS (a record whose length differs from the header's), CSV_QUOTE_NOT_CLOSED, INVALID_OPENING_QUOTE and CSV_MAX_RECORD_SIZE. Two of them report a position: the column-count error names the line, and the unclosed-quote error names the line where the file ended, not where the quote began, as the streams article found. For recoverable cases, skip_records_with_error with an on_skip callback lets you keep a list of what was skipped, which is the difference between a lossy import and a silent one.

A benchmark that lies

The timings above are from separate processes, and there is a reason. My first version of the benchmark looped over all seven configurations inside one Node process. The first four results were within about 15% of the ones above. The later ones were not:

columns + on_record        400135 records 15.62s
columns + skip_empty      2000000 records 16.55s
arrays + raw + info       2000000 records 14.22s

Those are two to three times slower than the same configurations in a fresh process (6.14 s, 6.55 s, 4.73 s). Each pass allocates millions of objects, the heap grows, and garbage collection costs more in each later pass. The conclusion I drew from the first run was that trim and raw were expensive; the conclusion from the second was that they were not. The honest lesson is to benchmark each configuration in its own process, repeat it, and distrust a table whose slowest rows are the last ones.

How to tell it worked

Compare the record count with an independent count: qsv count or DuckDB on the same file. Check the types of the first record, particularly any identifier. Run the parser against a file with a stray quote and confirm it fails with the code you expect and a non-zero exit. Print process.resourceUsage().maxRSS and confirm it does not grow with the file size. And run a benchmark for the configuration you will deploy, in a fresh process, at the size you will see.

When csv-parse is the slow choice

It is not the fastest parser available. In the benchmark in the guide, Papa Parse as a stream took 1.12 s on this file and a hand-written split 1.15 s, against 4.16 s for csv-parse, so a job whose only goal is throughput on a clean file may want another tool. The trade is that csv-parse gives you the strictness and the error detail, and a pipeline for data from outside is rarely limited by the parser. If the work is mostly SQL-shaped, loading the file into a database or querying it with DuckDB is faster than any JavaScript, as the database loading article measured.