Skip to content
datahacks.dev
CSV

Converting CSV to JSON in Node With a Few Lines of Code

Convert a CSV to JSON or NDJSON in Node with csv-parse, keep ZIP codes as text, and stream 2,000,000 rows at 120 MB instead of 1.6 GB.

8 min read
Summarize in AI
Flat illustration of a spreadsheet grid with a selected cell and a formula bar, with the JSON logo.

Converting a CSV to JSON is one of the shortest programs you will write in Node, and one of the easiest to get subtly wrong: every value arrives as a string, an identifier gets turned into a number, and the whole file lands in memory as one array. This article does the conversion in three lines, fixes the types, and then does it for 2,000,000 rows in constant memory. It uses csv-parse (documentation) for the reading and the JSON format as RFC 8259 defines it; the options of that parser are in its own article, and the stream basics are in the streams article.

Three lines

For a file that fits in memory, the sync API is enough:

tojson_small.mjsJavaScript
import fs from "node:fs";
import { parse } from "csv-parse/sync";

const text = fs.readFileSync("/batches/csv-0001-0200/lab/data/py/people.csv", "utf8");
const rows = parse(text, { columns: true });
console.log(JSON.stringify(rows.slice(0, 2), null, 2));
console.log("--- typed, with a cast that leaves zip alone");
const zips = parse(fs.readFileSync("/batches/csv-0001-0200/lab/data/py/zips-blank.csv", "utf8"), {
  columns: true,
  cast: (v, ctx) => (ctx.header || ctx.column === "zip" ? v : v === "" ? null : /^-?\d+(\.\d+)?$/.test(v) ? Number(v) : v),
});
console.log(JSON.stringify(zips));
console.log("--- cast: true");
console.log(JSON.stringify(parse(fs.readFileSync("/batches/csv-0001-0200/lab/data/py/zips-blank.csv", "utf8"), { columns: true, cast: true })));
Shell
node tojson_small.mjs
Text
[
  {
    "id": "1",
    "name": "Ana Silva",
    "city": "Lisbon",
    "note": "likes tea"
  },
  {
    "id": "2",
    "name": "Smith, Jane",
    "city": "Boston",
    "note": "said \"hello\""
  }
]
--- typed, with a cast that leaves zip alone
[{"id":1,"zip":"02134","state":"MA"},{"id":2,"zip":"","state":"CA"},{"id":3,"zip":"00501","state":"NY"}]
--- cast: true
[{"id":1,"zip":2134,"state":"MA"},{"id":2,"zip":"","state":"CA"},{"id":3,"zip":501,"state":"NY"}]

The first block is the result of parse(text, { columns: true }) on the people file: an array of objects keyed by the header, written with JSON.stringify(rows, null, 2). The quoted comma in "Smith, Jane" and the escaped quotes in "said \"hello\"" come through correctly, because the CSV parser did the unquoting and JSON.stringify did the JSON escaping. Look at the values: "id": "1". A CSV carries no types, so the parser gives you strings, and JSON that is meant for an API or a database will want numbers.

Types, on purpose

The second block shows a cast function that decides per column:

JavaScript
cast: (v, ctx) => (ctx.header || ctx.column === "zip" ? v : v === "" ? null : /^-?\d+(\.\d+)?$/.test(v) ? Number(v) : v)

Read it from the inside: the header row and the zip column are returned unchanged; an empty string becomes null; a value that is entirely digits (with an optional sign and fraction) becomes a number; anything else stays a string. The output is {"id":1,"zip":"02134","state":"MA"}, then {"id":2,"zip":"","state":"CA"} for the row with a blank ZIP, and {"id":3,"zip":"00501","state":"NY"}. The blank ZIP stayed "" and not null because the ZIP column is excluded from the rule; handle the blank explicitly if you want it null.

The third block is the shortcut, cast: true, and it is the trap: {"id":1,"zip":2134,...} and {"id":3,"zip":501,...}. The leading zeros of 02134 and 00501 are gone before JSON ever sees them, and a ZIP code that was text in the source is now a number in the API. The same happens in pandas and Papa Parse. Write the cast function and name the columns; it takes five minutes and it is the schema of the conversion. For money, think before converting at all: Number("103.10") is 103.1, and sums of such values accumulate error. Keeping an amount as the string "103.10" or as integer cents is a decision your JSON consumer should know about.

A big file: stream it

The same conversion on the 2,000,000 row file has three possible shapes, and they differ by an order of magnitude in memory. The script takes the mode as an argument:

tojson_big.mjsJavaScript
import fs from "node:fs";
import { pipeline } from "node:stream/promises";
import { Transform } from "node:stream";
import { parse } from "csv-parse";

const src = "/batches/csv-0001-0200/work/orders-2m.csv";
const mode = process.argv[2];
const t = performance.now();
let n = 0;
if (mode === "ndjson") {
  await pipeline(
    fs.createReadStream(src), parse({ columns: true }),
    new Transform({ writableObjectMode: true, transform(rec, _e, cb) { n++; cb(null, JSON.stringify(rec) + "\n"); } }),
    fs.createWriteStream("/tmp/orders.ndjson"));
} else if (mode === "array-stream") {
  await pipeline(
    fs.createReadStream(src), parse({ columns: true }),
    new Transform({
      writableObjectMode: true,
      transform(rec, _e, cb) { cb(null, (n++ === 0 ? "[\n" : ",\n") + JSON.stringify(rec)); },
      flush(cb) { cb(null, "\n]\n"); },
    }),
    fs.createWriteStream("/tmp/orders.json"));
} else if (mode === "array-memory") {
  const { parse: parseSync } = await import("csv-parse/sync");
  const rows = parseSync(fs.readFileSync(src), { columns: true });
  n = rows.length;
  fs.writeFileSync("/tmp/orders-mem.json", JSON.stringify(rows));
}
const mb = (p) => (fs.statSync(p).size / 1e6).toFixed(0);
const file = { ndjson: "/tmp/orders.ndjson", "array-stream": "/tmp/orders.json", "array-memory": "/tmp/orders-mem.json" }[mode];
console.log(`${mode.padEnd(13)} ${n} records -> ${mb(file)} MB in ${((performance.now() - t) / 1000).toFixed(1)}s, peak rss ${(process.resourceUsage().maxRSS / 1024).toFixed(0)} MB`);
Shell
node tojson_big.mjs ndjson
node tojson_big.mjs array-stream
node --max-old-space-size=8192 tojson_big.mjs array-memory
Text
ndjson        2000000 records -> 324 MB in 8.8s, peak rss 118 MB
array-stream  2000000 records -> 326 MB in 8.4s, peak rss 120 MB
array-memory  2000000 records -> 324 MB in 12.7s, peak rss 1614 MB
  • **ndjson**: the parser streams records into a Transform that writes one JSON.stringify(record) and a newline per record. 8.8 s, a peak of 118 MB and a 324 MB file. This is newline-delimited JSON, one object per line.

  • **array-stream**: the same, but the Transform writes a [ before the first record, a comma before each later one and ] at the end, so the output is one valid JSON array. 8.4 s, 120 MB, 326 MB. Streaming did not need the whole array.

  • **array-memory**: parse from csv-parse/sync on the whole file, then JSON.stringify(rows) in one call. 12.7 s and 1,614 MB, with --max-old-space-size=8192 set so that the heap limit would not interfere. It is the shortest code, and with the default heap limit a larger file would fail.

The two streaming versions use a tenth of the memory and are faster. Which output format to choose depends on the consumer. NDJSON can be read line by line, appended to, split and processed in parallel, and it is what log pipelines and many database importers expect. A single array is what most browsers' fetch(...).json() and most APIs expect, and it must be parsed whole. For a file of this size the consumer will want NDJSON.

Check the output as you would check any conversion: read it back with a different program.

Shell
head -c 300 /tmp/orders.ndjson | head -n 2
wc -l /tmp/orders.ndjson
node -e '
const fs=require("fs");const rl=require("readline").createInterface({input:fs.createReadStream("/tmp/orders.ndjson")});
let n=0,s=0;rl.on("line",l=>{const o=JSON.parse(l);n++;s+=Number(o.amount)});rl.on("close",()=>console.log("parsed back:",n,"records, amount total",s.toFixed(2)))'
Text
{"order_id":"1","customer_id":"22313","zip":"99154","amount":"103.1","created_at":"2025-11-22 13:08:24","status":"shipped","country":"GB","note":"rush order"}
{"order_id":"2","customer_id":"193489","zip":"34494","amount":"31.59","created_at":"2025-03-03 08:03:33","status":"failed","country":"BR","no2000000 /tmp/orders.ndjson
parsed back: 2000000 records, amount total 160174646.36

The first lines are the NDJSON: {"order_id":"1","customer_id":"22313","zip":"99154","amount":"103.1",...}. Every field is a string, as in the small example, because I used columns: true with no cast. wc -l counted 2,000,000 lines, and a line-by-line JSON.parse loop read back 2,000,000 records with an amount total of 160174646.36, matching the CSV. That is the test: a count and a sum, computed from the output by a program that did not write it.

The shorter way

For a file on disk, a database engine converts types and formats in one statement:

Shell
cd /batches/csv-0001-0200/work
time duckdb -c "COPY (SELECT * FROM 'orders-2m.csv') TO '/tmp/orders-dd.ndjson' (FORMAT json)"
ls -l /tmp/orders-dd.ndjson | awk '{printf "%.0f MB\n", $5/1e6}'
head -n 1 /tmp/orders-dd.ndjson
Text

real	0m1.698s
user	0m3.261s
sys	0m2.716s
313 MB
{"order_id":1,"customer_id":22313,"zip":"99154","amount":103.1,"created_at":"2025-11-22 13:08:24","status":"shipped","country":"GB","note":"rush order"}

DuckDB wrote 313 MB of NDJSON in 1.7 s. Its first line is {"order_id":1,"customer_id":22313,"zip":"99154","amount":103.1,"created_at":"2025-11-22 13:08:24",...}: integers are numbers, the amount is a number, and the ZIP is a string because DuckDB's sniffer keeps leading-zero values as text. That is faster by five times than the Node pipeline and chooses types for you, which is the argument for it when the job is a one-off conversion; the DuckDB guide covers the reading. The Node version wins when the conversion is part of a larger program, runs in a service without DuckDB, or needs per-record logic that SQL cannot express.

How to tell it worked

Count the records in the output and in the input. Compare a numeric total. Look at the types of the first record, especially the identifiers, and at a record with an empty field. Validate the JSON with a parser that did not produce it. Check that a value with a comma, a quote, an accent and a line break survives. And for a streamed array, run JSON.parse on a small sample of it first: a missing comma or bracket makes the whole file invalid, and the error only shows up at the consumer.

When to skip JavaScript for this

When it is a one-time conversion of a file you have on disk, use DuckDB or Miller (mlr --icsv --ojson cat file.csv) and spend the time on checking the result. When the data must be typed precisely, define the schema first, on the consumer's side, and convert to it. When the file is enormous and the output will be loaded into a database, skip the JSON and load the CSV directly, as in the database loading article. Use Node when the conversion lives in your application, and then stream it.

Also inScripting
Share

Keep reading

Get new articles by email

We send one email when a new article is published. No spam.

Unsubscribe in one click. No spam, ever.