Fixing Garbled Characters in a Downloaded CSV From a Web App

The export works. The file downloads, the rows are right, and on a customer's machine Zoë reads Zoë. This is among the most common CSV bug reports a web app gets, and the cause is almost never in the CSV: it is in what the program that opens it assumes about the bytes. This article shows what the garbling looks like and what each pattern means, checks the bytes and headers a server actually sends, and gives the fix for Express and for a download built in the browser. The browser side is in the download article, and the same symptom on the data side in the strange characters article.

Read the garbling

The pattern tells you what happened.

• **Zoë, José, São**: valid UTF-8 bytes read by something that assumed a single-byte Windows code page. Each accented character became two characters, starting with Ã or Â. The data is fine, and the reader is wrong. This is the classic Excel symptom.

• **Zo�, Jos�**: bytes that are not valid UTF-8, read as UTF-8. The � (U+FFFD) is a replacement for a byte the decoder could not use. The original byte is gone from the text you see, so the file was probably Windows-1252 or Latin-1, or the bytes were damaged earlier.

• **Zo?, Jos?**: a literal question mark in the bytes. The character was lost where it was encoded, usually by a conversion to a charset that could not represent it (ASCII, or a database column with the wrong charset). Nothing at the reading end can restore it.

• **Z’-style chains**: text that was decoded wrongly and encoded again, more than once. The fix is at the source, and the damage may already be in your database.

You can reproduce the first pattern in a browser console with the file's own bytes:

// in a browser: the UTF-8 bytes of the file, decoded two ways
const bytes = new TextEncoder().encode("name,city\nZoë,Zürich\nJosé,São Paulo\n");
new TextDecoder("windows-1252").decode(bytes).split("\n")
new TextDecoder("utf-8").decode(bytes).split("\n")

// windows-1252
[ "name,city", "Zoë,Zürich", "José,São Paulo", "" ]
// utf-8
[ "name,city", "Zoë,Zürich", "José,São Paulo", "" ]

The UTF-8 bytes for Zoë,Zürich decoded as windows-1252 print Zoë,Zürich, and decoded as UTF-8 print Zoë,Zürich. Same bytes, two readings. The ë pair is the two bytes c3 ab of ë, each shown as its own character.

What the server sends

Three things are in the response: the bytes, the Content-Type header with its charset, and optionally a byte order mark at the start. I served the same text, Zoë and José rows, from five Express and plain-Node routes and read back each response's header and first bytes:

import http from "node:http";
import express from "express";

const csv = "name,city\r\nZoë,Zürich\r\nJosé,São Paulo\r\n";
const app = express();
app.get("/express-send", (req, res) => { res.type("text/csv"); res.send(csv); });
app.get("/express-send-buffer", (req, res) => { res.type("text/csv"); res.send(Buffer.from(csv)); });
app.get("/express-bom", (req, res) => { res.type("text/csv; charset=utf-8"); res.attachment("people.csv"); res.send("\ufeff" + csv); });
app.get("/express-latin1", (req, res) => { res.type("text/csv; charset=utf-8"); res.send(Buffer.from(csv, "latin1")); });
app.get("/express-accent-name", (req, res) => { res.attachment("café résumé.csv"); res.send(csv); });
app.listen(3111, () => console.log("express on 3111"));

http.createServer((req, res) => {
  res.writeHead(200, { "Content-Type": "text/csv" });
  res.end(csv);
}).listen(3112, () => console.log("raw http on 3112"));

node client.mjs

plain node http              | text/csv                   | 6e 61 6d 65 2c 63 69 74
express res.send(string)     | text/csv; charset=utf-8    | 6e 61 6d 65 2c 63 69 74
express res.send(Buffer)     | text/csv; charset=utf-8    | 6e 61 6d 65 2c 63 69 74
express + BOM + attachment   | text/csv; charset=utf-8    | ef bb bf 6e 61 6d 65 2c
                             | content-disposition: attachment; filename=people.csv
latin1 bytes, utf-8 label    | text/csv; charset=utf-8    | 6e 61 6d 65 2c 63 69 74
                             | decoded as utf-8: "Zo�,Z�rich"
attachment with accents      | text/csv; charset=utf-8    | 6e 61 6d 65 2c 63 69 74
                             | content-disposition: attachment; filename="caf? r?sum?.csv"; filename*=UTF-8''caf%C3%A9%20r%C3%A9sum%C3%A9.csv

• **Plain Node http** with Content-Type: text/csv: the header stayed text/csv with no charset. A client then has to guess, and the guesses differ.

• **Express res.send(string)** after res.type("text/csv"): the header became text/csv; charset=utf-8. Express adds the charset for text types. Sending a Buffer gave the same header, so the label is attached either way, which means a correct label does not prove the bytes are UTF-8.

• **Express with a BOM and res.attachment**: the first bytes were ef bb bf 6e 61 6d 65 2c, the BOM followed by name,, and the Content-Disposition was attachment; filename=people.csv.

• **A latin1 buffer labeled charset=utf-8**: the header said UTF-8, the bytes were Latin-1, and decoding them as UTF-8 gave Zo�,Z�rich. This is a real bug pattern: a database driver, a template or a Buffer.from(text, "latin1") upstream produced the wrong bytes, and the header only describes them. Check the bytes with a hex dump, not the label.

• An accented file name: res.attachment("café résumé.csv") produced filename="caf? r?sum?.csv"; filename*=UTF-8''caf%C3%A9%20r%C3%A9sum%C3%A9.csv. The plain filename is an ASCII fallback with question marks, and filename* carries the real name encoded as RFC 6266 describes. Modern browsers use filename*. If you set the header by hand, filename="café.csv" with a raw accented character is the way to get a garbled file name too.

The charset header does not control Excel

Here is the part I cannot test, and why it matters. When a user opens a downloaded .csv by double-clicking it, no browser and no web server is involved any more: the operating system hands the file to Excel, which sees bytes with no header. Excel on Windows is widely reported to decide the encoding from the bytes alone, and to treat a file without a byte order mark as being in the system's legacy code page. That is why a perfectly labeled UTF-8 response still opens as Zoë. The widely used fix is the BOM, which Excel reads as "this is UTF-8". It is a workaround for one program's behavior. Other programs treat the BOM as three odd characters at the start of the first header, as the BOM article shows, so I make it a decision about the audience.

To add it in Express, put the character in front of the text:

app.get("/export.csv", (req, res) => {
  res.type("text/csv; charset=utf-8");
  res.attachment("export.csv");
  res.send("" + csv);
});

That is the route tested above:  encodes as ef bb bf in UTF-8, and the response started with those three bytes. For a streamed response, write the BOM once as the first chunk, before the header row, and not per chunk. For a file built in the page, new Blob(["", csv], ...), as in the download article. If your users also use Excel on a Mac, or LibreOffice, or Google Sheets, test them separately; I have not.

The alternatives to a BOM

• Tell the user to import. Excel's Data tab can import text with an explicit encoding, 65001 UTF-8. It works and it asks a non-technical user to do something every time.

• Offer a second format. An .xlsx file carries its own encoding and never garbles, at the price of a library and a larger file. If the audience is "people with Excel", it may be the right product.

• Offer both CSVs. A "CSV (UTF-8)" and a "CSV for Excel" link say what they are, and the BOM version is the one for spreadsheets.

• Use UTF-16. Excel has historically opened UTF-16 text with a BOM and tab separators; I have not tested it and would not start there.

Checking the whole path

When a customer reports garbled text, do not guess. Get the file from the same endpoint, save it without opening it, and look at the bytes:

head -c 16 export.csv | xxd

ef bb bf first means a BOM is present. Then look at a byte you know: c3 ab for ë means UTF-8, eb alone means Latin-1 or Windows-1252, 3f means the character was replaced by a question mark before it got to you. Then ask what the opening program assumes. If the bytes are right and the display is wrong, the fix is a BOM or an import step. If the bytes are wrong, find the layer that encoded them: the database connection charset, the driver, a Buffer.from(..., "latin1"), a legacy library, a copy that was saved through a text editor in the wrong encoding.

How to tell it worked

Download from the real endpoint, in the real browser, and open the file in the program your users use, with a test row that has an accent, a non-Latin character, an emoji and a comma. Check the first bytes. Check the file name in the save dialog if it contains an accent. Run the same file through a script that parses it, so you find out if the BOM breaks it. And put the test row in the export's automated test, so that a change to a driver or a framework does not bring the garbling back.

What I could not test about Excel

I did not run Excel, and I did not test every client: the bytes and headers above are what a Node client received, and the behavior of Excel with them is what is widely reported, not what I observed. Excel's behavior also differs between versions and between Windows and Mac. If your users are mostly on one platform, make that your test platform, and keep the check of the bytes as the one part of the diagnosis that does not depend on anyone's software.