Handling CSV Files Full of Customer Data Under GDPR

Most of what a team does with customer data in CSV form is perfectly ordinary: a finance export, a mailing list, a file for a support vendor. The legal side of it arrives in the form of a question that is hard to answer if you have never tracked your files: where are all the copies? A person asks you to delete their data, and the database is easy; the 14 CSVs that left in the last two years are the problem. This article is about the engineering habits that make the GDPR manageable for files, in the order they pay back. It is not legal advice, and it assumes data about people in the EU or UK, where the rules apply. The broader risk map is in CSV security risks that catch teams out.

A file is processing

The regulation does not care about formats. Personal data is any information relating to an identifiable person, and processing includes storing, copying, sending and deleting. A CSV with names and email addresses is personal data, and every export of it is a new instance, with its own location, its own readers and its own life. The database table has a controller, a purpose and access controls. The export starts with none of those, and you have to attach them on purpose.

Three articles of the regulation shape the engineering. Article 5 lists the principles: lawfulness, purpose limitation, data minimization, accuracy, storage limitation, and integrity and confidentiality. Article 17 gives people the right to erasure in many cases, and Articles 15 to 20 give access and portability. Article 32 requires appropriate technical and organizational security measures. Read them in the regulation text; they are shorter than you expect.

Minimize at the query

Data minimization is the principle most easily met in code. Export the columns the recipient needs, and nothing else. A SELECT * into a file is an unreviewed decision to share every column the table has now and every one it gets later. A named list is a decision someone made once:

COPY (
  SELECT customer_ref, plan, region, signup_month, mrr
  FROM customers
) TO 'plan-report.csv' (HEADER);

Compare that with a file that includes the date of birth, the home address and the support notes, because they happened to be in the table. The first can be defended if a regulator asks why the recipient had it. The second cannot. Stripping personal data from a CSV before sharing it works through the generalization and tokenization steps.

Give every file an owner, a purpose and a date

The storage limitation principle means you keep personal data no longer than needed. For a file, that is a date. The simplest mechanism that works is a register: a table, a spreadsheet or a wiki page with one row per export. The columns are enough to answer the questions a request will raise:

None of this is exotic. It turns "where are the copies" from an investigation into a query. Automate it where you can: an export job that writes its own row to the register, with the deletion date computed from a retention setting, is better than one that relies on someone remembering to fill in a form. A scheduled job that deletes files past their date and records the deletion closes the loop, and it is a handful of lines against a folder.

Know how to find a person in a file

When someone asks for access or erasure, the work is to find their rows. In the database, it is a key lookup. In files, it is a search through every export that could contain them, and it only works if the files carry a key you can search on. A token column (see hashing or masking columns in a CSV safely) lets you search for one person without the file holding their email, because you compute the token from their address and look for it. A file with no identifier at all, such as a pure aggregate, cannot contain a person's record and drops out of the problem.

Erasure in files is often done by regenerating, not by editing: delete the old file and produce a new one from the database after the erasure has been applied. That works for files that are reproducible from a query, which is another argument for exporting from code and not by hand. A file that someone built by hand from three sources, and that exists nowhere else, has to be edited and its owner told.

The search itself is a one-liner in DuckDB across a folder of exports, which is the reason to keep them in one place:

SELECT filename, count(*) AS rows
FROM read_csv('exports/*.csv', filename=true, all_varchar=true, union_by_name=true)
WHERE customer_ref = 'd9c7cb00ec215078'
GROUP BY filename;

It reads every file, adds the file name to each row and reports where the token appears. I have written this as an illustration of the pattern and did not run it for this article, so test it on your own files first (querying a folder of CSV files).

Protect the file where it lives and where it travels

Article 32 asks for security appropriate to the risk. For files, the practical floor is:

• Store exports on a system with named-user access, not on a laptop's downloads folder, and not on a shared link that anyone can open.

• Encrypt in transit (TLS for transfers, an encrypted channel for anything leaving the organization) and at rest (disk encryption on laptops and servers, server-side encryption in object storage).

• Do not attach exports to email. Sending a CSV by email safely, and better options covers what to do instead.

• Restrict the people who can run the export, and log each run with who and what.

Be wary of the quiet spread: a file synced to a cloud drive reaches every device that account is signed in to, and a file opened in a browser-based spreadsheet is copied to that provider's servers. These are processing steps by providers, and your contracts with them (data processing agreements) have to cover them.

Vendors and cross-border transfers

When you send a file to a vendor, such as a mailing service, an analytics firm or an accountant, you are sharing data with a processor, which needs a written agreement, and possibly a transfer outside the EU or UK, which needs a legal mechanism. The engineering side is small: send the minimized file, over a channel with access control, record it in the register and set the deletion date in the contract. The paperwork is for your legal team. The part engineers get wrong is the informal one: a one-off file to a contractor's personal email, which none of the agreements cover.

Breaches in files

A lost laptop with a CSV on it, a mis-sent email attachment and a public link left open are all personal data breaches if the data is at risk, and some must be reported to the supervisory authority within 72 hours of becoming aware. The register above is what lets you say what was in the file and who the data subjects were. Without it, the answer is "we are not sure", and that is the answer that turns an incident into a larger one.

What this does not decide for you

This article cannot tell you whether your processing is lawful. The legal basis (consent, contract, legitimate interests and others), whether you need a data protection impact assessment, whether a recipient is a processor or a joint controller, and what to do about transfers outside the EU are matters for your data protection officer or a lawyer, and they depend on your facts. It also does not cover other regimes: the UK GDPR is nearly identical but not the same text, and laws in California, Brazil and elsewhere use different rules and definitions.

What it gives you is the set of habits that make any of those answers possible: named columns, a register, a deletion date, a way to find a person in a file and a secure place to keep them. Teams that have those can answer a regulator, a customer or an auditor in a day. Teams that do not spend weeks. For the file format side, RFC 4180 is no help on privacy, and the regulation's text above is the primary source for the duties described.