Generating a JSON Schema From Sample Data and Fixing It

When you meet a JSON file for the first time, typing a schema by hand is slow and error-prone. Tools that infer a schema from sample documents give you a first draft in seconds. The draft is useful for what it gets right, which is structure, and misleading for what it cannot know, which is every rule that lives in people's heads. I generated schemas from the event data at different sample sizes to see exactly which parts to keep and which to rewrite.

Generating with genson

genson reads documents and merges what it sees into one schema:

import json
from genson import SchemaBuilder

builder = SchemaBuilder(schema_uri="https://json-schema.org/draft/2020-12/schema")
for line in open("events.ndjson"):
    builder.add_object(json.loads(line))
schema = builder.to_schema()

For 200,000 records that took 1.2 s. The result (shortened) had a type for every field, nested properties for user and props, items for tags, and a required list at each object level:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "id":     { "type": "integer" },
    "ts":     { "type": "string" },
    "user":   { "type": "object", "properties": { "id": {"type": "integer"}, "plan": {"type": "string"} }, "required": ["id", "plan"] },
    "event":  { "type": "string" },
    "amount": { "type": ["null", "number"] },
    "tags":   { "type": "array", "items": { "type": "string" } },
    "props":  { "type": "object",
                "properties": { "page": {"type": "string"}, "ms": {"type": ["integer", "string"]}, "experiment": {"type": "string"} },
                "required": ["ms", "page"] }
  },
  "required": ["amount", "event", "id", "props", "tags", "ts", "user"]
}

By construction, the schema accepts every record that built it: validating all 200,000 against it produced zero rejects. That is the property a generated schema has and a useful one, as a baseline against which later files are compared. Its limit is the same property seen from the other side: it can never say "this record is wrong", because it was built from the records.

The schema_uri argument matters. Without it genson wrote http://json-schema.org/schema#, which names no particular draft. Pass the draft you intend to use so that the validator and editors interpret it as you expect (writing your first JSON Schema covers why the draft counts).

The sample decides the schema

Because the builder only knows what it has seen, the sample size changes the answer. I rebuilt the schema from the first few records:

records   props.experiment present   props.ms type
9         no                          integer
10        yes                         integer
999       yes                         integer
1,000     yes                         integer or string

The first record with an experiment key is the tenth, so a schema from nine records had no such property at all. It is not in required because later records may lack it, but it is missing from properties as well, and with the usual additionalProperties: false a record with it would be rejected. A schema from 999 records said ms is an integer, and the 1,000th record, with "slow", changed it to ["integer", "string"].

That last case deserves thought. The tool did not flag "slow" as an error. It recorded that strings occur there, which makes the generated schema accept the corrupt data. A hand-written schema with "ms": {"type": "integer"} rejected those 200 records and told us about them. The step from draft to contract is where you decide which of the variants the tool found are legitimate and which are bugs to keep out. Read every union type in the generated output as a question: is this really allowed?

Run the generator on as much data as you can, and on files from different periods and sources. A schema built from one day's export will miss the field that only appears at month end. Genson can merge: add_schema(existing) combines a saved schema with new documents, which is how you extend a draft with a new file. After adding the single record {"id": "abc"} to a schema built from five events, id became ["integer", "string"] and required shrank to ["id"], since no other key was in every record. One odd document can loosen a schema everywhere.

What the tool cannot infer

The generated draft is missing all of these, and they are most of the value of a schema:

• Enums. event was {"type": "string"}. Four values appear in 200,000 records, and the tool does not know whether a fifth is allowed. You do: write the enum.

• Formats. ts was a plain string. Add "format": "date-time", and test that your validator checks it (using JSON Schema enums, formats and patterns). The same holds for an email or a UUID: the tool saw strings.

• Limits and patterns. minimum, maxLength, pattern and maxItems are never produced.

• Uniqueness. tags has no uniqueItems. Validating with it added showed 33,333 records with the same tag twice, which a generated schema is blind to.

• Required for the right reason. amount was listed as required because it appeared, as null, in every record. Whether it must be present or merely happened to be, is a producer rule. Likewise props was required because every sampled record had one.

• Closed objects. No additionalProperties. Add it where you want unknown keys rejected.

• Descriptions and titles. The documentation part of a schema is yours to write.

Treat the generated output as a map of the data's structure, and rewrite it into a schema by going through that list for each property. It takes less time than writing from nothing, and it is much faster than discovering the structure by reading JSON.

Cross-checking with a database engine

A second tool that reads the same data gives you a check on the structure. DuckDB can print the structure of a JSON file as a type string:

SELECT json_group_structure(json)
FROM read_json_objects('events.ndjson', format = 'newline_delimited');

{"id":"UBIGINT","ts":"VARCHAR","user":{"id":"UBIGINT","plan":"VARCHAR"},"event":"VARCHAR",
 "amount":"DOUBLE","tags":["VARCHAR"],"props":{"page":"VARCHAR","ms":"JSON","experiment":"VARCHAR"}}

It agrees on the shape and disagrees on the type of ms, which it gave as JSON because of the mix of numbers and strings. That is another way to find the dirty field. DESCRIBE SELECT * FROM read_ndjson('events.ndjson') went further and inferred ts as timestamp with time zone and id as bigint. Those are database column types and not JSON Schema rules, but they are a useful hint when you are designing the table the data will land in (letting DuckDB infer a schema for nested JSON). Counting how often each key is present, with the jq command from the JSON cheat sheet, tells you whether a key deserves to be in required.

Draft to contract: a workflow

1. Generate from the largest, most varied sample you have, with the draft you intend to use.

2. Delete accidental required entries, and keep only fields the producer promises.

3. Read each union type and decide: keep it, or narrow it and treat the rest as errors.

4. Add enums, formats, patterns and limits from the producer's documentation, not from the sample.

5. Close the objects you control.

6. Validate the full sample against the result and read every rejection. Each is either a bug in the data (report it) or a rule that is too tight (relax it).

7. Commit the schema with a set of valid and invalid example documents.

Step 6 is where the generated schema earns its place. In this data, the hand-edited schema rejected 33,467 records and the reasons were two clear patterns, which is more than the generated one could say.

For a single document, the JSON schema generator in the browser is quicker than a script, with the same limits.

When a generated schema is the wrong starting point

There is an authoritative schema. If the API publishes an OpenAPI document or a JSON Schema, use it. Inferring from samples what a producer already documents only reproduces its bugs.

The sample is tiny or unrepresentative. Three records from a test environment will produce a schema that rejects the first production document. Wait for real data, or write by hand from the specification.

The data is dirty on purpose. For a log with free-form context, a precise inferred schema is a waste: validate the stable envelope and leave the rest open (validating JSON with JSON Schema in real projects).

You want typed code. Tools that generate classes or interfaces from samples (such as quicktype, which I did not run here) give types directly; the same cautions about the sample apply.

The builder's options are described in the genson documentation, and DuckDB's structure functions are in its JSON functions reference. Format the sample with the JSON formatter first if you are reading it by eye.