Validating JSON With JSON Schema in Real Team Projects

JSON has no types beyond string, number, boolean, null, array and object, and no way to say that a field is required. A producer can rename a key, send a string where a number was promised, or leave out a field, and the JSON is still perfectly valid. The failure then shows up three systems later as a KeyError, a NULL in a report or a wrong total. JSON Schema is the standard way to write down what a document is allowed to look like, in a JSON file of its own, and check documents against it with a library in whichever language you use.

This guide covers what I would want to know before putting JSON Schema into a project: how to write a schema for real data and what it found when I ran it against 200,000 records, the keywords that do most of the work, the traps in formats, drafts and references, the cost of validating (the difference between the fast and slow way in Python is a factor of 500), how to check JSON that already sits in a database, how to generate a first schema from samples and how far to trust it, and how to change a schema without breaking its consumers. Each section points to a shorter article on the detail.

What a schema is for

A JSON Schema is a document that describes other documents. It can say that a value is an object with certain properties, that some of them are required, that one is an integer of at least 1, that another is one of four strings, that a list has at most five items and no repeats. It cannot compute anything or look at other records. It answers one question for one document: does this match.

That narrow job is useful in four places.

At the edge of a service. An HTTP handler that receives a body can check it against a schema and answer 400 with a list of problems before the data goes anywhere. The schema doubles as documentation: OpenAPI 3.1 uses JSON Schema for request and response bodies, so one description serves the validator and the docs.

At ingestion. A pipeline that reads files from partners or exports from other systems validates each record on the way in, writes the failures to a reject file with the reason, and loads the rest. The difference from loading and hoping is that the problem is reported once, next to the record, and not discovered by an analyst two weeks later.

In a database. When JSON is stored in a jsonb or JSON column, the database does not know what is inside. A constraint can enforce part of the shape, and a schema check in the application enforces the rest (checking a JSON column against a schema in PostgreSQL).

In configuration and CI. Config files, feature flags and fixtures are JSON or YAML. A schema check in CI turns a typo in a key into a failed build instead of a silently ignored setting. Editors that understand a schema also give autocompletion and inline errors, which is how most people first meet JSON Schema.

It is not a replacement for business rules. "The end date is after the start date" and "this id exists in another table" cannot be expressed. A schema checks shape, and the rest belongs in code or in the database.

A schema for a real record

Here is the schema I used for the event data. It is deliberately ordinary, with the keywords that show up in most real schemas:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://example.com/schemas/event.json",
  "title": "Event",
  "type": "object",
  "required": ["id", "ts", "user", "event"],
  "properties": {
    "id": { "type": "integer", "minimum": 1 },
    "ts": { "type": "string", "format": "date-time" },
    "user": { "$ref": "#/$defs/user" },
    "event": { "enum": ["view", "click", "buy", "refund"] },
    "amount": { "type": ["number", "null"], "minimum": 0 },
    "tags": {
      "type": "array",
      "items": { "type": "string", "pattern": "^[a-z]+$" },
      "uniqueItems": true,
      "maxItems": 5
    },
    "props": {
      "type": "object",
      "required": ["page"],
      "properties": {
        "page": { "type": "string", "pattern": "^/" },
        "ms": { "type": "integer", "minimum": 0 },
        "experiment": { "enum": ["a", "b"] }
      },
      "additionalProperties": false
    }
  },
  "$defs": {
    "user": {
      "type": "object",
      "required": ["id", "plan"],
      "properties": {
        "id": { "type": "integer" },
        "plan": { "enum": ["free", "pro", "team"] }
      },
      "additionalProperties": false
    }
  }
}

Reading it from the top: $schema names the draft of the specification the schema is written against, $id gives the schema a URI that other schemas can refer to, and type: object says the document is an object. required lists the properties that must be present. properties describes each property by its own sub-schema. $defs holds named pieces that can be reused with $ref, here the user object. The rest is the vocabulary: enum restricts to listed values, minimum and maxItems bound numbers and lengths, pattern is a regular expression, uniqueItems forbids repeated array elements, and additionalProperties: false rejects any key that is not listed. The walk through each keyword is in writing your first JSON Schema.

What it found in 200,000 records

I ran the schema over the whole event file with the Python jsonschema library, using a reused Draft 2020-12 validator with the format checker enabled:

import json
from jsonschema import Draft202012Validator, FormatChecker

schema = json.load(open("event.schema.json"))
Draft202012Validator.check_schema(schema)
validator = Draft202012Validator(schema, format_checker=FormatChecker())

for n, line in enumerate(open("events.ndjson"), 1):
    for error in validator.iter_errors(json.loads(line)):
        print(n, error.json_path, error.message)

It took 12.1 s, used 23 MB of memory, and reported exactly two kinds of problems:

33333  uniqueItems  tags      first at record 6: ['new', 'new'] has non-unique elements
200    type         props/ms  first at record 1000: 'slow' is not of type 'integer'

Neither was in the specification of the data, and both are the kind of finding a schema exists to produce. A third of the records had the same tag twice, which a generator that picked two tags with overlapping formulas had produced and which any real producer with a similar bug would produce too. And 200 records, one in a thousand, carried the text "slow" in the field that is otherwise a number of milliseconds. A numeric average over props.ms would have failed or, in a tool that coerces, skipped those rows silently. The schema moved both problems from "somewhere in a report" to a record number and a path.

The decision about what to do with such records belongs to the people who own the data. The schema's job is to make the question visible. In a pipeline the usual choices are to reject the record, to repair it (de-duplicate the tags, drop the string), or to relax the schema because the data is right and the schema was wrong. Relaxing is a legitimate answer, and then the schema documents the real shape.

The keywords that do most of the work

Most schemas are built from a small set of keywords. The groups:

Two behaviors in that table surprise people. First, every keyword applies only to values of the type it is about. minimum does nothing to a string, and minLength does nothing to a number. A schema with only {"minimum": 1} accepts the string "hello". That is why almost every property schema starts with type. Second, **properties does not make anything required.** A property listed in properties is checked if it is present and may be absent. The required array is what forces presence. Details and the null-versus-missing distinction are in required fields and optional fields in JSON Schema.

I tried a set of documents against the schema to see the messages the library gives (Python wording):

missing required    [$] 'user' is a required property
wrong type          [$.id] '1' is not of type 'integer'
float integer       OK                     (id: 1.0)
bool as integer     [$.id] True is not of type 'integer'
extra in user       [$.user] Additional properties are not allowed ('x' was unexpected)
bad enum            [$.event] 'hover' is not one of ['view', 'click', 'buy', 'refund']
negative amount     [$.amount] -1 is less than the minimum of 0
null amount         OK                     (type allows null)
props missing page  [$.props] 'page' is a required property
unknown top key     OK                     (no additionalProperties at top level)

Three of these are worth remembering. 1.0 passes as an integer, because JSON Schema defines an integer as a number with a zero fractional part, whatever the text looks like. A boolean does not pass as an integer, although in Python True == 1. And an unknown key at the top level passes, because additionalProperties was not set there: by default a schema is open, and unknown keys are fine. Closing an object is a choice with consequences for evolving it, which is covered under versions below.

Arrays and nested objects have their own rules for items, prefixItems and contains (validating arrays and nested objects with JSON Schema).

Formats, enums and patterns

enum is exact: the value must equal one of the listed values, with the same type. pattern is a regular expression in ECMA-262 syntax that matches anywhere in the string unless you anchor it, which is why the schema above writes ^[a-z]+$ and not [a-z]+. Without the anchors, "Ok1" would match because it contains a lowercase letter.

format is the keyword to treat carefully. It names a kind of string, such as date-time, email, uuid or uri, and in the current drafts it is an annotation by default: a validator is allowed to ignore it, and many do. My own test shows both ways to get it wrong. In Python, this document passed:

{"id": 1, "ts": "yesterday", "user": {"id": 1, "plan": "pro"}, "event": "view"}

I had passed format_checker=FormatChecker(), and the validator still raised nothing for "ts": "yesterday". Installing one extra package, rfc3339-validator, changed that: the same document then produced 'yesterday' is not a 'date-time'. So the library had skipped the check, with no warning, because the dependency for it was missing. In Node, Ajv behaves the other way: with the draft 2020-12 class and no ajv-formats, compiling the schema stopped with unknown format "date-time" ignored in schema at path "#/properties/ts", and after addFormats(ajv) the document failed with must match format "date-time".

The lesson is to test formats on purpose: put one deliberately bad value in a test and require the validator to reject it. Then you know the check is real. Enums, formats and patterns in more detail are in using JSON Schema enums, formats and patterns.

Drafts, and why $schema matters

JSON Schema has had several drafts (4, 6, 7, 2019-09, 2020-12), and the same keyword can behave differently between them. The $schema keyword says which one a schema follows. Differences I ran into in testing:

• In draft 7, a $ref hides its sibling keywords: they are ignored. In 2019-09 and later they apply. I wrote {"$ref": "#/$defs/s", "maxLength": 3} where s is a string. The 2020-12 validator rejected "abcdef"; the draft 7 validator accepted it.

• Reusable definitions are definitions in drafts up to 7 and $defs from 2019-09.

• Arrays of positional schemas are items: [...] in older drafts and prefixItems in 2020-12.

• dependencies was split into dependentRequired and dependentSchemas.

Libraries differ in what they support. Ajv's default class understands draft 7, and compiling my 2020-12 schema with it stopped with no schema with key or ref "https://json-schema.org/draft/2020-12/schema"; the fix is require("ajv/dist/2020"). fastjsonschema for Python supports drafts 4, 6 and 7 only. If you share a schema between languages, write it in the draft all of them support, which today is draft 7 for the widest reach, or 2020-12 if you control the toolchain.

Running a schema in Python

The library most people use is jsonschema. The shape of a correct use is: check the schema once, build one validator object, and call it for each document.

Draft202012Validator.check_schema(schema)          # is the schema itself valid?
v = Draft202012Validator(schema, format_checker=FormatChecker())

v.is_valid(doc)                 # bool
list(v.iter_errors(doc))        # every problem

Each error has a message, a json_path ($.user.plan), the validator keyword that failed and the schema fragment responsible. When there are many errors, best_match picks the most relevant one; for a document missing user, with a bad date and a bad enum, it chose 'user' is a required property, the shallowest. The full walk-through for Python is in validating JSON in Python with a schema, and the library overview is in validating JSON with jsonschema in Python.

Running a schema in Node

In Node the common validator is Ajv, which compiles the schema to a JavaScript function:

const Ajv2020 = require("ajv/dist/2020");
const addFormats = require("ajv-formats");

const ajv = new Ajv2020({ allErrors: true });
addFormats(ajv);
const validate = ajv.compile(schema);

if (!validate(doc)) console.log(validate.errors);

Errors are objects with instancePath (/user/plan), keyword and message. Ajv is strict by default: an unknown keyword (colour) throws strict mode: unknown keyword: "colour" and an unknown format throws as above. That is deliberate, since it turns schema typos into failures at startup. Without allErrors: true it stops at the first problem, which is faster and reports one error. More in validating JSON in Node with Ajv.

What validation costs

On the 20,000 first records of the event file I timed four ways to do the same check:

The 40-fold gap between the first two rows comes from what validate() does on every call: it checks the schema itself, builds a validator, and only then validates the document. Calling it in a loop pays that cost 20,000 times. The same loop with a validator object built once finished in 1.09 s. fastjsonschema generates Python code from the schema and ran 12 times faster again, at the price of lacking draft 2020-12 and producing less detailed errors (data must contain ['event', 'ts', 'user'] properties). Over the whole 200,000 line file it took 1.4 s and flagged 33,467 invalid records.

In Node, Ajv ran the full 200,000 records in 0.19 s with allErrors on and 0.13 s with it off, after 0.23 s to parse the lines. Both Ajv and fastjsonschema found 33,467 invalid records, the same count, which is a useful cross-check between libraries. The records with both problems explain why 33,467 is less than 33,333 plus 200.

Put together: for a service, build the validator at startup and call it per request. For a bulk job in Python, prefer fastjsonschema if the draft is old enough, and for a heavy workload consider validating in Node or in the database with a different tool.

Closed objects and composition

additionalProperties: false is the keyword people reach for to make a schema strict, and it has a catch that only shows up when you combine schemas. It looks only at the properties and patternProperties next to it in the same schema object, and knows nothing about what an allOf branch declared. I tested a schema that extends a base with allOf and then closes the result:

allOf + additionalProperties: false   {"id":1,"name":"x"}  ->  Additional properties are not allowed ('id' was unexpected)
allOf + unevaluatedProperties: false  {"id":1,"name":"x"}  ->  valid
allOf + unevaluatedProperties: false  {"id":1,"zzz":1}     ->  Unevaluated properties are not allowed ('zzz' was unexpected)

The base declares id, the extension declares name, and additionalProperties: false in the extension rejects the base's own id as unexpected, because the extension's properties does not list it. unevaluatedProperties (draft 2019-09 and later) fixes this by considering every property that any applicable subschema has evaluated, so the combined schema accepts both and rejects only zzz. If you build schemas from parts with allOf, $ref or conditional branches and want them closed, use unevaluatedProperties. In draft 7 and earlier the only workaround is to repeat every property in the closing schema, which is why many older schemas leave objects open.

Reusing pieces with $ref

A schema that repeats the same address object in billing and shipping gets long and drifts. $ref points at a definition, either inside the document (#/$defs/user) or in another schema by its $id. I put an address schema in its own file and referred to it from an order schema as "$ref": "address.json". Resolution is relative to the $id of the referring schema. Current Python libraries do not download remote references on their own; without the address schema registered, validation stopped with Unresolvable: address.json, and once I registered it the errors came out with paths into the right place ($.shipping.zip 'ab' does not match '^[0-9]{5}$'). A pointer to nowhere gives a different error, PointerToNowhere: '/$defs/nope' does not exist.

References can be recursive, which is how trees and nested comments are described. A node definition that has children as an array of node validated a three level tree and reported the wrong value at $.children[0].children[0].name. A document nested 2,000 levels deep made the Python validator raise RecursionError, which matters if the input is untrusted: limit the depth before you validate. The syntax and the traps are in reusing definitions with $ref in JSON Schema.

Generating a first schema from data

Writing a schema by hand for an unfamiliar file is slow, and tools can draft one from samples. I used genson on the first 999 and then the first 1,000 event records. The inferred props.ms was {"type": "integer"} for 999 records and {"type": ["integer", "string"]} once the 1,000th, which holds "slow", was included. The tool records what it has seen, so the schema describes the sample, not the intent.

The generated schema also contained none of the constraints that make a schema valuable. event was a plain string and not an enum, ts had no format, tags had no uniqueItems, every key that appeared in all sampled records was listed as required (including amount, which was always present, though null), and $schema was a generic http://json-schema.org/schema#. Treat generated output as a starting point, delete what is accidental, and add the rules you know. That workflow is in generating a JSON Schema from sample data, and the JSON schema generator does the same in the browser for a single document.

Checking JSON that lives in a database

A jsonb column accepts any JSON. PostgreSQL has no built-in JSON Schema validator, but a CHECK constraint can enforce the shape that matters most. I built one on a table of 200,000 events:

ALTER TABLE evc ADD CONSTRAINT evc_shape CHECK (
  jsonb_typeof(doc) = 'object'
  AND doc ?& array['id','ts','user','event']
  AND jsonb_typeof(doc->'id') = 'number'
  AND doc->>'event' IN ('view','click','buy','refund')
  AND doc#>>'{user,plan}' IN ('free','pro','team')
  AND coalesce(jsonb_typeof(doc#>'{props,ms}'), 'number') = 'number'
) NOT VALID;

NOT VALID makes the constraint apply to new rows immediately without scanning the existing ones. Inserting {"id": 1, "event": "view"} then failed with new row for relation "evc" violates check constraint "evc_shape", and so did an array and a document with a string id. Running ALTER TABLE evc VALIDATE CONSTRAINT evc_shape failed with check constraint "evc_shape" of relation "evc" is violated by some row, because of the 200 rows with "slow". A query for the opposite condition found them (ids 1000, 2000, 3000 and so on). You then repair those rows and validate again. The same approach with jsonpath predicates (doc @@ '$.id > 0 && ...') is more compact. The complete treatment, including what I could not do without an extension, is in checking a JSON column against a schema in PostgreSQL.

Changing a schema over time

A schema is a contract between a producer and consumers, and contracts change. To see which changes are safe, I validated four kinds of documents against five versions of a small schema:

Read the columns as consumers. A consumer holding the v1 schema rejects any document with a new field, because v1 is closed. That is the cost of additionalProperties: false: every addition by a producer is a breaking change for old consumers. A schema that stays open accepts unknown keys, and a producer can add fields safely. Making a field required breaks every old document, so it is only safe when old documents are migrated first. Adding an enum value breaks consumers who have not learned the value yet. The practical rules, with a numbering scheme, are in versioning a JSON Schema without breaking consumers.

Turning errors into something people can act on

The default output of a validator is written for a developer. Before it goes to an API client or a data owner, three things need attention.

Quantity. A bad array of 1,000 strings against items: {"type": "integer"} produced 1,000 separate errors, one per element. Return the first handful and a count (and 995 more), or the reply becomes larger than the request. Ajv's allErrors: false and Python's next(validator.iter_errors(doc), None) both stop at the first failure when that is enough.

Location. Python gives error.json_path ($.user.plan) and Ajv gives instancePath (/user/plan), which is a JSON Pointer without the $. Pick one format for your API and convert, because clients should not have to know which library you used. JSON Pointer is defined in RFC 6901 and is the more common choice in APIs.

Content. The messages quote the offending value. A maxLength failure on a password field printed 'hunter2hunter2' is too long. Echoing a rejected value back to the client is acceptable for an enum and wrong for a secret, and writing it to a log is worse. Build the response from error.json_path and error.validator (the keyword) and write your own text, and never log the instance of fields that may hold credentials.

For a pipeline, record the keyword and path, not the message, so rejects can be grouped. In the event data, grouping by (keyword, path) reduced 33,533 errors to two lines, which is what an owner can act on.

Testing the schema itself

A schema is code, and it is easy to get wrong in both directions: too strict, so valid documents fail, or too loose, so nothing is caught. Keep two folders next to it, valid/ and invalid/, with a handful of small documents each, and run them in CI. The invalid ones should each break exactly one rule, named in the file name (missing-user.json, bad-plan.json), and the test should assert the keyword and path of the failure, not just that something failed. That catches the silent-format problem above, because an invalid/bad-timestamp.json that passes is a failing test. Add a case for every bug a schema change was meant to fix. Run check_schema (or Ajv's compile) on the schema in the same job: it rejects a schema with a typo in a keyword name or an enum that is not an array.

Where to put validation

The best location depends on who can fix the problem. Validate where the data enters, so that the producer gets the error. Validate again before an expensive or irreversible step. Do not validate on every internal call, since the second check of the same document buys nothing. In practice:

1. At an API boundary, return 400 with the error list, including the path of each failure.

2. At file ingestion, write rejects to a side file with the line number and message, and carry on with the good records (finding one bad line in a huge JSON Lines file deals with lines that are not even JSON).

3. In the database, keep the constraints to what is cheap and stable, and let the application do the full schema check.

4. In CI, validate configuration and fixtures on every commit.

Log the failure counts, not just the failures. A sudden rise in rejects is how a producer's silent change shows up, and a schema is the cheapest detector for it.

When JSON Schema is the wrong tool

You need cross-field or cross-record rules. "The end is after the start" and "this id is unique across the file" are outside what a schema can say. Put them in code or a constraint.

The data is not JSON yet. If the real source is a CSV, a form or a database row, validate that structure at the source with the tools it has, before it is converted. JSON Schema validates the JSON you produce, after the fact.

The shape is genuinely open. A bag of arbitrary user attributes should not get a closed schema. Validate the parts you rely on and leave the rest.

You need typed code, not just a check. If the goal is typed objects in a language, a model library gives you both validation and types in one step (parsing JSON into typed objects with Pydantic), and JSON Schema can be exported from the model.

The specification is published at json-schema.org, and the Python library's reference is at python-jsonschema.readthedocs.io. The Ajv options are documented at ajv.js.org. To look at a document before writing rules for it, use the JSON formatter.