JSON
Checking a JSON Column Against a Schema in PostgreSQL
Enforce the shape of a PostgreSQL jsonb column with CHECK constraints, NOT VALID and jsonpath, including the NULL trap that lets a string id pass.
8 min read
Tag
quality
52 posts
JSON
Generating a JSON Schema From Sample Data and Fixing It
Draft a JSON Schema from sample documents with genson, see what it infers and what it misses, and turn the draft into a contract with DuckDB as a cross-check.
7 min read
YAML
Defining Data Contracts in YAML That Fail the Build
Write a data contract as a YAML file, check a table against it with DuckDB, and fail the job on nulls, duplicates, bad values and drift, with tested output.
7 min read
JSON
Reusing Definitions With ref in JSON Schema Documents
Reuse schema pieces with $ref and $defs: local pointers, references between files by $id, recursive trees, and resolver and draft traps in Python and Node.
7 min read
JSON
Using JSON Schema Enums Formats and Patterns Correctly
Where Python and Node disagree on JSON Schema formats and patterns: email, uri, hostname, digits, trailing newlines, plus exact enum and const behavior.
6 min read
JSON
Validating Arrays and Nested Objects With JSON Schema
Validate arrays of objects, tuples and deep nesting with JSON Schema: items, prefixItems, contains, uniqueItems and error paths, tested in Python.
6 min read
YAML
Finding Duplicate Keys Across a Config Repository Reliably
Find YAML keys repeated inside a file and keys that differ across environment files, with ruamel.yaml, yamllint and a flatten pass over a repository.
6 min read
JSON
Required Fields and Optional Fields in JSON Schema
How required, null, empty strings and defaults differ in JSON Schema, with test results for dependentRequired and if-then rules for conditional fields.
6 min read
JSON
Writing Your First JSON Schema From a Real Document
Build a JSON Schema from a user profile document one keyword at a time, run it in Python after each step, and catch typos that a validator ignores silently.
7 min read
JSONGuide
Validating JSON With JSON Schema in Real Team Projects
A working guide to JSON Schema: write a schema, run it in Python and Node, handle formats, refs and versions, and check JSON stored in a database.
19 min read
CSV
Writing Data Quality Checks for CSV Files in dbt and DuckDB
Read a CSV in a dbt model with DuckDB and test it with unique, accepted_values, relationships and SQL tests: 6 of 8 failed on a faulty file.
12 min read
CSV
Making a CSV Contract With Your Data Provider in Writing
What to agree with whoever sends you CSV files, as a document plus a manifest your code can check: six deliveries, six different rejections, 193,708 rows.
12 min read