Accept comments and unquoted keys in a JSON config file
Parse a hand-edited config with a lenient JSON dialect, so comments, trailing commas and unquoted keys load instead of failing on one character.
How-to › Section 15
Parse, validate and transform data#
Reading the formats integrations depend on, building parsers from grammars, validating shapes at boundaries, and transforming nested data.
16 guides in 4 topics. Every guide compares its approaches and shows the output its commands printed.
- Parse config and data formats 6
- Write a grammar and parser 7
- Validate data shapes 1
- Transform nested data 2
Parse config and data formats 6 guides
Each format and when to choose it: JSON, JSONC, JSON5, jsonic (unquoted keys, comments, implicit top level, plugins, jsonic-cli), YAML, TOML, INI, CSV (RFC 4180, headers, custom separators, streaming, strict and lenient modes), XML, Markdown, CSS, JSON Lines, Atom and RSS feeds, protobuf IDL and Zon; error positions, streaming large files, when lenient input is the wrong choice.
Decide when lenient parsing is the wrong choice
Give a lenient parser to files people edit and a strict one to bodies programs send, keep the loaders apart, and let a test refuse a lenient import at the boundary.
Include one config file from another
Pull a shared fragment into a config file with @tabnas/multisource, track where each value came from, and refuse cycles and paths that leave the config directory.
Load a CSV into a DataFrame with the column types you want
Declare each column's type in pandas, Polars, or DuckDB, so ZIP codes and ids keep their zeros and a renamed or dropped column stops the load.
Parse one config format identically in TypeScript and Go
Compile one ABNF grammar with tabnas into a TypeScript and a Go parser, and run one corpus of accept and reject fixtures in both CI jobs to prove they agree.
Read a protobuf IDL file without running protoc
List the services, methods, and field numbers in a tree of .proto files from a script, and refuse the misspelled custom option that a plain parser accepts.
Write a grammar and parser 7 guides
Building a parser for your own format: ABNF (RFC 5234) compiled to a rule table with @tabnas/abnf (a state machine as data, not generated parser code), attaching behavior by rule name, extending an existing grammar without forking, inspecting with @tabnas/debug, railroad diagrams, the support libraries (expr, path, directive, hoover, multisource), an LSP and playground for your format, benchmarking with measure, and emitting GBNF for constrained decoding; against peggy, ohm, chevrotain, nearley, ANTLR, tree-sitter, Lark and hand-written recursive descent.
Add a literal to a JSON grammar without forking the parser
Add a date literal to a JSON parser with a tabnas lexer plugin instead of a copied grammar, and compare it with a peggy fork and a chevrotain subclass.
Attach parse actions to grammar rules by name
Keep the grammar as plain ABNF, bind behavior to the rule names the compiler assigns, and test that every handler still names a rule and fires after a rename.
Generate a parser for a custom format in Go
Build a Go parser for a nested config format with pigeon, participle, and the ANTLR Go target, and catch a PEG rule that doubles its work per level of nesting.
Recover from syntax errors and report several in one parse
Turn on error recovery in chevrotain, ANTLR, tree-sitter or Lark, collect every error in one pass, and test that recovery keeps its place inside nested brackets.
Remove left recursion from a PEG grammar
Rewrite a left-recursive rule so peggy accepts it, fold the action from the left to keep subtraction left-associative, and compare with ohm-js and nearley.
Write a Pratt parser for operator precedence in filters
Parse and, or, not, comparisons, and arithmetic with a table of binding powers and one loop, then measure it against layered grammar rules in peggy and Lark.
Write a recursive descent parser by hand in TypeScript
Parse a filter language with a tokenizer, a method per grammar rule, and peek and expect helpers, then measure it against the same grammar in peggy and chevrotain.
Validate data shapes 1 guide
Runtime validation of inputs, config and SDK responses, and exporting JSON Schema from a validator (for tool definitions and OpenAPI alike): Zod, Joi, Yup, Valibot, ArkType, io-ts, AJV with JSON Schema, Pydantic, go-playground/validator, and gubu (a schema that looks like the data, Required, Optional, Default, Min, Max, One, Exact, Check, Coerce, Email, Url, Uuid, JSON Schema export, TypeScript and Go).
Write a validator whose schema looks like the data
Write the schema as an example of an accepted value, with constructors for types and literals for defaults, so schema and sample read side by side.
Transform nested data 2 guides
Getting, setting, merging, walking, injecting and transforming nested structures, and flattening API responses for analysis: lodash, ramda, immer, jq, JSONPath, JSON Pointer, json-patch, deepmerge, pandas.json_normalize, and struct (getpath, setpath, merge, walk, inject, transform, validate, flatten, ported to 23 languages at parity).
Merge nested config objects with predictable precedence
Layer defaults, file, environment and flags with a deep merge so a later source overrides only the keys it sets, and clone first because merge mutates.
Read a value at a nested path without null checks
Read a deep value by a path held as a string, so a response mapper is a table of paths rather than a chain of optional accesses written out once per field.
Generate the client instead of writing it#
Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.