Repolode
Submit
toon-format

toon-format/toon

A compact, human-readable encoding of JSON data designed to reduce token usage when passing structured data to LLM prompts.

View on GitHubWebsite
Stars
25,528
Forks
1,280
Open issues
2
Last commit
12 days ago

What is it?

TOON (Token-Oriented Object Notation) re-encodes the JSON data model in a format that combines YAML-style indentation for nested objects with CSV-style tabular rows for uniform arrays. Each array declares its length and field list in a header, so models can see how many rows to expect and which fields each row contains.

The project provides a TypeScript SDK with encode and decode functions, streaming APIs for large datasets, and a CLI that converts between JSON and TOON and reports estimated token savings. Encoding is lossless, so data can round-trip back to the original JSON. Official implementations and community ports target a shared spec with a conformance test suite.

TOON fits best with arrays of uniform objects. For deeply nested or non-uniform data, the project notes that compact JSON may be more efficient, and for purely flat tabular data CSV is smaller. The repository includes benchmarks comparing retrieval accuracy and token counts across JSON, YAML, XML, and CSV.

Key features

  • 01Encodes JSON objects and arrays into a YAML-like indented format with CSV-style tables for uniform data.
  • 02Array headers declare row count and field names, helping models detect truncated or malformed output.
  • 03Lossless round-trip between JSON and TOON.
  • 04TypeScript SDK with encode, decode, and streaming APIs.
  • 05CLI for converting files or stdin and showing token savings with --stats.
  • 06Supports comma, tab, and pipe delimiters.

Use cases

  • Passing large lists of uniform records, such as database query results, to an LLM while keeping prompt size down.
  • Converting JSON API responses into a compact form before inserting them into prompts.
  • Inspecting how many tokens a JSON payload would consume in TOON versus other formats.
  • Building LLM pipelines in TypeScript that need a reversible, validated data representation.

Quick start

  1. 1

    Try TOON on your own JSON data with npx, no install required

    cat data.json | npx @toon-format/cli --stats

Things to know

  • Best suited to uniform arrays or keyed objects; deeply nested or non-uniform data often gives better results with plain JSON.
  • The format is described as stable but still evolving, so details may change.

Topics

#llm#serialization#tokenization#data-format#json#prompt-engineering

FAQ

What is toon?
A compact, human-readable encoding of JSON data designed to reduce token usage when passing structured data to LLM prompts. TOON (Token-Oriented Object Notation) re-encodes the JSON data model in a format that combines YAML-style indentation for nested objects with CSV-style tabular rows for uniform arrays. Each array declares its length and field list in a header, so models can see how many rows to expect and which fields each row contains.
Is toon free to use? What license does it have?
toon is open source under the MIT license. Check the license terms before using it in commercial products.
Is toon still maintained?
Actively maintained. The last commit to toon was on Oct 9, 2026.
What can I use toon for?
Passing large lists of uniform records, such as database query results, to an LLM while keeping prompt size down. Converting JSON API responses into a compact form before inserting them into prompts. Inspecting how many tokens a JSON payload would consume in TOON versus other formats. Building LLM pipelines in TypeScript that need a reversible, validated data representation.
How do I install toon?
Try TOON on your own JSON data with npx, no install required: cat data.json | npx @toon-format/cli --stats
Where can I get toon?
The source code and installation instructions are on GitHub: https://github.com/toon-format/toon

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects