Repolode
Submit
mlcommons

mlcommons/croissant

A high-level metadata format for machine learning datasets, built on schema.org JSON-LD, that describes files, structure and ML semantics in one file.

View on GitHubWebsite
Stars
915
Forks
128
Open issues
226
Last commit
17 days ago

What is it?

Croissant is a metadata format for machine learning datasets. It combines four layers in a single JSON-LD file: dataset metadata including responsible-ML information, descriptions of the resource files, the structure that arranges raw data into records and fields, and the ML semantics that describe how the data is typically used. It is designed to work with existing datasets rather than replace them, so that they become easier to find, load and support with tools.

The format extends schema.org's Dataset vocabulary, which makes Croissant files indexable by web search engines. The Python package mlcroissant can read a Croissant file from a local path or URL, print its metadata and iterate over records. Several platforms already expose Croissant metadata, including Kaggle, Hugging Face, OpenML and Dataverse, and TensorFlow Datasets includes a CroissantBuilder for loading them.

The specification is developed by a Task Force of the MLCommons Association's Datasets Working Group. The implementation and examples are Apache 2.0 licensed, and the specification is published under CC BY-ND 4.0.

Key features

  • 01Combines metadata, resource files, record structure and ML semantics in one JSON-LD file.
  • 02Builds on schema.org Dataset vocabulary so datasets stay searchable on the web.
  • 03Provides the mlcroissant Python package to load and inspect Croissant datasets.
  • 04Supports loading from local files or remote URLs, including Hugging Face API endpoints.
  • 05Includes a CroissantBuilder integration for TensorFlow Datasets.

Use cases

  • Publishing a machine learning dataset with standard, machine-readable metadata.
  • Loading a dataset described by a Croissant file into a TensorFlow or PyTorch pipeline.
  • Making datasets discoverable through search engines and dataset catalogs.
  • Building tools that parse or generate dataset metadata in a common format.

Quick start

  1. 1

    Install the mlcroissant Python package (requires Python 3.10+)

    pip install mlcroissant
  2. 2

    Load an example dataset and print its metadata and records

Things to know

  • Croissant is under community development, so the mlcroissant API may change.
  • Loading some datasets in the example workflow depends on TensorFlow Datasets, which is an extra dependency.

Topics

#datasets#metadata#json-ld#schema-org#machine-learning#data-format

FAQ

What is croissant?
A high-level metadata format for machine learning datasets, built on schema.org JSON-LD, that describes files, structure and ML semantics in one file. Croissant is a metadata format for machine learning datasets. It combines four layers in a single JSON-LD file: dataset metadata including responsible-ML information, descriptions of the resource files, the structure that arranges raw data into records and fields, and the ML semantics that describe how the data is typically used. It is designed to work with existing datasets rather than replace them, so that they become easier to find, load and support with tools.
Is croissant free to use? What license does it have?
croissant is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is croissant still maintained?
Actively maintained. The last commit to croissant was on Oct 5, 2026.
What can I use croissant for?
Publishing a machine learning dataset with standard, machine-readable metadata. Loading a dataset described by a Croissant file into a TensorFlow or PyTorch pipeline. Making datasets discoverable through search engines and dataset catalogs. Building tools that parse or generate dataset metadata in a common format.
How do I install croissant?
Install the mlcroissant Python package (requires Python 3.10+): pip install mlcroissant Load an example dataset and print its metadata and records
Where can I get croissant?
The source code and installation instructions are on GitHub: https://github.com/mlcommons/croissant

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects