Parquet / Avro Header Decoder
Read Parquet and Avro file headers, schemas and metadata.
This bench is being wired up
The parquet-avro-decoder decoder is on the roadmap. The heavier binary and cryptographic parsers ship after the core text and token decoders — all of which are already live and working.
Browse the live decodersInput is processed on your device. Nothing you paste is sent anywhere.
What is the Parquet / Avro Header Decoder?
A Parquet/Avro decoder reads the metadata sections of these two big-data file formats — Parquet's footer schema and row-group statistics, Avro's JSON schema header — to reveal a file's structure without scanning its data.
Parquet is columnar: values from the same column are stored together, which compresses far better and lets a query engine read only the columns it needs. The schema and per-row-group statistics live in a footer at the end of the file, immediately before the PAR1 magic, so metadata is read with a single seek to the tail.
Avro is row-oriented and self-describing in a different way: the writer's schema is embedded as JSON in the file header, followed by blocks of records separated by a sync marker. That design makes Avro good for streaming and for schema evolution, where readers reconcile their expected schema against the writer's.
In practice you reach for the metadata when a pipeline breaks. Parquet's row-group min/max statistics explain why predicate pushdown did or didn't skip data, and Avro's embedded schema explains why a consumer can't read what a producer wrote.
Parquet / Avro Header Decoder at a glance
- Parquet magic
- PAR1 at both start and end of file
- Parquet layout
- Columnar, row groups, footer holds the schema
- Avro header
- Obj1 magic plus a JSON schema
- Avro layout
- Row-oriented blocks with a 16-byte sync marker
How to use the Parquet / Avro Header Decoder
- 1Load the file — only the header and footer regions need to be read.
- 2Review the recovered schema: field names, types and nesting.
- 3For Parquet, check row-group counts and column statistics.
When you would reach for it
- Confirming the schema of a Parquet file before writing a query against it.
- Diagnosing an Avro schema-evolution mismatch between producer and consumer.
- Checking row-group sizing when a Parquet dataset queries slowly.
Parquet / Avro Header Decoder — frequently asked questions
Last reviewed · questions or corrections: contact@decoder.tools