A Parquet/Avro decoder reads the metadata sections of these two big-data file formats — Parquet's footer schema and row-group statistics, Avro's JSON schema header — to reveal a file's structure without scanning its data.
What is the Parquet / Avro Header Decoder?
Parquet is columnar: values from the same column are stored together, which compresses far better and lets a query engine read only the columns it needs. The schema and per-row-group statistics live in a footer at the end of the file, immediately before the PAR1 magic, so metadata is read with a single seek to the tail.
Avro is row-oriented and self-describing in a different way: the writer's schema is embedded as JSON in the file header, followed by blocks of records separated by a sync marker. That design makes Avro good for streaming and for schema evolution, where readers reconcile their expected schema against the writer's.
In practice you reach for the metadata when a pipeline breaks. Parquet's row-group min/max statistics explain why predicate pushdown did or didn't skip data, and Avro's embedded schema explains why a consumer can't read what a producer wrote.
How to use the Parquet / Avro Header Decoder
- Load the file — only the header and footer regions need to be read.
- Review the recovered schema: field names, types and nesting.
- For Parquet, check row-group counts and column statistics.
When to use it
- Confirming the schema of a Parquet file before writing a query against it.
- Diagnosing an Avro schema-evolution mismatch between producer and consumer.
- Checking row-group sizing when a Parquet dataset queries slowly.
Parquet / Avro Header Decoder at a glance
| Parquet magic | PAR1 at both start and end of file |
| Parquet layout | Columnar, row groups, footer holds the schema |
| Avro header | Obj1 magic plus a JSON schema |
| Avro layout | Row-oriented blocks with a 16-byte sync marker |
Frequently asked questions
Why is Parquet's schema at the end of the file?
Because the writer only knows final statistics after writing all the data. Putting the footer last lets it be written in a single pass, and readers just seek to the end to get everything they need.
When should I use Avro instead of Parquet?
Avro for streaming and record-at-a-time writes, where whole rows are read together. Parquet for analytical queries that read a few columns from many rows.
Can I read a Parquet file's data here?
This tool focuses on the metadata and schema. Reading the encoded column chunks requires a full Parquet reader — but the schema is usually what you actually needed.
Last reviewed · found a mistake? contact@decoder.tools · view as markdown