jalapenojson

jalapenojson 0.8

A document is a list of rows. Every row has the same columns. The column names, types and sizes are written once at the top, as text; the values follow, in binary.

The values are stored column by column: all of a column's values are written together, one after another. Every column declares its size in bytes, so a value carries no length, and the value in any row of any column is found by arithmetic, without reading anything before it.

Numbers, booleans and dates are stored in binary, little-endian, at a width their type fixes. Text is stored as UTF-8 at the size its column declares.

This describes format 0.8. A reader accepts 0.8 and refuses every other version.

A document is not meant to be read in a plain text editor. The header is text, so a person can read the column names and types there, and a viewer shows the rest as rows.

The examples below write the header as text and every byte after it as two hex digits. Text after two spaces on a hex line is a comment. python/check_spec_examples.py turns each example back into bytes and reads it, so each one is a real document.

Layout

<major>.<minor>\n
<schema line>\n          schema 1
<schema line>\n          schema 2, and as many more as it needs
<counts>\n
<columns>

The first four parts are the header. The columns are the body.

jalapenojson
0.8
#id:i:4 #cur:s:3
2
e9 03 00 00 ea 03 00 00  id: 1001, 1002
4e 4f 4b 45 55 52        cur: NOK, EUR

Two rows: {id: 1001, cur: "NOK"} and {id: 1002, cur: "EUR"}. The header is 23 bytes. id starts right after it, at byte 23, and is two 4-byte integers. cur starts right after id, at byte 31, and is two 3-byte strings.

Version line

The first line. It is exactly the three bytes 0.8: no leading zeros, no spaces, nothing else.

A reader that implements 0.8 refuses 0.7 and 0.9 alike. It does not guess.

Schema lines

One line per schema. A line is one or more column declarations separated by a single space:

#<name>:<type>:<size>     a sized column
#<name>:<number>          a list column

name is UTF-8, at least one byte long. It may not contain :, a space, a newline, or NUL; any other character is allowed, a tab included. A schema may not declare the same name twice. Two names are the same when their bytes are, so a and A are different names.

type is one of nine letters. A list column's type is instead the number of the schema its rows use. See Types and Nested lists.

size is an ASCII integer. It is the number of bytes each value of the column takes. See Types for the sizes each type allows.

A document declares one schema or more, as many as it needs. Schema 1 is the one the document's own rows use. The others exist to be referenced by list columns.

Counts

The line after the last schema line. It holds one ASCII integer, or more than one separated by a single space.

The first is the number of rows in the document. Then comes one number for each list column, in the order the list columns are declared: schema 1's from left to right, then schema 2's, and so on. Each is the number of rows in that list column's table; see Nested lists.

A document with no list column has one number on this line. A reader refuses a line with more or fewer numbers than the header calls for.

Columns

The body holds one table for the document's rows, then one table for each list column, in the same order as the counts. A table holds the columns of its schema in header order. A column holds its values for every row of the table, row 0 first.

A column starts where the previous column ends, or where the header ends for the first column. Nothing lies between two columns. A column holding no values takes no bytes.

column start = where the previous column ended, or where the header ended
column end   = column start + rows * size

The document ends where its last column ends. A reader refuses a document of any other length.

A column may start at any byte, so a column of numbers is not always aligned for its type. Reading its values one at a time needs no alignment. A reader that hands the column back as an array of its type, in a language where such an array must be aligned, copies the column when it is not.

Types

typemeaningsizestored as
iinteger1, 2, 4 or 8signed, two's complement
ffloat8IEEE 754 double
yboolean100 false, 01 true
sUTF-8 string1 or morethe text, then a break byte if it is shorter than the size
braw bytes1 or morethe bytes, exactly the size
ddate4signed 32-bit days since 1970-01-01
ttime of day8signed 64-bit microseconds since midnight
ndate and time, no zone8signed 64-bit microseconds since 1970-01-01 00:00:00
zdate and time with offset10signed 64-bit microseconds since 1970-01-01 00:00:00 UTC, then a signed 16-bit offset in minutes east of UTC
a numbera list of rows of the schema with that numbernonesee Nested lists

Every number in the body is little-endian.

A size that the table does not allow for its type is a malformed document: i:3, f:4, d:10 and y:2 are all errors.

Those nine letters and the numbers are every type there is. A column whose type is anything else is a malformed document: another byte, two letters, or a number with anything in it but ASCII digits.

Integers

i is a signed integer, as wide as its size: 1 byte holds −128 to 127, 2 bytes −32,768 to 32,767, 4 bytes −2^31 to 2^31−1, 8 bytes −2^63 to 2^63−1. Every bit pattern is a value.

jalapenojson
0.8
#n:i:2
3
2a 00 ff ff 00 80  n: 42, -1, -32768

Floats

f is an IEEE 754 double. It is always finite: a reader refuses a NaN or an infinity. -0.0 is a value.

jalapenojson
0.8
#price:f:8
2
00 00 00 00 00 00 f8 3f  price: 1.5
9a 99 99 99 99 99 b9 3f  price: 0.1

Booleans

y is one byte: 00 is false and 01 is true. Any other byte is an error.

jalapenojson
0.8
#id:i:2 #paid:y:1
2
e9 03 ea 03  id: 1001, 1002
01 00        paid: true, false

Strings and the break byte

An s value is UTF-8 text. A value shorter than its column's size is followed by a break byte, which is a NUL (00). It marks where the value stopped inside its field.

A value is the bytes of its field up to the first break byte, or the whole field when it holds none. What follows the break is not part of the value: a reader does not read it, check it or hand it back, so it may be anything at all. The field keeps its size either way.

jalapenojson
0.8
#name:s:5
3
42 6f 00 00 00                 name: Bo
41 6c 69 63 65                 name: Alice
42 6f 00 78 79                 name: Bo

The first and third rows both hold Bo. The bytes after the third row's break are not part of its value.

So a writer never has to fill the rest of a field. The encoders in this repository fill it with break bytes.

No s value may contain a NUL, since the first one ends it. A value longer than its column's size cannot be written; an encoder refuses it.

A value that fills its field carries no break byte. So Bo and Bo in the same s:5 column are different values, and both read back as what was written.

A reader refuses an s value that is not UTF-8.

Bytes

A b value is any bytes at all, NULs included, so a b column has no break byte: every value in it is exactly the column's size, and an encoder refuses one that is not.

Dates

The four date types count from 1970-01-01 in the proleptic Gregorian calendar. Every day has 86,400 seconds; there is no leap second.

  • d is a date with no time of day: days since 1970-01-01.
  • t is a time of day with no date: microseconds since midnight, from 0 to 86,399,999,999.
  • n is a wall clock with no zone: microseconds since 1970-01-01 00:00:00. It says what a clock read, not which instant.
  • z is an instant and the offset it was written with: microseconds since 1970-01-01 00:00:00 UTC, then the offset in minutes east of UTC, from −1439 to 1439. The wall clock it was written with is the instant plus the offset.

A d, n or z value's date, which for z is the date on its wall clock, falls in the years 1 to 9999. A value outside those ranges is an error.

jalapenojson
0.8
#joined:d:4 #seen:z:10
1
f4 4f 00 00                    joined: 2026-01-15
00 ce 3a 31 68 48 06 00 78 00  seen: 2026-01-15T09:30:00+02:00

joined is day 20,468. seen is the instant 2026-01-15 07:30:00 UTC, and 120 minutes east: 09:30 on the wall clock, two hours ahead of UTC.

docs/dates.md has the reasoning for the four types.

Row access

Every column has a size, so a reader finds any value by arithmetic:

row N of column M  =  <column M's start> + N * <column M's size>

The column starts follow from the header alone: the counts say how many rows each table holds, and the sizes say how long each column is. Reading one value costs the same whatever the document holds. A reader builds no index.

A list column has a fixed size too, so this holds for every schema, including one with list columns.

Nested lists

A type that is a number names a schema, counting from 1 in the order the schema lines appear. The number is ASCII digits, as many as it takes. The value in that column is a list of rows of that schema.

Each list column has a table of its own. It holds the rows of every list in that column, from every row whose schema has the column, in whichever table that row is. The counts line says how many rows each list column's table holds.

A list column's value is 16 bytes: an unsigned 64-bit number saying where the list's first row is in the column's table, then an unsigned 64-bit number saying how many rows the list holds. A list column declares no size, since its size is always 16.

jalapenojson
0.8
#order:i:4 #items:2
#sku:s:2 #qty:i:1
1 1
e9 03 00 00                                      order: 1001
00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00  items: row 0, 1 row
41 31                                            items table, sku: A1
02                                               items table, qty: 2

One order, 1001, carrying one item, {sku: "A1", qty: 2}. The counts line says the document has 1 row and the items table has 1.

Every row of a list column's table belongs to exactly one list in that column. So the lists in a column cover their table once: none overlaps another, none reaches past the table's end, and no row is left out. A reader refuses a document where they do not. An empty list holds 0 rows, and its start may be anything from 0 to the number of rows in the table.

Rows in a list column's table are rows like any other: they have their own list columns, whose lists are in those columns' tables.

More than one list column

A document declares as many schemas as it needs, and a schema may hold as many list columns as it needs:

jalapenojson
0.8
#order:i:4 #items:2 #ship:3
#sku:s:2 #qty:i:1
#city:s:6 #zip:s:4
1 2 1
e9 03 00 00                                      order: 1001
00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00  items: row 0, 2 rows
00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00  ship: row 0, 1 row
41 31 42 37                                      items table, sku: A1, B7
02 01                                            items table, qty: 2, 1
4f 73 6c 6f 00 00                                ship table, city: Oslo
30 31 35 30                                      ship table, zip: 0150

One order carrying two items and one shipping address. The counts are 1 row in the document, 2 in the items table and 1 in the ship table.

Trees

A schema may name itself, which is how a tree is written. A reference may point forward, because every schema line is read before any value.

jalapenojson
0.8
#id:i:1 #kids:1
1 3
01                                               id: 1
00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00  kids: row 0, 2 rows
02 03 04                                         kids table, id: 2, 3, 4
02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  kids table, kids: none
02 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00  kids: row 2, 1 row
03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  kids: none

Node 1 has two children, 2 and 3, and 3 has one child, 4. All three children are rows of the kids table, whatever their depth, and the kids column of those rows points into the same table.

Since every row of a table belongs to one list, following lists from the document's rows never reaches the same row twice, and so never loops.

Following them must also reach every row. A schema that names itself, directly or through other schemas, could otherwise hold a list that holds its own owner, or the owner of its owner: rows that cover their table once and yet hang from nothing. A reader refuses a document with a row that no chain of lists from the document's rows reaches. Where no schema names itself, the rule above already makes that impossible, so a reader has nothing more to check.

Limits

The format has none. A document holds any number of rows, a schema any number of columns, a document any number of schemas, and lists nest to any depth. A size, a count and a schema's number are as large as they need to be, and leading zeros mean nothing in any of them, however many there are.

A count or size larger than any document could hold is not an error by itself. The one question a reader asks of it is whether the columns it describes fit in the bytes that follow, and when they do not, the document is too short.

The value types have ranges, which are what the types mean rather than limits on a document: an i is as wide as its size, an f is a finite double, and a date's year is 1 to 9999.

What bounds a document is the machine reading it: the memory it has, and the largest object its language holds at once. A reader refuses nothing else for being too large.

What a reader refuses

  • A version that is not 0.8.
  • A header with no schema line, and a schema line that is not column declarations separated by single spaces.
  • A column with no size that is not a list column, and a size on a list column.
  • A size that is not an ASCII integer, or one its type does not allow.
  • A column name that is empty, is not UTF-8, or contains :, a space, a newline, or NUL.
  • Two columns with the same name in one schema.
  • A type that is neither one of the nine letters nor a number.
  • A number naming a schema the document does not declare. Schemas count from 1, so 0 names none.
  • A counts line that is not ASCII integers separated by single spaces, or that holds more or fewer of them than one plus the number of list columns.
  • A document longer or shorter than its header says.
  • A float that is not finite, a boolean byte that is not 00 or 01, and a date or time outside its range.
  • An s value that is not UTF-8.
  • A list that reaches past its table's end, two lists in one column that share a row, a row of a list column's table that no list holds, and a row that following lists from the document's rows never reaches.

An encoder additionally refuses a missing value, since there is no null to write in its place; an integer that does not fit its column's size; a float that is not finite; an s value longer than its column's size or containing a NUL; and a b value that is not exactly its column's size.

Complete example

jalapenojson
0.8
#id:i:4 #name:s:5 #joined:d:4 #seen:z:10
2
2a 00 00 00 07 00 00 00                          id: 42, 7
41 6c 69 63 65 42 6f 62 00 00                    name: Alice, Bob
f4 4f 00 00 aa 4f 00 00                          joined: 2026-01-15, 2025-11-02
00 ce 3a 31 68 48 06 00 78 00                    seen: 2026-01-15T09:30:00+02:00
00 1f cb a0 a4 42 06 00 00 00                    seen: 2025-11-02T23:05:00Z

Two rows: `{id: 42, name: "Alice", joined: 2026-01-15, seen: 2026-01-15T09:30:00+02:00} and {id: 7, name: "Bob", joined: 2025-11-02, seen: 2025-11-02T23:05:00Z}`.

The header is 47 bytes, and id starts right after it. name starts right after id, at byte 55. Alice fills its field and carries no break byte; Bob is three bytes in five and carries two. joined starts at byte 65 and seen at byte 73, and the document ends at byte 93.

Alice joined on a date with no time of day and was last seen at 09:30 two hours ahead of UTC. Bob was last seen at 23:05 UTC.

examples/example.jjson is this document, byte for byte.