jalapenojson

Why dates look like this

The README says what the four date tags are and how to use them. This file says why they are shaped that way, and what was considered and rejected, so the next person to touch them does not have to re-derive it.

What d used to be

Before the four tags, d existed but meant nothing. Python's decode routed it to the same branch as s, the JS decoder had a literal // s, d on that line, and the C parser stored the tag byte without ever reading it. Nothing validated a d value and nothing returned a date, so "not a date at all" was a perfectly good d.

Encoding was worse than untyped. Both encoders simply stringified whatever they were handed, so the same value produced different bytes depending on the language and the machine:

Python  date(2026,1,15)            -> 2026-01-15
Python  datetime(2026,1,15,9,30)   -> 2026-01-15 09:30:00      (a space, not "T")
JS      new Date(Date.UTC(...))    -> Thu Jan 15 2026 09:30:00 GMT+0000 (Coordinated Universal Time)

The JS spelling also moved with the writer's own timezone: the identical Date encoded to 09:30:00 GMT+0000 on a UTC machine and 18:30:00 GMT+0900 on one in Tokyo. Two machines writing the same value produced different files. For a serialization format that is a portability hole, and it is the concrete bug the four tags exist to close. js/test_jalapenojson.mjs still runs a case for it.

Why four tags and not one

The distinction that matters, and that most formats get wrong, is **wall clock versus absolute instant**:

ConceptExampleWhat did the clock read?Which instant?
Local date2026-01-15partlyno
Time of day09:30:00yesno
Naive datetime2026-01-15T09:30:00yesno
Datetime + offset2026-01-15T09:30:00+02:00yesyes

A naive datetime is not a point in time; it is a description of a clock face. A datetime with an offset is both, which is why z is the one to default to.

Collapsing these into one tag is what forces the guessing that caused the original bug: you cannot write a date without inventing a time, or an instant without inventing a zone. Leaving n out in particular pushes people to write a made-up offset into a z column, which is worse than saying "no zone here".

The tags cost nothing per row, because the type is in the header once per column rather than beside each value. That is why widening from one tag to four was cheap enough to be worth doing properly.

Why binary

Up to format 0.6 a date was text, 2026-01-15T09:30:00+02:00, because every number in the format was text and a binary date would have been the only binary value in it. Format 0.7 stores every number in binary, so dates went with them:

typestored asbytes
dsigned 32-bit days since 1970-01-014
tsigned 64-bit microseconds since midnight8
nsigned 64-bit microseconds since 1970-01-01 00:00:008
zsigned 64-bit microseconds since 1970-01-01 00:00:00 UTC, then signed 16-bit minutes east of UTC10

That is 4 to 10 bytes where the text took 10 to 25, a value sorts as an integer, and reading one is a load rather than a parse.

z stores the instant rather than the wall clock, so that comparing two z values, or sorting a column of them, needs no arithmetic. The wall clock it was written with is the instant plus the offset, and every reader hands that back.

Why there is no zoned datetime

A tag carrying an IANA name (2026-01-15T09:30:00+02:00[Europe/Oslo]) was considered and left out. It is only needed to keep a future local time correct across a DST rule change. If a government moves the changeover, an offset recorded today becomes wrong while a zone name stays right. That is real but rare, and it drags a timezone database into every implementation.

Until that case shows up, z plus a separate s column holding the zone name does the same job with no new machinery.

Why the limits

They exist so that no implementation can write a value another one cannot read. A format whose three implementations disagree about a value is back to the problem the types were added to fix.

  • Microseconds, not nanoseconds. A Python datetime has a microsecond field and no nanosecond one, so at microseconds Python is exactly lossless, and at nanoseconds it could not return a plain datetime at all. C could carry nanoseconds; the question is what Python would hand back.

A JavaScript Date is coarser than either, holding milliseconds only, which is why the JS decoder returns micros beside the Date rather than putting the fraction inside it. An earlier version did not, and silently truncated .123456 to .123 on a decode and re-encode.

  • Years 1 to 9999. That is the range of a Python date. A 64-bit count of microseconds reaches about 292,000 years either side of 1970, so the type could hold more, but a value Python cannot hand back is a value the three implementations do not agree on. For z the range applies to the wall clock, since that is what Python builds.
  • No leap second. Every day has 86,400 seconds, so second 60 cannot be written at all. Neither a Python time nor a JS Date can hold one.
  • Offsets from -1439 to 1439 minutes. That is what Python's timezone accepts: anything less than a day either way.

Why validation happens on access, not at parse

Opening a document reads its header and its list fields and nothing else. Checking every date while opening would make date columns cost something even when nobody reads them.

So a date is checked where it is read. In C that is the accessors in jj_date.h, which is why jalapenojson.h knows nothing about dates. In Python and JavaScript, decode() and the Table's get() and column() are where a value is converted, so the range check sits in the same place.

What each language hands back, and why JS is the odd one

Python maps cleanly onto date, time, naive datetime and aware datetime. C is fine because jj_date is defined here.

JavaScript has no type for a date without a zone, so d, t and n come back as canonical strings.

z cannot come back as a bare Date, and this is a constraint on any future reshaping of the JS decoder rather than a detail of today's one. A Date is an instant: it carries no offset, so decoding into one alone discards the +02:00 and shows the value in the reader's own timezone, which is the original bug again. It also holds only milliseconds, so it silently truncates anything finer. **Both the offset and the fractional second have to be carried beside the instant**, whatever the shape is called. Temporal would solve both properly and is worth revisiting once it is universally available.