[FEATURE](codegen) Add a stac-table-columns codegen format

[Seth Fitzsimmons (sethfitz)]

Description

Adds a stac-table-columns codegen format that renders the STAC Table extension's table:columns from the Pydantic models, one STAC properties fragment per feature type — 15 in all. It targets table extension v1.3.0, released 2026-09-07.

The models already hold everything a Column Object asks for, so the catalog is a projection with no authoring step. v1.3.0's Column Object is name, type and description, plus what it newly admits from STAC common metadata (data_type, unit) and from the Vector extension (vector:geometry_types). Everything else the IR carries — constraints, optionality, defaults, nested field descriptions, enum vocabularies — still has nowhere to go, so the renderer's second output is a gap log that records each of those as it is dropped.

Four decisions are explicit in the code, because nothing downstream can check them.

Column types use Arrow's names — int64, string, double, binary, struct<a: int32, b: string>, list<item: int32>. type is an unconstrained string, and real catalogs carry whatever their generating engine's stringifier produced; stac-extensions/table#1, "Firm up the definition of ColumnObject.type", has been open since 2021, lists three options and picks none. Arrow is what Overture distributes — the release is GeoParquet, and these are the names an Arrow reader reports on opening it — and it matches published catalogs, which name columns int64, string, double, binary. The vocabulary and the composite grammar both come from pyarrow's own stringifier rather than from memory.

Struct member order is the model's declaration order. For bbox that does not match what Overture ships — released files order it xmin, xmax, ymin, ymax, as does this repo's PySpark BBOX_STRUCT — and the discrepancy is accepted rather than encoded: a constant matching one publisher's file layout would state a fact about that publisher rather than about the schema. It is recorded in the constant's comment and pinned by a test, so it reads as a decision rather than an oversight.

type and data_type consequently disagree on floats, and that is deliberate rather than an oversight. data_type is bound to STAC common metadata's fixed vocabulary, which names them float32/float64; type is bound to Arrow, which names the same two float/double. Harmonising them would make one field lie about the vocabulary it claims to speak. A test pins the mismatch so a later reader finds it recorded rather than "fixes" it.

data_type is Overture's own base-type name, not a translation. STAC common metadata's data-type vocabulary (int8…int64, uint8…uint64, float32/float64, other) already matches Overture's numeric base-type names verbatim, so this is an allowlist against that vocabulary rather than a lookup table. Every column gets one — other is a member of the vocabulary, not an omission — so most Overture columns resolve to other, and only the numerics carry signal.

Flattening a discriminated union drops information, and every drop is logged. In segment, RoadClass and RailClass each contribute a class column; a flat column list has one slot for it, so one description is kept and the other disappears from the output. This happens seven times in segment. Nothing catches it on its own: the alternatives have identical physical types, so every compatibility check passes and the output looks complete. The renderer emits a flatten-collision gap at each site, the only record that something was dropped.

Ambiguity is refused, and vector:geometry_types is the one case that looks like an exception and is not. table:primary_geometry is emitted when exactly one geometry column exists and omitted, with the omission logged, when two do — nothing in the Pydantic model indicates which is primary. table:row_count is never emitted: a row count is a property of data, and this path has only a schema. A recursive model raises, because a type string cannot name a type and so a cycle cannot terminate. Union alternatives resolving to different physical types raise.

vector:geometry_types is defined as the geometry types present in the dataset, which by that same rule is a data fact this renderer cannot measure. It is emitted anyway, because it is read from GeometryTypeConstraint, which is validated on every row and is therefore an exhaustive bound on what a column can contain rather than a guess about what it does. A geometry column with no such constraint gets neither the field nor a claim about it — logged as a gap instead. Because the field belongs to another extension, the fragment declares the Vector extension in stac_extensions exactly when a column emits it, or it would not validate.

unit is not implemented, because Overture Schema has no consistent way to encode one. Where a unit varies per row it is a field — Speed carries a SpeedUnit, so a single unit string on that column would assert every row shares a unit when the schema allows mixing mph and km/h. Where a unit is fixed (Height, Elevation, Depth, roof bearing) it appears only in a free-text description. Neither is a column-level fact, and this renderer does not parse prose.

The first commit stands on its own: extraction now carries default and deprecated through to FieldSpec, which addresses #674.

Closes #723.

Reference

1. Add stac-table-columns renderer to overture-schema-codegen #723 — the issue this implements
2. https://github.com/stac-extensions/table — the extension being targeted
3. https://github.com/stac-extensions/table/releases/tag/v1.3.0 — the release this targets, which admits STAC common metadata on the Column Object and recommends vector:geometry_types on geometry columns
4. https://github.com/stac-extensions/vector — the extension vector:geometry_types comes from
5. Firm up the definition of ColumnObject.type stac-extensions/table#1 — the open thread on what type means
6. [FEATURE] Carry deprecation through extraction and render it in the Markdown reference #674 — wants the same deprecated carry the first commit adds

Checklist

1. Add relevant examples. — n/a, no schema change
2. Add relevant counterexamples. — n/a, no schema change
3. Update any counterexamples that became obsolete. — n/a, no schema change
4. Update in-schema documentation using plain English written in complete sentences, if an update is required. — n/a, no schema change
5. Update Docusaurus documentation, if an update is required. — not done; would follow if this stops being a draft
6. Review change with Overture technical writer to ensure any advanced documentation needs will be taken care of, unless the change is trivial and would not affect the documentation. — n/a, codegen-internal

[github-actions]

🗺️ Schema reference docs preview is live!

🌍 Preview	https://staging.overturemaps.org/schema/pr/724/schema/index.html
🕐 Updated	Sep 10, 2026 20:19 UTC
📝 Commit	9e860f2
🔧 env SCHEMA_PREVIEW	true

Note

♻️ This preview updates automatically with each push to this PR.

[Francisco Jiménez (jjcfrancisco)]

Looks good in general. I left few comments to add functionality due to new v1.3.0

[Francisco Jiménez (jjcfrancisco)]

With v1.3.0, we should add a data_type field here alongside type, plus a small mapping (spec vocab). Although, a lot of columns will resolve to "other" for Overture, only numerics like level carry real signal.

We should also add geometry_type (new in v1.3.0) for the geometry column. Overture already knows whether each feature type is Point / LineString / Polygon.

And unit from common metadata (now permitted on Column Object in v1.3.0). Overture carries real units like km/h or mph for speed limits

[Seth Fitzsimmons (sethfitz)]

data_type needed no mapping table: STAC common metadata's vocabulary (int8…int64, uint8…uint64, float32/float64, other) already matches Overture's own numeric base-type names, so it's an allowlist against that vocabulary rather than a translation.

One distinction: our earlier reading that there was no concrete vocabulary was about type, which is still unconstrained (stac-extensions/table#1 is open, lists three options, picks none). v1.3.0 draws the line explicitly: type takes the native name in the file format, data_type the standardized one from STAC common metadata. The vocabulary is stac-spec's and predates this; what's new is permission to use it on a Column Object, which v1.2.0 didn't mention at all.

type and data_type now disagree on floats. data_type says float32/float64 because that vocabulary is fixed; type says float/double because those are Arrow's names. There's a test pinning the mismatch so nobody later "fixes" one into the other.

On geometry: this emits vector:geometry_types, read from the model's GeometryTypeConstraint, since that's what's declared. The released v1.3.0 defines no geometry_type on the Column Object — it points at the Vector extension instead.

Overture Schema doesn't define units in a way that's compatible with what STAC is looking for, so unit isn't implemented. I have some ideas for a consistent unit definition that could then be surfaced in both the Markdown and STAC renderers:

class Unit(str, DocumentedEnum): ...
class LengthUnit(Unit): ...

Height = NewType("Height", Annotated[float64, LengthUnit.M])
Width  = NewType("Width",  Annotated[float64, LengthUnit.M])

height: Height | None = Field(default=None, description="Height of the building or part.")

The type names the quantity and the annotation carries the unit, so a height and a width stay distinct even though both are metres — mixing them is a type error rather than a silent bug. The wire format stays a plain double and Pydantic validation is unchanged, while the unit becomes a machine-readable fact a renderer can emit.

[Francisco Jiménez (jjcfrancisco)]

Despite DuckDB being the only executable example in the STAC repo, I'd argue for Arrow.

Prior art in the wild. I (randomly) picked three published Portolan catalogs:

• Madrid: defunciones-barrio-2025
• Microsoft ML road detections
• TriMet district boundary

All three have Arrow flavored primitives (int64, string, double, binary) although slightly diverge on struct grammar. It seems a little bit wild at the moment.

Even the STAC's own vocab is already Arrow(ish), the data_type list in STAC common metadata

[Seth Fitzsimmons (sethfitz)]

Agreed, and switched — type is Arrow now.

STAC's data_type vocabulary is already Arrow-shaped, so under DuckDB type a column carried two forms of the same physical type (type: "integer", data_type: "int32") that shared no vocabulary at all. Under Arrow they converge for every primitive, diverging only where Arrow's naming is finer than STAC's (string/binary against STAC's blanket other).

Both the primitive names and the composite grammar come from pyarrow's own stringifier rather than from memory, cross-checked against the Portolan catalogs you cited and against the Arrow schema of an actual 2026-08-19.0 release file read from S3. That third source also settled Geometry → binary: it is literally what Arrow reports for Overture's own geometry column. The semantic loss is logged as a gap, and vector:geometry_types carries the part the schema can assert.

One thing the switch surfaced without changing: the bbox struct is emitted in the BBox class's declaration order, xmin, ymin, xmax, ymax, while released Overture files order it xmin, xmax, ymin, ymax. A member-order constant matching one publisher's file layout would be a fact about that publisher rather than about the schema, so this renderer keeps the model's order and the discrepancy is accepted deliberately. It is recorded in the constant's comment and in a test, so it reads as a decision rather than an oversight.

One place the sources disagree. pyarrow's constructor stringifies list elements as list<item: …>, while Arrow reading Overture's own Parquet reports list<element: …>. This emits item, because the same round-trip that renames it also renders maps as map<string, string ('common')> — a file artifact rather than a type name — so following "what the reader reports" consistently produces something worse. Happy to flip it if you read the precedent differently; it is one line.

Separately, datetime maps to timestamp[us, tz=UTC], which describes the model rather than the release: no shipped Overture column is actually an Arrow timestamp (sources[].update_time is a string). That is right for a schema-derived renderer, but the "what we distribute" justification does not reach that field, so it is worth a second opinion.