Read MEOS's temporal types out of MEOS

.github/workflows/go.yml

@@ -43,6 +43,28 @@ jobs:
       - name: Vet and build, spark backend
         run: go vet -tags spark ./... && go build -tags spark ./...
 
+      - name: Check out MobilityDB master
+        uses: actions/checkout@v4
+        with:
+          repository: MobilityDB/MobilityDB
+          ref: master
+          path: mobilitydb
+
+      # catalog_gen.go is MEOS's own table of temporal types, so it is read from
+      # the MEOS sources rather than kept by hand, and this is what makes the
+      # copy in the tree the current one. A type MEOS gains arrives here as a
+      # difference to regenerate, which is the whole reason the table is
+      # generated: a hand-kept copy goes stale in silence instead.
+      - name: The generated catalog is what MEOS declares
+        run: |
+          go run ./tools/codegen -mobilitydb mobilitydb -o catalog_gen.go
+          if ! git diff --quiet -- catalog_gen.go; then
+            echo "::error::catalog_gen.go is not what MEOS declares. Regenerate it: go run ./tools/codegen -mobilitydb <a MobilityDB checkout> -o catalog_gen.go"
+            git diff -- catalog_gen.go
+            exit 1
+          fi
+          echo "the committed catalog is what MobilityDB master declares"
+
   # EVERY TEST THIS REPOSITORY CARRIES RUNS HERE, AND NONE OF THEM SKIPS.
   # A test skips when a precondition is missing from the job that collects it,
   # so this job supplies all three: the fixture database the Annex A data tests
@@ -77,12 +99,12 @@ jobs:
       # The vendored OGC schema bundle is compared against what OGC publishes.
       MFAPI_SCHEMA_FRESHNESS: "1"
       # The floor is every result line `go test -v` writes, subtests included:
-      # 77 top-level tests and 64 subtests. Subtests belong in it because a
+      # 84 top-level tests and 64 subtests. Subtests belong in it because a
       # t.Run whose body skips is as unrun as its parent would be, and deleting
       # a case from a table-driven test moves nothing else. Raise it when the
       # suite grows. Lowering it belongs in the same commit as the removal it
       # accounts for, because deleting a test is what a skip refusal cannot see.
-      MFAPI_TEST_FLOOR: "141"
+      MFAPI_TEST_FLOOR: "148"
     steps:
       - uses: actions/checkout@v4
 

GENERATION.md

@@ -0,0 +1,70 @@
+# Generation
+
+MobilityAPI is a tier over MEOS, so what it knows about MEOS is generated from
+MEOS rather than kept by hand. The generator lives in `tools/codegen`, its
+output is committed, and CI regenerates it against MobilityDB master and
+refuses a difference.
+
+## What is generated
+
+`catalog_gen.go` is the table of MEOS temporal types: for each one, its name,
+its base type, its bounding box, the token `asMFJSON` writes for it, whether it
+is spatial, whether it is a number, and whether MEOS interpolates it linearly.
+
+Run it against any MobilityDB checkout:
+
+```bash
+go run ./tools/codegen -mobilitydb <checkout> -o catalog_gen.go
+```
+
+## Where the facts come from
+
+The generator reads the MEOS sources that define them, in the checkout it is
+pointed at:
+
+| source | what is read |
+| --- | --- |
+| `meos/src/temporal/meos_catalog.c` | the `[T_X] = "x"` type names; each temporal type's `temptype_basetype` and `type_bboxtype`; and the predicates `temporal_type`, `tnumber_type`, `tspatial_type` and `temptype_supports_linear` |
+| `meos/src/temporal/type_out.c` | `temptype_as_mfjson_sb`, the switch that writes each type's MF-JSON type token |
+
+A type MEOS carries that the MF-JSON switch does not name has no MF-JSON form,
+and the empty token in the table says so. That is how `tdouble2`, `tdouble3`
+and `tdouble4`, which exist for temporal aggregation, stay off every surface
+without a list here naming them.
+
+The sources are read by anchoring on a definition and counting braces, never by
+matching a pattern across a span, and string literals are stepped over so the
+`{` inside `{\"type\":\"MovingFloat\",` is read as text rather than as
+structure.
+
+## What the tier adds, and what holds it complete
+
+Two things are the standard's rather than MEOS's, so they are stated in
+`main.go` next to the code that uses them:
+
+- `ogcScalar` maps a base type to the OGC temporal-property token Part 1
+  defines for it, and to the interpolation this tier assumes when a request
+  body names none. Part 1 defines four tokens and no more, so this is keyed by
+  base type: every temporal type MEOS builds over one of those bases is carried
+  automatically, and one over a fifth base is a question for the standard.
+- `tPropAlias` holds the further spellings a client may send for a type.
+
+`catalog_test.go` is what keeps the two halves in step. It asserts that every
+non-spatial type MEOS writes MF-JSON for, over a base the standard names,
+resolves through `tPropType` with the MEOS type and the MF-JSON token the
+generated table carries; that no spatial type resolves, since a spatial value
+is a moving feature's geometry rather than a temporal property; and that the
+store's value columns are one per scalar type. A temporal type MEOS adds over
+one of the standard's bases therefore arrives as a failing test rather than as
+a request the tier rejects at run time.
+
+## The catalog this could read instead
+
+`MobilityDB/MEOS-API` publishes `meos-idl.json`, the machine-readable catalog
+every language binding generates from, and reading it here rather than the C
+sources would put this tier on the same footing as the bindings. Two of the
+facts above are not in that catalog today: the MF-JSON type token has no field,
+and `typeRelations.byBase` holds one temporal type per base, so it carries
+`trgeometry` for the base `pose` and drops `tpose`, which shares that base.
+Both are single-source facts in `meos_catalog.c` and `type_out.c`, which is
+what the generator reads until the catalog states them.

catalog_test.go

@@ -0,0 +1,159 @@
+package main
+
+import (
+	"sort"
+	"strings"
+	"testing"
+)
+
+// The generated table is what the tier knows about MEOS's temporal types, so
+// these assert the shape the rest of the code relies on rather than a list of
+// names, which would be the copy the generator exists to remove.
+
+func TestCatalogIsPopulated(t *testing.T) {
+	if len(temporalTypes) < 10 {
+		t.Fatalf("the generated catalog holds %d temporal types, which is too few to have been read from MEOS; regenerate with go run ./tools/codegen", len(temporalTypes))
+	}
+	for _, tt := range temporalTypes {
+		if tt.Name == "" || tt.Base == "" {
+			t.Errorf("a catalog row carries no name or no base type: %+v", tt)
+		}
+		if got := temporalTypeByName[tt.Name]; got == nil || got.Name != tt.Name {
+			t.Errorf("temporalTypeByName does not index %q", tt.Name)
+		}
+	}
+}
+
+// A temporal property is a non-spatial value, so every non-spatial type MEOS
+// writes MF-JSON for and gives a base the standard has a token for must be one
+// the store can carry. A type MEOS adds over such a base arrives here as a
+// failure rather than as a request the tier rejects at runtime.
+func TestEveryScalarTypeTheStandardNamesIsCarried(t *testing.T) {
+	for _, tt := range temporalTypes {
+		if tt.Spatial || tt.MFJSON == "" {
+			continue
+		}
+		if _, named := ogcScalar[tt.Base]; !named {
+			continue
+		}
+		got, ok := tPropType(tt.Name)
+		if !ok {
+			t.Errorf("%s is a scalar temporal type over base %s, which OGC Part 1 names, and tPropType does not resolve it", tt.Name, tt.Base)
+			continue
+		}
+		if got.cast != tt.Name {
+			t.Errorf("tPropType(%q).cast = %q, want %q", tt.Name, got.cast, tt.Name)
+		}
+		if got.mf != tt.MFJSON {
+			t.Errorf("tPropType(%q).mf = %q, want the token MEOS writes, %q", tt.Name, got.mf, tt.MFJSON)
+		}
+	}
+}
+
+// A spatial type is a moving feature's geometry, never a temporal property, so
+// the store must not offer to hold one.
+func TestNoSpatialTypeIsATemporalProperty(t *testing.T) {
+	for _, tt := range temporalTypes {
+		if !tt.Spatial {
+			continue
+		}
+		if _, ok := tPropType(tt.Name); ok {
+			t.Errorf("tPropType resolves %q, which is a spatial type and belongs under tgsequence", tt.Name)
+		}
+	}
+}
+
+// The tokens a client may send resolve, in the case the standard writes them
+// and in lower case, and an unknown one does not.
+func TestTPropTypeResolvesTheTokensClientsSend(t *testing.T) {
+	for _, tok := range []string{"", "TReal", "treal", "tfloat", "measure", "TInteger", "tint", "integer", "TText", "ttext", "string", "TBoolean", "tbool", "boolean"} {
+		if _, ok := tPropType(tok); !ok {
+			t.Errorf("tPropType(%q) does not resolve", tok)
+		}
+	}
+	for _, tok := range []string{"tgeompoint", "tpose", "nonsense", "tjsonb"} {
+		if _, ok := tPropType(tok); ok {
+			t.Errorf("tPropType(%q) resolves, and it names no OGC temporal property", tok)
+		}
+	}
+}
+
+// Two MEOS types carry the OGC token TInteger, tint over int4 and tbigint over
+// int8, so the token alone does not name a type. Resolving it by whichever the
+// map yields last picks a different type from run to run, and a stored property
+// is then read out of a column its table may not have.
+func TestATokenSeveralTypesCarryResolvesToOne(t *testing.T) {
+	carriers := map[string][]string{}
+	for name, tt := range scalarTemporalTypes {
+		carriers[tt.ogc] = append(carriers[tt.ogc], name)
+	}
+	for token, names := range carriers {
+		got, ok := tPropType(token)
+		if !ok {
+			t.Errorf("the OGC token %q resolves to no type, and %v carry it", token, names)
+			continue
+		}
+		if got.ogc != token {
+			t.Errorf("tPropType(%q) resolves to a type whose token is %q", token, got.ogc)
+		}
+		if len(names) > 1 {
+			want := ogcCanonical[token]
+			if want == "" {
+				t.Errorf("%v all carry the token %q and ogcCanonical names none of them", names, token)
+			} else if got.cast != want {
+				t.Errorf("tPropType(%q).cast = %q, want the canonical %q", token, got.cast, want)
+			}
+		}
+	}
+	// The resolution the conformance fixture depends on: a TInteger property is
+	// stored in vint, which is the column a store created before tbigint has.
+	got, ok := tPropType("TInteger")
+	if !ok || got.cast != "tint" || got.col != "vint" {
+		t.Errorf("tPropType(\"TInteger\") = %+v, want tint in vint", got)
+	}
+}
+
+// The OGC token and the default interpolation are what the stored documents
+// carry, so they are asserted by value: a change to either rewrites what the
+// service answers for properties already in a store.
+func TestScalarBindingsAreTheStandardsOwn(t *testing.T) {
+	want := map[string][2]string{
+		"tbool":   {"TBoolean", "Step"},
+		"tint":    {"TInteger", "Step"},
+		"tbigint": {"TInteger", "Step"},
+		"tfloat":  {"TReal", "Linear"},
+		"ttext":   {"TText", "Discrete"},
+	}
+	for name, w := range want {
+		got, ok := tPropType(name)
+		if !ok {
+			t.Errorf("%s does not resolve", name)
+			continue
+		}
+		if got.ogc != w[0] || got.defInterp != w[1] {
+			t.Errorf("tPropType(%q) = ogc %q interp %q, want %q and %q", name, got.ogc, got.defInterp, w[0], w[1])
+		}
+	}
+}
+
+// The store's value columns are one per scalar type, named for it, and the DDL
+// fragment names each with its type.
+func TestValueColumnsAreOnePerScalarType(t *testing.T) {
+	cols := tPropValueColumnNames()
+	if len(cols) != len(scalarTemporalTypes) {
+		t.Fatalf("%d value columns for %d scalar types", len(cols), len(scalarTemporalTypes))
+	}
+	var names []string
+	for _, c := range cols {
+		names = append(names, c.name)
+		if c.name != "v"+strings.TrimPrefix(c.typ, "t") {
+			t.Errorf("column %q does not name its type %q", c.name, c.typ)
+		}
+		if !strings.Contains(tPropValueColumns(), c.name+" "+c.typ) {
+			t.Errorf("the DDL fragment does not declare %s %s", c.name, c.typ)
+		}
+	}
+	if !sort.StringsAreSorted(names) {
+		t.Errorf("the value columns are not in a stable order: %v", names)
+	}
+}