Consider replacing MixedTypeSet wrappers with a heterogeneous Schema root

[ehennestad]

Summary

Replace the generated MixedTypeSet wrapper classes with a heterogeneous openminds.abstract.Schema root, restricting allowed types per property with a validator instead of a class declaration.

This is the alternative already named in the developer notes at the top of openminds.internal.abstract.MixedTypeSet, which record that the original blocker — enumerations for controlled instances in the ControlledTerm subtypes — has since been removed. This issue works that note up into a reviewable proposal, with the results of a prototype.

It is deliberately not part of the stacked series (#102 onwards). That series fixes defects and reworks the traversal shared by the serializer, resolver and deserializer, all within this repository. This proposal touches the generator and every generated type class, so it needs its own review.

See the comment below before reading further. The alternative this issue was originally weighed against — migrating the indexing overrides to matlab.mixin.indexing.RedefinesDot and keeping the wrappers — turns out not to be possible, because RedefinesDot does not intercept declared properties. Removing the wrappers is therefore the only way to remove the indexing layer.

What it would look like

A property declared today as:

author (1,:) openminds.internal.mixedtype.datasetversion.Author ...
    {mustBeListOfUniqueItems(author)}

would become:

author (1,:) openminds.abstract.Schema ...
    {mustBeAllowedInstance(author, ["openminds.core.actors.Consortium", ...
                                    "openminds.core.actors.Organization", ...
                                    "openminds.core.actors.Person"]), ...
     mustBeListOfUniqueItems(author)}

with openminds.abstract.Schema gaining matlab.mixin.Heterogeneous.

What this buys

• The whole indexing layer disappears. subsref, subsasgn and numArgumentsFromSubscript on Schema exist almost entirely to hide the wrappers. With no wrappers, instance.author(1) returns the concrete instance natively and there is nothing to override.
• 1520 generated wrapper classes are deleted (375 in latest, across six version trees).
• Reference replacement becomes a positional element assignment. instance.author(2) = resolved preserves the other entries and the index. This is the operation that is broken today, both for unresolved references generally and for multi-valued properties specifically.
• The reference representation can collapse to one form — see the scope question below.

Prototype results

A standalone prototype confirmed the mechanics and turned up four things the developer note does not anticipate.

1. mustBeA does not work. The note suggests mustBeA for property validation. It tests the class of the array, which for a heterogeneous array is the root, so a mixed array is rejected outright:

Error setting property 'author'. Value must be one of these types: 'HeteroA' or 'HeteroB'.

An element-wise validator is required instead.

2. Subclass-specific methods are blocked on a mixed array, with MATLAB:noSuchMethodOrField. Only methods that are Sealed or defined on the root can be called. Today MixedTypeSet forwards method calls to .Instance, so this may be a live regression rather than a theoretical constraint. Needs an audit of what is called on mixed property values.

3. An abstract root works, except for array expansion. Forming arrays, element access and .empty are all fine without getDefaultScalarElement. Padding (arr(5) = x on a 2-element array) fails with MATLAB:class:MissingGetDefaultScalarElement, and there is no sensible default openMINDS instance. Note the current subsasgn handles this same case poorly already — its own comment says empty slots "default to empty double, but should be empty object of correct instance type" — so this converts a silent wrong result into an explicit error.

4. Mixing becomes globally legal. [person, dataset] is a valid array anywhere; the restriction applies only at property assignment. This is the semantic the wrappers were protecting, and giving it up is a genuine trade rather than a bug.

Unresolved references still work, but the guarantee moves

An unknown-type stub remains representable: it is simply another concrete subclass of the root, and the property validator permits it alongside the allowed types. Verified in the prototype:

h.author = [realInstance, stub]   ->  root array, 2 elements
h.author(2) = resolved            ->  element 2 replaced, element 1 untouched

The cost is where the guarantee lives. Today a stub is accepted structurally: MixedTypeSet holds a MixedTypeReference by construction, so there is nothing to forget. Under this proposal every generated validator must explicitly exempt the stub type, which makes it generator discipline. One validator emitted without the exemption silently breaks references for that property. That risk lands in the same layer that produced the missing LINKED_PROPERTIES entry on TermSuggestion.addExistingTerminology and the untyped MRIScannerUsage.fieldOfView.

Scope question: minimal or full

Minimal — only properties that currently use a wrapper become Schema-typed. Single-type linked properties keep their concrete class, so an unresolved Person must still be a Person carrying an IsReference flag. Two reference representations remain.

Full — every linked property becomes Schema-typed. A reference is then always the same object regardless of context, because the allowed types come from the property rather than from the stub. The IsReference flag disappears, and with it the ambiguity between a typed instance that is a reference and a typed instance that is genuinely empty. See #12.

Full is the only version that reduces the reference representations to one. It is also the larger change: every linked property loses its class declaration, and type errors move from MATLAB's own message to a validator's.

Open questions

1. Minimal or full.
2. What editor tab-completion does for a Schema-typed property. Generated classes exist partly to give users completion and inline documentation, and this has not been tested. It should be checked before the proposal is taken seriously.
3. Whether any code calls a subclass-specific method on a mixed property value, per prototype finding 2.
4. Migration mechanics. The change touches the generator, 744 type classes and 1520 wrapper classes across six version trees, so it needs coordination with the pipeline branch. A mechanical transform of the existing trees plus a generator change is possible, but the generator change has to land or the next pipeline run reverts it.

Relationship to other work

• Implement "visitor" methods on the Base schema / Instance class #69 — the traversal architecture proposal. Independent of this, but the per-edge write-back protocol proposed there becomes less necessary if references are replaceable by positional assignment.
• Need a mechanism for linked instances that are not resolved #12 — mechanism for unresolved linked instances. The "full" scope above is a direct answer to it.

🤖 Generated with Claude Code

[ehennestad]

RedefinesDot cannot replace the indexing overrides — which makes this proposal the only route

I prototyped matlab.mixin.indexing.RedefinesDot as the conservative alternative to this proposal: keep the wrappers, migrate the mechanism, leave the UX untouched. It does not work, and the reason is decisive enough to change how this issue should be read.

RedefinesDot does not intercept access to declared properties. It is only consulted for names that are not real, accessible properties or methods:

declaration	dotReference called?
public property	no — resolved directly
Hidden property	no — still a real accessible property
private property	yes
undeclared name	yes
method	no — dispatched natively

The entire purpose of Schema.subsref is to intercept access to declared properties — dataset.author — and unwrap the MixedTypeSet behind them. RedefinesDot structurally cannot do that.

The only way to make it apply would be to stop declaring the properties and hold everything in a private container. That would remove properties(), isprop, tab completion, inline help and MATLAB's own property validation — that is, everything the generated type classes exist to provide. It is strictly worse than the status quo.

What this means

The framing I used when opening this issue was wrong, and I have corrected the body accordingly. I described this proposal as the ambitious option and the RedefinesDot migration as the safe one. In fact:

• the RedefinesDot migration is not possible;
• removing the wrappers is the only way to remove the ~345 lines of subsref/subsasgn/numArgumentsFromSubscript, because with no wrapper there is nothing to unwrap and the overrides can simply be deleted rather than reimplemented.

That does not make this proposal correct on its own — the four findings above still stand, particularly the loss of the structural guarantee for unresolved references and the scope question. But the "just modernize the mechanism instead" alternative is off the table, and anyone weighing this issue should know that the indexing layer stays exactly as it is unless something like this is done.

Still open

The tab-completion question in the open questions list is now the important one. If a Schema-typed property loses completion of the concrete type's properties, this proposal trades one UX problem for another, and the current design may simply be the best available under MATLAB's type validation.

🤖 Generated with Claude Code

[ehennestad]

Spike results: the mechanics work, the ergonomics do not

I built this on a local branch on top of the current stack: Schema made a matlab.mixin.Heterogeneous root, the 375 mixed properties in the latest tree retyped from their wrapper to openminds.abstract.Schema, and the 380 lines of subsref / subsasgn / numArgumentsFromSubscript deleted outright. Per-property type restriction was left out on purpose, to keep the spike to the structural question; the element-wise validator was prototyped separately and works.

998 of 1018 tests pass with the full type sweep. Mixed-type arrays, serialization, deserialization and round tripping all work with no indexing code at all:

d.keyword = [accessChannelTerm, dataTypeTerm];   % two different types, one property
class(d.keyword)        % openminds.abstract.ControlledTerm
class(d.keyword(1))     % openminds.controlledterms.AccessChannel
d.digitalIdentifier.identifier   % chained access, concrete type, no unwrapping

So the structural argument holds. The four remaining failures all share one cause, and that cause is what decides this issue.

The blocker: heterogeneous hierarchies disable constructor conversion

matlab.mixin.Heterogeneous turns off MATLAB's implicit conversion of a value into a member class via that class's constructor. Confirmed with a minimal A/B: two identical hierarchies, the only difference being the mixin.

Assigning a char to a property typed as a class with a matching constructor:
  NON-heterogeneous root: OK   -> PlainTerm (name=male)
  HETEROGENEOUS root    : FAIL  MATLAB:class:NoConversionDefined

For openMINDS that means this stops working:

subject1 = Subject('species', 'musMusculus', 'biologicalSex', 'male', ...);

That is line 6 of gettingStarted.mlx. The string shorthand for controlled terms is one of the most visible conveniences in the library, and it is lost for every one of the 112 controlled term types, not only for the properties this proposal retypes. biologicalSex is declared openminds.controlledterms.BiologicalSex and is untouched by the change; it breaks purely because the shared root became heterogeneous.

convertObject does not recover it

The error names convertObject as the hook, and it is genuinely called, but it is not given the target class:

>> convertObject: default=char toConvert=char

defaultObject arrives as the value being converted, not as a prototype of the destination type. I could only make the conversion succeed by hardcoding the target class inside convertObject. A single convertObject on Schema cannot know whether 'male' should become a BiologicalSex or a Species, so there is no generic implementation. Defining getDefaultScalarElement does not change what convertObject receives.

Deserialization is recoverable, because there the target type is known: StructAdapter.fromStruct can construct nested instances explicitly from LINKED_PROPERTIES / EMBEDDED_PROPERTIES instead of relying on implicit conversion. That is what took the spike from 727 to 998 passing. The user-facing string shorthand has no equivalent, because there is nothing at the point of assignment that knows the destination type.

Two further findings

The migration cannot be incremental. Heterogeneous refuses to load a class that defines subsasgn:

Class 'openminds.abstract.Schema' is not allowed to define a 'subsasgn' method.

So adopting the mixin and deleting the indexing overrides is one atomic change across the generator, every type class and every wrapper. There is no staged path.

The indexing override was silently widening SetAccess. basicNeuroscienceDataset.mlx does researchCenter.id = createId(...), and id is declared SetAccess = protected. That worked only because assignment was routed through subsasgn, which is a class method and therefore ran with protected access. Deleting it makes the tutorial fail with MATLAB:class:SetProhibited. This is worth fixing on its own terms, independently of this issue: either id should be publicly settable through a documented method, or the tutorial should stop doing it.

Where that leaves the proposal

Answering the open questions from the issue body:

1. Minimal or full — minimal, clearly. Measured across the model: 375 linked/embedded properties are mixed-typed and 555 are single-typed. properties('openminds.abstract.Schema') is empty.
2. Tab completion — much less of a problem than I expected, and it argues for minimal. A mixed property today is declared as its wrapper, whose only public property is Instance, so completion is already useless on all 375. Retyping them to Schema moves them from useless to empty. Under "full" the 555 single-typed properties would lose completion that works today, for no additional structural gain, since the indexing layer is already gone under minimal.
3. Subclass methods on mixed arrays — nothing in the test suite hit this in the spike.
4. Migration mechanics — atomic, per the finding above.

My reading: the ergonomic cost outweighs the structural gain. Deleting 1520 wrapper classes and 380 lines of indexing is real, and the fact that the spike reaches 998/1018 shows it is achievable. But paying for it with the controlled-term string shorthand across the whole library is a bad trade, and there is no way to keep both — the mixin is all-or-nothing on the shared root.

Unless someone sees a way to restore conversion that I have missed, I would close this in favour of leaving the wrappers in place, and record that subsref/subsasgn cannot be removed: RedefinesDot cannot replace them (see the comment above) and this is the only alternative.

The spike is on a local branch and is not going to be pushed unless it is wanted for reference.

🤖 Generated with Claude Code