Skip to content

refactor(document): share the element registry and adapter eleven engines pasted - #823

Merged
andiwand merged 3 commits into
mainfrom
refactor/shared-element-registry
Sep 6, 2026
Merged

refactor(document): share the element registry and adapter eleven engines pasted#823
andiwand merged 3 commits into
mainfrom
refactor/shared-element-registry

Conversation

@andiwand

@andiwand andiwand commented Sep 6, 2026

Copy link
Copy Markdown
Member

🤖 Generated with Claude Code

Closes #770 (parts 1–2 of the four it lists; the style registries and the
DecodedFile accessors are still open).

Every engine that builds an element tree shipped its own copy of the same flat
store, the same five tree links, the same element_at / append_child, and the
same ~30 lines of adapter navigation followed by one four-line
*_adapter(id) hook per element type. This puts that machinery in
internal/common/ and ports all eleven engines onto it.

37 files changed, 979 insertions(+), 2916 deletions(-)

380 of those insertions are the two new headers. Registry code across the ten
registries: 3028 → 1726 lines. No engine writes a *_adapter(id) hook or a
navigation method any more.

What is shared

internal/common/element_registry.hpp

  • ElementNode<Id> — the five links plus the type, stored Id wide.
  • SideTable<T> (hashed) and SortedSideTable<T, Id> (odf's sorted deque,
    promoted) — a per-type payload keyed by element id. Both carry their own
    bounds check, so an engine's accessor is return m_texts.at(id); and the
    per-payload check_*_id is gone.
  • ElementRegistry<Element, Id> — the store (id = index + 1), element_at,
    append_child, link_child, check_element_id, and the id-overflow guard.

The store is a std::deque for every engine now. That is what odf already did,
for the reason it applies everywhere: create_element_ hands the parser back an
Element &, and a vector both invalidates it and peaks holding two copies. The
ooxml engines were on a vector; nothing there held a reference across a create,
so this is a hazard removed rather than a bug fixed.

internal/common/element_adapter.hpp

ElementAdapter<Adapters…> inherits the per-type adapters named in its pack and
answers all 24 *_adapter(id) hooks from them:

template <typename Adapter, ElementType type>
const Adapter *adapter_(const ElementIdentifier element_id) const {
  if constexpr ((std::is_same_v<Adapter, Adapters> || ...)) {
    return element_type(element_id) == type ? this : nullptr;
  } else {
    return nullptr;
  }
}

Taking the adapters as template arguments rather than using CRTP keeps the
condition off the derived class, which would not be complete when a compiler
that instantiates virtual member bodies eagerly gets there. It also checks out
against the tree: the hook ↔ ElementType mapping was already 1:1 and identical
in all eleven engines.

It also carries the defaults every engine had verbatim — element_is_unique and
element_is_self_locatable true, element_is_editable false, and the two
util::document path forwards. Only odf and ooxml text override
element_is_editable, which is the only place the answer was ever different.

RegistryElementAdapter<Registry, Adapters…> adds the six navigation methods
over m_registry->element_at(id).

What each engine keeps

Its payload structs, its create_*_element, its secondary-chain append_*, and
its real per-type methods. rtf is now the whole pattern in 34 header lines:

class ElementRegistry final
    : public internal::ElementRegistry<ElementNode<ElementIdentifier>> {
public:
  struct Text final {
    std::string text;
  };

  std::tuple<ElementIdentifier, Element &> create_element(ElementType type);
  std::tuple<ElementIdentifier, Element &, Text &> create_text_element();

  [[nodiscard]] Text &text_element_at(const ElementIdentifier id) {
    return m_texts.at(id);
  }
  [[nodiscard]] const Text &text_element_at(const ElementIdentifier id) const {
    return m_texts.at(id);
  }

private:
  SideTable<Text> m_texts;
};

csv packs a kind and a coordinate into its id rather than keeping a registry,
so it takes ElementAdapter alone and keeps its own navigation.

Drift swept up on the way

  • clear() was dead in all ten registries — nothing has ever called it.
  • The three ooxml append_child had lost the "child already has a parent" guard
    every other engine kept; link_child restores it.
  • Thrown messages said DocumentElementRegistry::, a name no class has carried
    for a while.

Docs

AGENTS.md taught the copy — it named ppt_element_registry.* as the thing to
model, and rtf/AGENTS.md said its registry was "copied from oldms/text". The
element-adapter pattern section now leads with the machinery is shared — do
not write it again
, says what an engine actually writes, and points at rtf.
odf/AGENTS.md and ooxml/AGENTS.md follow.

Verification

  • Full suite: 1442 passed, 6 skipped, 0 failed. The reference HTML is
    byte-identical, which is the real claim here — this changes no behaviour.
  • -Werror build clean, clang-tidy clean on all 27 touched translation units.
  • The element struct is the same size as before (odf 32 bytes, ooxml 56) —
    ElementNode as a base costs nothing — and peak RSS on a 4.5 MB .ods is
    unchanged.

No CHANGELOG.md entry: refactoring is in the generated per-PR list already.

Merge order

#821 collapses the four drawing element types into frame, which touches four
entries in the shared hook list and in odf's adapter pack. It is the breaking
one, so this should go first and #821 rebase onto it — the conflict is a handful
of lines either way.

@andiwand
andiwand force-pushed the refactor/shared-element-registry branch from cccff0f to daf3527 Compare September 6, 2026 08:11
@andiwand
andiwand changed the base branch from main to fix/page-layout-direction-and-msvc-raw-string September 6, 2026 08:11
Base automatically changed from fix/page-layout-direction-and-msvc-raw-string to main September 6, 2026 08:14
andiwand and others added 2 commits September 6, 2026 10:15
…xml pasted

Every engine that builds an element tree writes the same flat store, the same
tree links and the same adapter navigation. Two headers in `internal/common`
now hold that shape, and odf and the three ooxml engines are ported onto it:

- `internal::ElementRegistry` owns the element vector, the id/index
  convention, `element_at`, `append_child` and `link_child`, over an element
  the engine derives from `ElementNode` and a payload side table (hashed, or
  odf's sorted one) that carries its own bounds check.
- `internal::ElementAdapter` answers every `*_adapter(id)` hook from the
  adapters it is given, and `RegistryElementAdapter` adds the six navigation
  methods.

No behaviour change: the reference output is byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VEsRyBu8o4TGJGn3thNEDc
…try and adapter

rtf, markdown, iwork, the three oldms formats and csv follow odf and ooxml onto
`internal::ElementRegistry` and `internal::RegistryElementAdapter`; csv, which
packs its ids rather than keeping a registry, takes the hook base alone and
keeps its own navigation.

The element store is now a `std::deque` for every engine, which is what makes
handing a parser an `Element &` safe. The registries lose the `clear()` nothing
ever called and the per-payload `check_*_id`, and the three ooxml `append_child`
regain the "child already has a parent" guard the others kept.

The side tables reach their const and non-const accessors through one static
that deduces the constness from its argument, rather than a `const_cast` back
from the const overload.

`AGENTS.md`, `odf/AGENTS.md`, `ooxml/AGENTS.md` and `rtf/AGENTS.md` taught the
copy; they now point at the shared shape.

No behaviour change: the reference output is byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VEsRyBu8o4TGJGn3thNEDc
@andiwand
andiwand force-pushed the refactor/shared-element-registry branch from daf3527 to 6ac7626 Compare September 6, 2026 08:17
The stray `private:` `rtf`'s adapter was left with, its registry pointer having
moved to the base, and the `document_util` / `document_path` includes eleven
adapters kept after the two path forwards moved with it.

Dropping them turned up `element_adapter.hpp` returning `DocumentPath` by value
on nothing but a forward declaration - it had been compiling on whichever
include the including `.cpp` happened to write first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VEsRyBu8o4TGJGn3thNEDc
@andiwand
andiwand merged commit df778a9 into main Sep 6, 2026
25 checks passed
@andiwand
andiwand deleted the refactor/shared-element-registry branch September 6, 2026 08:38
andiwand added a commit that referenced this pull request Sep 6, 2026
Forty-two accessors were the same body twice, once for each constness, with
only the spelled-out return type telling the two apart. An explicit object
parameter deduces that, so each pair is one function returning `T &` or
`const T &` from a single body.

The shared registry (#823) is where it pays most: `SideTable` and
`SortedSideTable` had written the workaround out by hand — two public overloads
delegating to a `static` helper templated on the object, under a comment naming
the trick — and every engine then repeated the pair for each of its payload
accessors. Both go, and a registry's accessor is

    [[nodiscard]] auto &text_element_at(this auto &self,
                                        const ElementIdentifier id) {
      return self.m_texts.at(id);
    }

`pdf`'s `Array` and `Dictionary` lose eleven more.

`ooxml_text_list`'s numbering walk would have dropped its Y-combinator with
them, but NDK 28.1's clang 19 segfaults on a capturing lambda that recurses
through an explicit object parameter, so it keeps passing itself along and says
why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QoTh7BEUSL2z9BBThgsEi7
andiwand added a commit that referenced this pull request Sep 6, 2026
Forty-two accessors were the same body twice, once for each constness, with
only the spelled-out return type telling the two apart. An explicit object
parameter deduces that, so each pair is one function returning `T &` or
`const T &` from a single body.

The shared registry (#823) is where it pays most: `SideTable` and
`SortedSideTable` had written the workaround out by hand — two public overloads
delegating to a `static` helper templated on the object, under a comment naming
the trick — and every engine then repeated the pair for each of its payload
accessors. Both go, and a registry's accessor is

    [[nodiscard]] auto &text_element_at(this auto &self,
                                        const ElementIdentifier id) {
      return self.m_texts.at(id);
    }

`pdf`'s `Array` and `Dictionary` lose eleven more.

`ooxml_text_list`'s numbering walk would have dropped its Y-combinator with
them, but NDK 28.1's clang 19 segfaults on a capturing lambda that recurses
through an explicit object parameter, so it keeps passing itself along and says
why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QoTh7BEUSL2z9BBThgsEi7
andiwand added a commit that referenced this pull request Sep 6, 2026
Forty-two accessors were the same body twice, once for each constness, with
only the spelled-out return type telling the two apart. An explicit object
parameter deduces that, so each pair is one function returning `T &` or
`const T &` from a single body.

The shared registry (#823) is where it pays most: `SideTable` and
`SortedSideTable` had written the workaround out by hand — two public overloads
delegating to a `static` helper templated on the object, under a comment naming
the trick — and every engine then repeated the pair for each of its payload
accessors. Both go, and a registry's accessor is

    [[nodiscard]] auto &text_element_at(this auto &self,
                                        const ElementIdentifier id) {
      return self.m_texts.at(id);
    }

`pdf`'s `Array` and `Dictionary` lose eleven more.

`ooxml_text_list`'s numbering walk would have dropped its Y-combinator with
them, but NDK 28.1's clang 19 segfaults on a capturing lambda that recurses
through an explicit object parameter, so it keeps passing itself along and says
why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QoTh7BEUSL2z9BBThgsEi7
andiwand added a commit that referenced this pull request Sep 6, 2026
Forty-two accessors were the same body twice, once for each constness, with
only the spelled-out return type telling the two apart. An explicit object
parameter deduces that, so each pair is one function returning `T &` or
`const T &` from a single body.

The shared registry (#823) is where it pays most: `SideTable` and
`SortedSideTable` had written the workaround out by hand — two public overloads
delegating to a `static` helper templated on the object, under a comment naming
the trick — and every engine then repeated the pair for each of its payload
accessors. Both go, and a registry's accessor is

    [[nodiscard]] auto &text_element_at(this auto &self,
                                        const ElementIdentifier id) {
      return self.m_texts.at(id);
    }

`pdf`'s `Array` and `Dictionary` lose eleven more.

`ooxml_text_list`'s numbering walk would have dropped its Y-combinator with
them, but NDK 28.1's clang 19 segfaults on a capturing lambda that recurses
through an explicit object parameter, so it keeps passing itself along and says
why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QoTh7BEUSL2z9BBThgsEi7
andiwand added a commit that referenced this pull request Sep 6, 2026
* build!: raise the standard to C++23

The whole matrix already supports it: clang 18, gcc 14, apple-clang, MSVC 19.40,
NDK 28.1 and emsdk 3.1.73. It costs no source change at all — the two transitive
includes libc++'s C++23 headers stopped handing out went in with #827, which is
the whole of it.

Three ceilings sit under the standard, and AGENTS.md records all three because
none is discoverable from a local build. The library half is capped by emsdk
3.1.73's libc++ 18.1, the oldest here and the newest emsdk conan-center
packages. `std::format` is unusable on any slice — the apple profiles deploy to
macOS 12 / iOS 15, and libc++ marks the floating-point `to_chars` that `<format>`
instantiates as macOS 13.3 / iOS 16.3, which is why #828 formats through `fmt`.
And NDK 28.1's clang 19 segfaults on a capturing recursive lambda taking
`this auto self`.

The public headers stay C++20 — no `target_compile_features(odr PUBLIC …)` and
no `cppstd` in `package_info`, so a consumer picks its own standard, and
`check_min_cppstd` sits in `validate_build` where it constrains building `odr`
rather than using it.

On MSVC there is no `/std:c++23`; CMake maps `CXX_STANDARD 23` to
`/std:c++latest`.

* refactor: let deducing `this` write the const overload

Forty-two accessors were the same body twice, once for each constness, with
only the spelled-out return type telling the two apart. An explicit object
parameter deduces that, so each pair is one function returning `T &` or
`const T &` from a single body.

The shared registry (#823) is where it pays most: `SideTable` and
`SortedSideTable` had written the workaround out by hand — two public overloads
delegating to a `static` helper templated on the object, under a comment naming
the trick — and every engine then repeated the pair for each of its payload
accessors. Both go, and a registry's accessor is

    [[nodiscard]] auto &text_element_at(this auto &self,
                                        const ElementIdentifier id) {
      return self.m_texts.at(id);
    }

`pdf`'s `Array` and `Dictionary` lose eleven more.

`ooxml_text_list`'s numbering walk would have dropped its Y-combinator with
them, but NDK 28.1's clang 19 segfaults on a capturing lambda that recurses
through an explicit object parameter, so it keeps passing itself along and says
why.

* perf: stop zeroing the buffer the next read overwrites anyway

`resize` fills the new tail with zeros; every one of these then writes over all
of it. `resize_and_overwrite` hands the chunk over unwritten instead — 12.3 GB/s
to 16.7 GB/s on `read_u8s` against an in-memory stream, which is what a zip
entry is.

The callback must not throw, so a short read shrinks the string back to the
offset it started from and the throw happens at the call site. `ppt`'s
`read_raw_text_bytes` loses its second `resize` with it: the callback returns
`gcount()` and the string is already the right length.

`xls_io`'s string body is left alone — its `read_bytes` throws from inside, and
a non-throwing path just for this is not worth the buffer it saves.

* refactor: build the two derived vectors with `ranges::to`

Both loops did nothing but map a range onto a vector, and both fed it straight
into one call. `views::transform | ranges::to<std::vector<std::string>>()` says
that in the expression that uses it, so neither needs a named variable any more.

`type1_charstring` takes the iterator-pair `std::reverse` next to it with them,
per the ranges convention.

* test(build): hold the public headers to C++20

The bump made `odr` C++23 and deliberately did not pass that on — no
`target_compile_features(odr PUBLIC …)`, no `cppstd` in the conan
`package_info` — so a consumer keeps whatever standard it picked, as long as
`src/odr/*.hpp` stays C++20. Nothing checked that, and a `this auto &self` in a
public header would have broken someone else's build rather than ours.

`odr_public_headers_cpp20` includes all seventeen of them and compiles at
C++20. One object file, no test data, no link, no gtest — it either compiles or
it does not. It lives in `test/CMakeLists.txt` and so builds under `ODR_TEST`,
which is also what keeps it out of the conan package: `exports_sources` ships
no `test/`, and an unconditional target naming a file that is not there fails
the package build at generate time.

* docs: cut the prose from the comments this branch adds

The C++23-does-not-propagate rationale now lives in AGENTS.md alone; the
test file and its cmake target state what they are and stop there.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor: the per-format element registries, document classes and style registries are copy-paste siblings

1 participant