Generalize the catalog contract: BeamMCP.Catalog carries tools, resources and prompts - #21
Merged
Merged
Conversation
…and prompts
BREAKING, to a host contract rather than to the wire. Taken now, before the
resources and prompts slices exist, so it is taken once.
(a) MIGRATION PATH: a clean break, no deprecated delegate.
The decisive argument is not taste, it is the version policy already in
place. Breaks land at the minor while this package is 0.x, and the README
recommends `~> 0.3.0`, which EXCLUDES 0.4.0 -- measured in slice 003's
version table. No consumer is carried across by a routine mix deps.update;
that tight pin exists for exactly this.
And a delegate would cost the thing the contract is for: keeping
ToolCatalog as a shim means two entry points into "which tools exist",
which is the second reader that decision (b) forbids.
(b) THE SINGLE-LOOKUP GUARANTEE: preserved, and strengthened.
tool_catalog.ex promised it in prose and left two call sites --
server.ex:185 called catalog.all() directly, Catalog.fetch/2 called it
again. Now both go through Catalog.tools/1, one function, so advertise and
call cannot drift apart without deleting it. Slice 002 fixed a real defect
of exactly this shape: tools/list honoured an injected catalog and
tools/call ignored it.
Pinned by effect, not by prose: catalog_test.exs compares what tools/list
advertises against what tools/call accepts, for a catalog whose tool exists
in no other catalog in the suite -- so neither path can pass by the
coincidence that made slice 002's original test green.
(c) THE fetch/2 @SPEC: documented honestly, not caught.
Catching would turn a host bug into :error, which is indistinguishable from
"no such tool" -- the advertise-versus-call confusion this behaviour exists
to prevent, reintroduced by the error handling meant to be defensive. The
raise conditions are listed in the @doc and one is pinned by a test.
THE COUNT IN PR #10's LEDGER IS WRONG AND IS CORRECTED. It claims three
shapes raise. Measured on this tree
(slices/008-catalog-generalization/logs/probe-fetch-spec.txt):
host spec.name is a binary ArgumentError
capabilities/0 returns a non-list map BadMapError <- not the
capabilities/0 returns nil Protocol.UndefinedError one error
module not loaded UndefinedFunctionError
Four, not three: "returns a non-list" is two different raises depending on
the shape. And a FIFTH case the ledger missed entirely, which is worse than
a raise because nothing fails: a bare map entry returns {:ok, %{name: :echo}}
SUCCESSFULLY, violating the @SPEC silently. That one is closed by init-time
validation rather than by documentation.
CALLBACK RENAMED, not just re-typed. Keeping all/0 while changing its return
from a list to a map compiles against every existing host and fails at the
first request with a BadMapError. capabilities/0 makes the break arrive at
compile time as an unimplemented callback. The option is :catalog, not
:tool_catalog, for the same reason: a catalog carrying resources and prompts is
not a tool catalog.
A malformed catalog is refused at Server.new/1 -- runtime, so calling the host
is safe. Transport.HTTP.init/1 checks only the export, deliberately: under
Plug's default init_mode it is the host's COMPILE time, and a correct catalog
reading config would fail there.
The call-site list I was given missed several: server.ex's alias, moduledoc,
typespec and new/1 key; stdio.ex's two moduledoc mentions; http.ex's alias,
moduledoc, opts lookup and the ArgumentError text naming all/0. server.ex:311
(dispatch) is NOT affected -- it consumes a ToolSpec, which is unchanged.
187 tests, 0 failures -- quoted from the run, and corrected: the first
version of this message typed 185, which no command had printed.
Gate OK. GATE_EXIT=0
Signed-off-by: Ayla Croft <aylacroft@proton.me>
…e reader
Mcat1 drops :prompts from @required_keys -- one key out of three, no message
change, no behaviour change for a correct host. If the smallest relaxation of
the check survives, the check is decoration. It dies twice over: at the startup
refusal, and independently at the README claim test, because the README promises
that refusal.
Mcat2 leaves tools/list reading the catalog and makes tools/call answer "which
tool does this name mean" from the request instead -- slice 002's defect in the
direction it actually occurred, advertise and call on different sources.
Mcat2's first version returned {:ok, spec} unconditionally. Elixir narrowed
find_tool/2's return to that shape, called the caller's `:error ->` clause
unreachable, and --warnings-as-errors failed the build: the score line came back
with no test count at all. That is a compiler kill, which records a kill that
never happened. It keeps an :error branch now so the suite scores it rather than
the compiler -- corrected so it could be scored, not weakened until it killed.
Every name a client can actually send is still callable under it.
Both targets differ from the harness default, so each is scored by its own
invocation with TARGET set; the mutant headers say which.
Signed-off-by: Ayla Croft <aylacroft@proton.me>
… scores PLAN and FINDINGS in slices/008-catalog-generalization/, plus the scored mutation archive. The PLAN specifies no review rounds, and says so rather than leaving the absence to be inferred: the brief was a single pass, nobody is scoring this slice, and a round boundary with no reviewer behind it is a heading. The mutants and the gate stand in. FINDINGS records four things that were measured rather than assumed: - PR #10's ledger says three host shapes make fetch/2 raise. Four do, and a fifth case the ledger missed is worse than any of them: a bare map entry RETURNS {:ok, %{name: :echo}} successfully, violating the @SPEC silently. All five are refused at Server.new/1 now. - The brief's call-site list was wrong in both directions -- two readers missing, one listed that is not affected because it consumes a ToolSpec. - Mcat2's first version was a compiler kill, left in the log above the corrected run. - 4a2d2e9's message typed "185 after" where the run printed 187. The commit was amended before anything was pushed; recorded here too, because an amended commit leaves no trace and the defect is the habit, not the digit. Also stated plainly: the Linear issue was filed at record time, not before the code, which is a departure from slice 007 and costs something real -- nothing external witnessed the acceptance criteria before they were met. Signed-off-by: Ayla Croft <aylacroft@proton.me>
Found after the records were written, by grepping for ToolCatalog across the whole tree rather than across lib/. Four tracked scripts still declared @behaviour BeamMCP.ToolCatalog, defined all/0, and passed tool_catalog: -- three separate breaks each, none of which the gate can see, because it never compiles tools/*.exs. They are not archives. tools/probe_ping.exs and the three network probes are instruments a later slice re-runs, and an instrument that no longer runs is the same defect as an archive nobody can refetch. Proven by running them, not by reading them: mix run tools/probe_ping.exs -> tools/list serves echo, exit 0 N=1 MIX_ENV=test mix run tools/probes/write_shape.exs -> PROBE_DONE, exit 0 N=1 MIX_ENV=test mix run tools/probes/drain_mechanism.exs -> PROBE_DONE, exit 0 N=1 MIX_ENV=test mix run tools/probes/loss_site.exs -> PROBE_DONE, exit 0 All three network probes still report {297, :econnreset, ..., true}, the shape slice 006 measured, so the contract change moved nothing they observe. Signed-off-by: Ayla Croft <aylacroft@proton.me>
Criterion 1 was written as "nothing in lib/ reads tool_catalog.ex" and was true while four tracked instruments outside lib/ were broken by the same change. The finding is the scope of the grep, not the four files. Signed-off-by: Ayla Croft <aylacroft@proton.me>
The publication step added in 009 refuses new tracked paths under slices/; these six were written before the records moved out of the tree. Copied to the internal repository and verified identical by diff -r before untracking, then removed. The allowlist is unchanged: it grandfathers published history, and this branch is not merged, so there is no published history here to grandfather. Signed-off-by: Ayla Croft <aylacroft@proton.me>
HackTuah
force-pushed
the
slice/008-catalog-generalization
branch
from
September 13, 2026 16:42
f19492a to
d0cbb8b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces
BeamMCP.ToolCatalogwithBeamMCP.Catalog, whose one callback carries all threecatalog-shaped concepts MCP has:
resourcesandpromptsare required and may be empty. Nothing reads them today. They existso that serving them later (SCR-283, SCR-284) adds a reader rather than changing this contract a
second time.
This is a breaking change, and it lands at a minor on purpose
Every host implementing
BeamMCP.ToolCatalogmust change. Under this package's 0.x policy abreak lands at a minor —
0.4.0— and the README recommends~> 0.3.0(README.md:17),which excludes
0.4.0. So no existing consumer is carried across this break bymix deps.update;a host that wants it asks for it. That policy is not a note in a file, it is pinned by
readme_claims_test.exswith amajor == 0guard.The callback is renamed too,
all/0→capabilities/0. Keeping the name while changing thereturn from a list to a map would compile against every existing host and fail at the first
request with a
BadMapError— a silent shape change. Renaming makes the break arrive at compiletime as an unimplemented callback, which is the loudest place it can arrive.
The three decisions
(a) Clean break, no deprecated delegate. Two reasons; the second is load-bearing. The version
policy above already protects consumers. And a delegate would create a second reader:
ToolCatalog.all/0living besideCatalog.capabilities/0lets a host implement both and makethem disagree, with nothing to catch it. The compatibility shim would reintroduce the exact defect
the contract exists to prevent, to soften a break the version policy already handles.
(b) The single-lookup guarantee is preserved and structurally stronger. Before,
tools/listcalled
catalog.all()andToolCatalog.fetch/2calledcatalog.all()— two call sites of thehost's function, agreeing by convention. Now there is one reader:
server.ex:185advertises through it;Catalog.fetch/2resolves through it. The host's functionis called from one place in
lib/, so the two paths cannot be given different sources withoutediting that line. Pinned by effect in
catalog_test.exs: the namestools/listadvertisesare compared against the names
tools/callaccepts, for a catalog whose tool exists in no othercatalog in the suite — so neither path can agree by coincidence, which is how slice 002's original
test could pass.
(c)
fetch/2's@specis documented honestly, not caught. It says{:ok, t} | :errorand amalformed host catalog makes it raise. Catching would make a host bug indistinguishable from "no
such tool": a broken catalog would present exactly as a working one that lacks that tool. That is
the advertise-versus-call disagreement this behaviour exists to prevent, reintroduced by the error
handling meant to be defensive.
Catalog.validate/1atServer.new/1refuses these at startupinstead, so the raise conditions are unreachable for a server that started.
The fetch/2 ledger was wrong, and the correction is worse than the count
PR #10's ledger claims three host shapes raise. Reproduced on this tree
(
slices/008-catalog-generalization/logs/probe-fetch-spec.txt) — four raise, and a fifth casethe ledger missed does not raise at all:
The fourth line is the one that matters. It returns
{:ok, %{name: :echo}}successfully,violating the
@specsilently and handing a struct-shaped thing that is not a struct to dispatch.A raise is a loud wrong answer; this is a quiet one. All five are refused at
Server.new/1now.Red before green
The init-time refusal was removed from
Server.new/1first(
logs/red-malformed-catalog.txt) — five tests, eachExpected exception ArgumentError but nothing was raised:Restored (
logs/green-malformed-catalog.txt):13 tests, 0 failures,TEST_EXIT=0.Mutation — both KILL, zero variance
logs/mutation-catalog.txt, tree8f47e1f80c4dc356b3d84d72eb1d468cfa461c49, 2 passes each. Twoinvocations, because the mutants have different targets:
Mcat1drops:promptsfrom@required_keys— the smallest possible relaxation, one key out ofthree. Killed by the startup refusal and, independently, by the README claim test, because the
README promises that refusal.
Mcat2leavestools/listreading the catalog and makestools/callanswer "which tool does this name mean" from the request — slice 002's defect in thedirection it actually occurred. Killed by the single-lookup test and eleven others.
lib/is byte-identical between the scored tree and the head of this branch(
git diff 8f47e1f HEAD -- lib/→ no output), so the table is a score for the tree that ships.Mcat2's first version was a compiler kill, and it is recorded as one
It returned
{:ok, spec}unconditionally. Elixir narrowedfind_tool/2's return to that shape,declared the caller's
:error ->clause unreachable, and--warnings-as-errorsfailed the build— the score line came back with no test count at all:
CONVENTIONS.md: a compiler kill records a kill that never happened. No test ran. The mutant keeps
an
:errorbranch now so the suite scores it instead of the compiler. Corrected so it could bescored, not weakened until it killed — every name a client can actually send is still callable
under it. The failed run is kept in the log above the corrected one.
The call-site list was measured, and the brief's was wrong in both directions
The brief named four readers. The tree has five:
server.ex:313andhttp.ex:225were missing from it;server.ex:311was on it and is notaffected, because it consumes a
ToolSpec, not the catalog.The two init paths check different things on purpose.
Server.new/1callscapabilities/0—host code, at runtime.
Transport.HTTP.init/1does not, because Plug's defaultinit_modeis:compile:init/1runs at the host's compile time, where a correct catalog reading config orETS would crash. It checks
function_exported?/3only and leaves the shape tonew/1.An amended commit, recorded rather than hidden
4a2d2e9's message read "172 tests before, 185 after". 185 was typed, not measured — the runprinted
187 tests, 0 failures. CONVENTIONS.md forbids exactly that. The commit was amendedbefore anything was pushed and is now
8f47e1f, and the correction is also written into FINDINGS,because an amended commit leaves no trace of the original and the defect is the habit, not the
digit.
Also in here: four tracked probes the gate cannot see
tools/probe_ping.exsand the three probes intools/probes/each carried three breaks from thischange —
@behaviour BeamMCP.ToolCatalog,def all, andcatalog:spelledtool_catalog:. Thegate never compiles
tools/*.exs, so it stayed green over all of them. Found by grepping thewhole tree rather than
lib/, fixed, and proven by running them:mix run tools/probe_ping.exsserves
echofromtools/list, and each network probe printsPROBE_DONEatN=1with the same{297, :econnreset, ..., true}slice 006 measured.That gap is a defect in the verification instrument, not in this slice, and is filed separately.
Gate, on the head of this branch
187 tests, 0 failures.
Records:
slices/008-catalog-generalization/. Issue: SCR-294.