Skip to content

examples: Separate extraction from manual generation - #377

Open
fingolfin wants to merge 6 commits into
masterfrom
claude/autodoc-issue-376-a4ac71
Open

examples: Separate extraction from manual generation#377
fingolfin wants to merge 6 commits into
masterfrom
claude/autodoc-issue-376-a4ac71

Conversation

@fingolfin

@fingolfin fingolfin commented Aug 28, 2026

Copy link
Copy Markdown
Member

Makes it possible to run a package's manual examples from its test
suite without building the manual, and without storing the generated
.tst files in the repository.

Extraction without a manual build

New AutoDocExtractExamples, meant for a package's tst/testall.g:

LoadPackage( "mypkg" );
dirs := DirectoriesPackageLibrary( "mypkg", "tst" );
Add( dirs, AutoDocExtractExamples( "mypkg" ) );
TestDirectory( dirs, rec( exitGAP := true ) );

It reads the package's own makedoc.g, so the settings describing the
manual — source files, scaffolding, extract_examples options — are not
duplicated. The mode is requested through a GAP global option, the same
mechanism already used for nopdf and documented as
Read( "makedoc.g" : nopdf ), so makedoc.g itself is untouched. For a
makedoc.g ending in QUIT, the AUTODOC_EXTRACT_ONLY environment
variable does the same for a separate process.

The documentation directory is staged into a temporary directory and
generated into the copy. This is needed because ComposedDocument
resolves relative <#Include SYSTEM ...> against a single path, and
handwritten doc files commonly include generated ones by relative name
(<#Include SYSTEM "_entities.xml">). As a side effect nothing is
written into the package, so this also works when the package directory
is read-only, as with a distribution or system-wide install.

Real source locations

Extracted tests used to point at the intermediate XML file:

-# doc/_Chapter_Overview.xml:61-73
+# gap/CanonicalPcPres.gd:53-65

The parser already tracked file and line, but only for error messages.
Example nodes now record their position, generated XML carries it in an
XML comment which the GAPDoc backends ignore, and extraction rewrites
GAPDoc's origin list from those markers before parsing. Working on the
origin list rather than on the extraction results also covers chunks,
which are expanded at their insertion point.

This should also make the changeSources-style rewriting asked for in
#375 tractable, since the examples can now be traced back to the file
that has to be edited.

Reproducible paths for worksheets

A worksheet has no package directory to resolve source positions
against, so they were resolved against the working directory. The
recorded paths therefore depended on where AutoDocWorksheet was
called from, and since the comparison is textual, reaching the inputs
through a symlink defeated it as well and left only the bare file name.

They are now anchored on the deepest directory the input files share,
which is derived from the given paths alone. Paths recorded for a
package are unaffected and stay relative to the package directory.

Minimum GAP version

Raised to 4.13, which is where ChangeDirectoryCurrent entered the
library. It was previously available only from the io package, which
is not built in every installation. The tests no longer load io and
it is no longer listed in TestPackages.

Not covered

Fenced markdown examples sitting directly in section text get no marker
and keep the previous behaviour: the parser records no source positions
for that content at all. The @BeginExample family works everywhere,
and fenced blocks work inside @Description text and chunks.

Testing

make check passes, from the package root, from an unrelated working
directory, and through a symlinked GAP root that reproduces the CI
layout — the last two being the configurations that caught the
worksheet path bug.

Beyond the package's own suite, extraction was checked against real
packages: for congruence and circle — both using
scaffold.MainPage := false with a handwritten main page that includes
_entities.xml — the extracted files are byte-identical to the ones
committed in those packages today.

Fixes #376

AI disclosure

Prepared with Claude Code (Claude Opus 5), which investigated the
issue, proposed the approach and wrote the implementation and tests
under review.

🤖 Generated with Claude Code

Add AutoDocExtractExamples, which collects a package's manual
examples by running its makedoc.g with only the parts of the build
that extraction needs. Everything is written below a temporary
directory, so the extracted tests need not be stored in the
repository, and they can be produced from a read-only package
directory such as a system-wide install.

The settings describing a manual stay in makedoc.g alone: the mode
is requested through a global option, the mechanism already used
for nopdf, or through the AUTODOC_EXTRACT_ONLY environment
variable for a makedoc.g which ends in QUIT and therefore has to
run as a separate process.

Generated XML now records where each example was written, as an
XML comment which the GAPDoc output backends ignore. Extraction
rewrites GAPDoc's origin list from those markers, so a failing
example points at the .autodoc file or AutoDoc comment which has
to be edited, rather than at the XML generated from it.

Fenced markdown examples sitting directly in section text are not
covered, as the parser records no source positions there.

Fixes #376

AI disclosure: prepared with Claude Code (Claude Opus 5), which
investigated the issue, proposed the approach and wrote the
implementation under review.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@fingolfin
fingolfin force-pushed the claude/autodoc-issue-376-a4ac71 branch from 4346d9e to 181ac4e Compare August 28, 2026 23:35
Source positions recorded for a worksheet were resolved against the
working directory, since a worksheet has no package directory to
resolve them against. The recorded paths therefore depended on where
AutoDocWorksheet was called from, and the comparison is textual, so
reaching the inputs through a symlink defeated it as well and left
only the bare file name.

Anchor them on the deepest directory the input files share instead.
That is derived from the given paths alone, so the generated files no
longer vary with the working directory or with symlinks. Paths
recorded for a package are unaffected and stay relative to the
package directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.45283% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.03%. Comparing base (90ce1f7) to head (079b3f5).

Files with missing lines Patch % Lines
gap/ToolFunctions.gi 88.72% 15 Missing ⚠️
gap/Examples.gi 96.00% 2 Missing ⚠️
gap/Magic.gi 94.87% 2 Missing ⚠️
gap/AutoDocMainFunction.gi 94.44% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #377      +/-   ##
==========================================
- Coverage   93.20%   93.03%   -0.17%     
==========================================
  Files          13       15       +2     
  Lines        2958     3131     +173     
==========================================
+ Hits         2757     2913     +156     
- Misses        201      218      +17     
Files with missing lines Coverage Δ
gap/DocumentationTree.gi 95.66% <100.00%> (-0.44%) ⬇️
gap/Examples.gd 100.00% <100.00%> (ø)
gap/Magic.gd 100.00% <ø> (ø)
gap/Markdown.gi 97.23% <100.00%> (+0.01%) ⬆️
gap/Parser.gi 96.71% <100.00%> (+<0.01%) ⬆️
gap/ToolFunctions.gd 100.00% <100.00%> (ø)
gap/AutoDocMainFunction.gi 93.58% <94.44%> (+0.43%) ⬆️
gap/Examples.gi 96.00% <96.00%> (ø)
gap/Magic.gi 75.47% <94.87%> (+1.56%) ⬆️
gap/ToolFunctions.gi 93.29% <88.72%> (-1.86%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

fingolfin and others added 4 commits August 31, 2026 08:41
AutoDocExtractExamples reads a package's makedoc.g with the package
directory as the working directory, using ChangeDirectoryCurrent.
That function needs GAP 4.13, or the io package on older versions,
while AutoDoc itself supports GAP 4.11; naming it unconditionally
made loading AutoDoc warn about an unbound global there.

Look the function up by name and skip the directory change when it is
missing. The script is passed as an absolute path, so AutoDoc still
locates the package either way; only a script reading further files by
relative path needs the working directory, and that case is now
reported and documented.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Exercise the argument handling of AutoDocExtractExamples and
AUTODOC_ExtractOnlyDirectory, and the anchoring performed by
AUTODOC_CommonParentDirectory, including the cases where inputs share
no directory and where extract-only mode is off.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An Error raised inside a call does not pop the options stack on older
GAP, so the invalid-value test for AutoDocExtractOnly left that option
set for the remainder of the session and every later test failed on it.
Clear the stack after provoking that error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ChangeDirectoryCurrent is in the GAP library since 4.13. It was
previously obtained from the io package, which is not built in every
installation, and AutoDocExtractExamples had to look the function up
at runtime and carry on without it.

Requiring GAP 4.13 removes both: the extraction code calls the
function directly, and the tests no longer load io.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make it easier separate generating the manual from extracting example, remove need to store extracted .tst files

1 participant