examples: Separate extraction from manual generation - #377
Open
fingolfin wants to merge 6 commits into
Open
Conversation
Add AutoDocExtractExamples, which collects a package's manual examples by running its makedoc.g with only the parts of the build that extraction needs. Everything is written below a temporary directory, so the extracted tests need not be stored in the repository, and they can be produced from a read-only package directory such as a system-wide install. The settings describing a manual stay in makedoc.g alone: the mode is requested through a global option, the mechanism already used for nopdf, or through the AUTODOC_EXTRACT_ONLY environment variable for a makedoc.g which ends in QUIT and therefore has to run as a separate process. Generated XML now records where each example was written, as an XML comment which the GAPDoc output backends ignore. Extraction rewrites GAPDoc's origin list from those markers, so a failing example points at the .autodoc file or AutoDoc comment which has to be edited, rather than at the XML generated from it. Fenced markdown examples sitting directly in section text are not covered, as the parser records no source positions there. Fixes #376 AI disclosure: prepared with Claude Code (Claude Opus 5), which investigated the issue, proposed the approach and wrote the implementation under review. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fingolfin
force-pushed
the
claude/autodoc-issue-376-a4ac71
branch
from
August 28, 2026 23:35
4346d9e to
181ac4e
Compare
Source positions recorded for a worksheet were resolved against the working directory, since a worksheet has no package directory to resolve them against. The recorded paths therefore depended on where AutoDocWorksheet was called from, and the comparison is textual, so reaching the inputs through a symlink defeated it as well and left only the bare file name. Anchor them on the deepest directory the input files share instead. That is derived from the given paths alone, so the generated files no longer vary with the working directory or with symlinks. Paths recorded for a package are unaffected and stay relative to the package directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #377 +/- ##
==========================================
- Coverage 93.20% 93.03% -0.17%
==========================================
Files 13 15 +2
Lines 2958 3131 +173
==========================================
+ Hits 2757 2913 +156
- Misses 201 218 +17
🚀 New features to boost your workflow:
|
AutoDocExtractExamples reads a package's makedoc.g with the package directory as the working directory, using ChangeDirectoryCurrent. That function needs GAP 4.13, or the io package on older versions, while AutoDoc itself supports GAP 4.11; naming it unconditionally made loading AutoDoc warn about an unbound global there. Look the function up by name and skip the directory change when it is missing. The script is passed as an absolute path, so AutoDoc still locates the package either way; only a script reading further files by relative path needs the working directory, and that case is now reported and documented. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Exercise the argument handling of AutoDocExtractExamples and AUTODOC_ExtractOnlyDirectory, and the anchoring performed by AUTODOC_CommonParentDirectory, including the cases where inputs share no directory and where extract-only mode is off. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An Error raised inside a call does not pop the options stack on older GAP, so the invalid-value test for AutoDocExtractOnly left that option set for the remainder of the session and every later test failed on it. Clear the stack after provoking that error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ChangeDirectoryCurrent is in the GAP library since 4.13. It was previously obtained from the io package, which is not built in every installation, and AutoDocExtractExamples had to look the function up at runtime and carry on without it. Requiring GAP 4.13 removes both: the extraction code calls the function directly, and the tests no longer load io. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes it possible to run a package's manual examples from its test
suite without building the manual, and without storing the generated
.tstfiles in the repository.Extraction without a manual build
New
AutoDocExtractExamples, meant for a package'stst/testall.g:It reads the package's own
makedoc.g, so the settings describing themanual — source files, scaffolding,
extract_examplesoptions — are notduplicated. The mode is requested through a GAP global option, the same
mechanism already used for
nopdfand documented asRead( "makedoc.g" : nopdf ), somakedoc.gitself is untouched. For amakedoc.gending inQUIT, theAUTODOC_EXTRACT_ONLYenvironmentvariable does the same for a separate process.
The documentation directory is staged into a temporary directory and
generated into the copy. This is needed because
ComposedDocumentresolves relative
<#Include SYSTEM ...>against a single path, andhandwritten doc files commonly include generated ones by relative name
(
<#Include SYSTEM "_entities.xml">). As a side effect nothing iswritten into the package, so this also works when the package directory
is read-only, as with a distribution or system-wide install.
Real source locations
Extracted tests used to point at the intermediate XML file:
The parser already tracked file and line, but only for error messages.
Example nodes now record their position, generated XML carries it in an
XML comment which the GAPDoc backends ignore, and extraction rewrites
GAPDoc's origin list from those markers before parsing. Working on the
origin list rather than on the extraction results also covers chunks,
which are expanded at their insertion point.
This should also make the
changeSources-style rewriting asked for in#375 tractable, since the examples can now be traced back to the file
that has to be edited.
Reproducible paths for worksheets
A worksheet has no package directory to resolve source positions
against, so they were resolved against the working directory. The
recorded paths therefore depended on where
AutoDocWorksheetwascalled from, and since the comparison is textual, reaching the inputs
through a symlink defeated it as well and left only the bare file name.
They are now anchored on the deepest directory the input files share,
which is derived from the given paths alone. Paths recorded for a
package are unaffected and stay relative to the package directory.
Minimum GAP version
Raised to 4.13, which is where
ChangeDirectoryCurrententered thelibrary. It was previously available only from the
iopackage, whichis not built in every installation. The tests no longer load
ioandit is no longer listed in
TestPackages.Not covered
Fenced markdown examples sitting directly in section text get no marker
and keep the previous behaviour: the parser records no source positions
for that content at all. The
@BeginExamplefamily works everywhere,and fenced blocks work inside
@Descriptiontext and chunks.Testing
make checkpasses, from the package root, from an unrelated workingdirectory, and through a symlinked GAP root that reproduces the CI
layout — the last two being the configurations that caught the
worksheet path bug.
Beyond the package's own suite, extraction was checked against real
packages: for
congruenceandcircle— both usingscaffold.MainPage := falsewith a handwritten main page that includes_entities.xml— the extracted files are byte-identical to the onescommitted in those packages today.
Fixes #376
AI disclosure
Prepared with Claude Code (Claude Opus 5), which investigated the
issue, proposed the approach and wrote the implementation and tests
under review.
🤖 Generated with Claude Code