ts: add cookbook testsuite to validate documented recipes - #671
Merged
Conversation
completion_tcsh_start expects the prompt right after "set prompt=..."
using a bare "$completion_prompt\$" pattern. That command own echoed-
back input already contains the exact prompt text ("set prompt=
'MODULESTEST> '"), so under load Expect can match that mid-flight echo
before tcsh has even processed Enter, once a partial read happens to
land right after it. That races the rest of session setup and can leave
the completion script not fully sourced, surfacing later as a hang on
the first real completion listing. Require the same leading
carriage-return-plus-newline prefix completion_tcsh_raw already relies
on for its own matching, so this only matches the real prompt tcsh
prints once the command is done.
Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Xavier Delaruelle <xavier.delaruelle@cea.fr>
Add a 'cookbook' DejaGnu tool, alongside modules/install/lint/completion,
that builds a sandboxed fixture from the exact files a cookbook recipe
ships under doc/example/<name>/, then runs the commands demonstrated in
the recipe's "Usage example" documentation in the order and against the
output parsed straight out of the recipe's .rst file, rather than a
hand-copied duplicate that could silently drift from it.
cookbook_parse_transcript (005-init_ts.exp) walks a recipe's
'.. parsed-literal::' blocks and returns {command output} pairs, with
role markup stripped from the output. cookbook_output_re turns one such
illustrated output into a regexp a sandboxed run's real output is
checked against, widening the two kinds of detail a doc transcript
cannot reproduce as-is: a run of '-' padding of unpredictable width, and
whatever placeholder path a recipe test maps to its own sandbox path.
Implements the first recipe test, sync-remote-appdir.exp, which also
fixed the recipe doc's own inaccuracy caught along the way: 'module
load' was shown printing a "Loading foo/2.1" banner it does not actually
print at normal verbosity, now reflected by adding '-v' to the doc's
load commands instead.
Wired into 'make testcookbook' and 'script/mt cook'. Documented in
doc/source/devel/testsuite.rst.
Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Xavier Delaruelle <xavier.delaruelle@cea.fr>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cookbookDejaGnu tool, alongside modules/install/lint/completion, that builds a sandboxed fixture from the exact files a cookbook recipe ships underdoc/example/<name>/, then runs the commands demonstrated in the recipe's "Usage example" documentation in order and checks that similar output is obtained, parsed straight out of the recipe's.rstfile rather than a hand-copied duplicate that could silently drift from it.sync-remote-appdir.exp, which caught (and fixed) a real inaccuracy in the recipe doc itself:module loadwas shown printing a "Loading foo/2.1" banner it does not actually print at normal verbosity — now reflected by adding-vto the doc's load commands instead.make testcookbookandscript/mt cook, documented indoc/source/devel/testsuite.rst.ts: anchor tcsh initial prompt match to a fresh output line, fixing a race in the tcsh completion testsuite (from issue-tcsh-completion-timeout).