feat(bash): index shell scripts (functions, call edges, source imports) - #1612
Open
Nickcom4 wants to merge 1 commit into
Open
feat(bash): index shell scripts (functions, call edges, source imports)#1612Nickcom4 wants to merge 1 commit into
Nickcom4 wants to merge 1 commit into
Conversation
Shell was the biggest remaining gap for infrastructure repos: a hooks or scripts tree is dozens of small files wired together by `source`, and none of it was visible to the graph. Closes colbymchenry#239. Wiring is the usual four files plus one branch in extractCall. Two choices are worth calling out: - The grammar is VENDORED. tree-sitter-wasms ships an ABI-14 bash build that traps under web-tree-sitter 0.25 the moment it meets a `case` statement ("resolved is not a function", taking the whole parse with it), so it dies on nearly every real script. The prebuilt ABI-15 wasm from the tree-sitter-bash 0.25.1 npm package parses the same files cleanly; this is the same class of problem as the vendored Lua and C# grammars. - Builtins are filtered out of call edges. Every `command` node is a call site, so without a filter `echo`/`printf`/`local` outnumber real function calls by roughly an order of magnitude and bury them. External tools (`git`, `jq`) are deliberately kept as unresolved references, since that is what makes "which scripts shell out to jq" answerable. Measured on a 339-file hooks/scripts tree: 10,339 nodes and 19,772 edges in about 0.5s, 1,466 functions, 276 resolved `source` dependencies, and cross-file `callers`/`impact` working through them (one library function returned 20 callers across 20 different hook files, 147 affected symbols).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Shell / Bash support (
.sh,.bash,.zsh,.ksh,.bats). Closes #239.Infrastructure repos are the case this helps: a hooks or scripts tree is dozens of small files wired together by
source, and none of that was visible to the graph before.What it extracts
foo() { }andfunction foo { }commandnodesource x/. x, resolved to the sourced filereadonly/declare -rbecome constantsTwo choices worth reviewing
The grammar is vendored.
tree-sitter-wasmsships an ABI-14 bash build that traps under web-tree-sitter 0.25 the moment it meets acasestatement:The whole parse dies, so nearly every real script would have failed.
check-grammar.mjsdoes not catch it unless the sample happens to contain acase. The prebuilt ABI-15 wasm from thetree-sitter-bash0.25.1 npm package (MIT, byte-identical to the tarball artifact) parses the same files cleanly, so it is vendored undersrc/extraction/wasm/the same way Lua and C# are, and there is a regression test pinning thecasebehaviour.Builtins are filtered out of call edges. Every
commandnode is a call site, so without a filterecho,printfandlocaloutnumber real function calls by about an order of magnitude and bury them. Calls to external tools (git,jq,curl) are deliberately kept as unresolved references, since that is what makes "which scripts shell out to jq" answerable at all. The filter is a list of names afunctiondefinition can never legally take.Function-local
local x=...is deliberately not indexed; it is not a symbol anyone queries and it was roughly 6 locals per function on the corpus below.Extraction measured on a real shell tree
339 files of Claude Code hooks and operational scripts (about 170k lines):
Cross-file resolution through
sourceworks, which is the point of the feature:Diff
src/types.ts,src/extraction/grammars.ts,src/extraction/languages/{bash.ts,index.ts}: the standard four-file wiring plus the new extractorsrc/extraction/tree-sitter.ts: onebashbranch inextractCall(the generic path readsnamedChild(0), which is the assignment in aFOO=1 cmdprefix, and cannot drop builtins)src/resolution/import-resolver.ts:resolveBashSource, mirroringresolveLuaRequire. Sourced paths arrive as a literal tail (lib/_log.sh) because the directory is nearly always an expansion__tests__/extraction.test.ts: 6 extraction tests plus a detection testsrc/extraction/wasm/tree-sitter-bash.wasm: the vendored grammarFull suite green: 3,015 passed, 0 failed.
Not covered
/add-langskill.