Hi Simon,
I have been reading imas-codex over the last few days (the fork at
Simon-McIntosh/imas-codex, since that is where the current code lives) and
would like to contribute a small, test-only change. Before opening a PR I
want to check scope and a couple of behaviours with you.
What I have done
Going through the tools registered in imas_codex/llm/server.py and
grepping tests/ for each tool name, three had no test references:
get_dd_changelog, get_dd_migration_guide (its migration_guide module
is covered, only the handler is not) and signal_analytics.
I wrote mock-backed unit tests for the first and the last, no source
changes:
tests/tools/test_dd_changelog.py (15 tests): VersionTool.get_dd_changelog
parameter contract, result shaping, error path; format_dd_changelog_report
header, version-range labels, rows, truncation hint.
tests/llm/test_signal_analytics.py (33 tests): group_by validation,
routing between the plain query and the CHECKED_WITH join, check_status
and error_type filters, parameterisation, error handling, _format_analytics.
Branch: https://github.com/721AndrewWang/IMAS-Codex/tree/test/dd-changelog-signal-analytics
(based on your fork's main at 7745119). Locally: 48 passed, ruff check
and ruff format --check clean, tests/tools + tests/llm still 496 passed.
Two behaviours I pinned but did not change
_signal_analytics rejects unknown group_by values but silently drops
unknown filters keys. The test asserts the current behaviour and says so
in its docstring. If you would rather it error like group_by, I can
change the tool and flip the test in the same PR.
get_dd_changelog compares versions as strings in Cypher
(change.version > $from_version) while the same file has
_version_sort_key for numeric ordering. It works for the DD versions that
exist today (3.22 to 3.42, 4.x); flagging it in case it is unintended.
Two CI observations on the fork
- The last 40
Test runs all fail at "Format check with ruff". With the
locked ruff==0.15.1, two files need formatting:
tests/standard_names/test_detachment_attribution_wiring.py and
tests/standard_names/test_sourceless_name_recovery.py. Happy to include
that in the PR or leave it to you.
- Because those runs stop at the format step, the pytest step has not run
recently. Locally, the CI command uv sync --extra test --no-dev produces
an environment where tests/conftest.py fails to import
(ModuleNotFoundError: No module named 'imas_standard_names'), since
imas-standard-names is only declared in the dev group. Syncing with the
dev group works. This may bite once the format step is green.
Questions
- Should the PR target this repository or Simon-McIntosh/imas-codex?
- Anything in the test layout you would like done differently?
Thanks,
Jianan Wang
Hi Simon,
I have been reading imas-codex over the last few days (the fork at
Simon-McIntosh/imas-codex, since that is where the current code lives) and
would like to contribute a small, test-only change. Before opening a PR I
want to check scope and a couple of behaviours with you.
What I have done
Going through the tools registered in
imas_codex/llm/server.pyandgrepping
tests/for each tool name, three had no test references:get_dd_changelog,get_dd_migration_guide(itsmigration_guidemoduleis covered, only the handler is not) and
signal_analytics.I wrote mock-backed unit tests for the first and the last, no source
changes:
tests/tools/test_dd_changelog.py(15 tests):VersionTool.get_dd_changelogparameter contract, result shaping, error path;
format_dd_changelog_reportheader, version-range labels, rows, truncation hint.
tests/llm/test_signal_analytics.py(33 tests):group_byvalidation,routing between the plain query and the
CHECKED_WITHjoin,check_statusand
error_typefilters, parameterisation, error handling,_format_analytics.Branch: https://github.com/721AndrewWang/IMAS-Codex/tree/test/dd-changelog-signal-analytics
(based on your fork's
mainat 7745119). Locally: 48 passed,ruff checkand
ruff format --checkclean,tests/tools+tests/llmstill 496 passed.Two behaviours I pinned but did not change
_signal_analyticsrejects unknowngroup_byvalues but silently dropsunknown
filterskeys. The test asserts the current behaviour and says soin its docstring. If you would rather it error like
group_by, I canchange the tool and flip the test in the same PR.
get_dd_changelogcompares versions as strings in Cypher(
change.version > $from_version) while the same file has_version_sort_keyfor numeric ordering. It works for the DD versions thatexist today (3.22 to 3.42, 4.x); flagging it in case it is unintended.
Two CI observations on the fork
Testruns all fail at "Format check with ruff". With thelocked
ruff==0.15.1, two files need formatting:tests/standard_names/test_detachment_attribution_wiring.pyandtests/standard_names/test_sourceless_name_recovery.py. Happy to includethat in the PR or leave it to you.
recently. Locally, the CI command
uv sync --extra test --no-devproducesan environment where
tests/conftest.pyfails to import(
ModuleNotFoundError: No module named 'imas_standard_names'), sinceimas-standard-namesis only declared in thedevgroup. Syncing with thedev group works. This may bite once the format step is green.
Questions
Thanks,
Jianan Wang