Skip to content

Tests for get_dd_changelog and signal_analytics, plus two CI observations #38

Description

@721AndrewWang

Hi Simon,

I have been reading imas-codex over the last few days (the fork at
Simon-McIntosh/imas-codex, since that is where the current code lives) and
would like to contribute a small, test-only change. Before opening a PR I
want to check scope and a couple of behaviours with you.

What I have done

Going through the tools registered in imas_codex/llm/server.py and
grepping tests/ for each tool name, three had no test references:
get_dd_changelog, get_dd_migration_guide (its migration_guide module
is covered, only the handler is not) and signal_analytics.

I wrote mock-backed unit tests for the first and the last, no source
changes:

  • tests/tools/test_dd_changelog.py (15 tests): VersionTool.get_dd_changelog
    parameter contract, result shaping, error path; format_dd_changelog_report
    header, version-range labels, rows, truncation hint.
  • tests/llm/test_signal_analytics.py (33 tests): group_by validation,
    routing between the plain query and the CHECKED_WITH join, check_status
    and error_type filters, parameterisation, error handling, _format_analytics.

Branch: https://github.com/721AndrewWang/IMAS-Codex/tree/test/dd-changelog-signal-analytics
(based on your fork's main at 7745119). Locally: 48 passed, ruff check
and ruff format --check clean, tests/tools + tests/llm still 496 passed.

Two behaviours I pinned but did not change

  1. _signal_analytics rejects unknown group_by values but silently drops
    unknown filters keys. The test asserts the current behaviour and says so
    in its docstring. If you would rather it error like group_by, I can
    change the tool and flip the test in the same PR.
  2. get_dd_changelog compares versions as strings in Cypher
    (change.version > $from_version) while the same file has
    _version_sort_key for numeric ordering. It works for the DD versions that
    exist today (3.22 to 3.42, 4.x); flagging it in case it is unintended.

Two CI observations on the fork

  • The last 40 Test runs all fail at "Format check with ruff". With the
    locked ruff==0.15.1, two files need formatting:
    tests/standard_names/test_detachment_attribution_wiring.py and
    tests/standard_names/test_sourceless_name_recovery.py. Happy to include
    that in the PR or leave it to you.
  • Because those runs stop at the format step, the pytest step has not run
    recently. Locally, the CI command uv sync --extra test --no-dev produces
    an environment where tests/conftest.py fails to import
    (ModuleNotFoundError: No module named 'imas_standard_names'), since
    imas-standard-names is only declared in the dev group. Syncing with the
    dev group works. This may bite once the format step is green.

Questions

  • Should the PR target this repository or Simon-McIntosh/imas-codex?
  • Anything in the test layout you would like done differently?

Thanks,
Jianan Wang

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions