Skip to content

fix: handle empty SMW ask results in semantic_search - #150

Open
LukasGold wants to merge 1 commit into
mainfrom
fix/semantic-search-empty-results
Open

fix: handle empty SMW ask results in semantic_search#150
LukasGold wants to merge 1 commit into
mainfrom
fix/semantic-search-empty-results

Conversation

@LukasGold

Copy link
Copy Markdown
Contributor

Fixes the crash behind #145 and explains the "random results" in #111.

Root causes

Three separate defects, all reproduced live against healthbatt.projects01.open-semantic-lab.org.

1. Zero-result queries raise AttributeError (fixed here)

SMW's action=ask serialises a non-empty result set as a JSON object keyed by page title, but an empty one as a JSON array. semantic_search called .values() on it unconditionally (src/osw/wiki_tools.py:257), so every zero-result query raised AttributeError: 'list' object has no attribute 'values' instead of returning [].

[[HasUuid::b5cbec50-fb7c-4c65-b208-2a20c6c32fb8]]  -> results: []   (list)
[[HasLabel::BMD_StandardCU_HealthBatt]]            -> results: {..} (dict)

Affects single queries, sequential batches and parallel batches. return_json=True happened to survive because it returns before that line. OSW.query_instances() (src/osw/core.py:2007) is affected too, so querying a category with no instances raised instead of returning an empty list.

2. Silent truncation at limit (surfaced here)

SearchParam.limit defaults to 1000 and there is no pagination, so a larger match set is silently reduced to an arbitrary subset. Measured on healthbatt: [[Category:Item]] matches 2637 pages, semantic_search returned 1000 with no indication.

3. exists flickers server-side (surfaced here, not fixable in this library)

The exists == "1" filter silently discards results. The field only ever takes '1' or '', but it is not stable: across 8 identical calls, 8 pages flipped between the two, and no page was consistently ''. The flips are perfectly correlated across pages (calls 0,2,3,4,7 agree; calls 1,5,6 agree), which points at inconsistent state between backends or an SMW query-result cache rather than at real page deletions. Consecutive identical calls returned 998 or 994 titles.

This is the "random results" from #111. It needs to be addressed on the OSL instance side; this PR only makes the loss visible.

Changes

  • _ask_results_as_dict() normalises the ask payload to a mapping, so an empty result set yields [] instead of raising
  • warnings.warn when the result count meets the requested limit, emitted before the return_json early return so both return modes get it
  • warnings.warn when entries are dropped by the exists != "1" filter, naming how many
  • 7 unit tests covering zero results in both return modes, sequential and parallel batches with an empty query, both warnings, and the no-warning case

Filter semantics, the default limit, and prefix_search are unchanged.

Verification

  • pytest tests/ --ignore=tests/integration -> 61 passed, 1 skipped
  • New tests confirmed failing first with the expected AttributeError / DID NOT WARN
  • Re-ran the live probes against healthbatt: sequential batch, parallel batch and return_json with a zero-result query all return results now instead of raising; both warnings fire with real counts

Out of scope, noticed while probing

single_query += f"|limit={query.limit}" is appended unconditionally, so a caller-supplied limit in the query string is silently overridden: [[Category:Item]]|limit=2 is sent as [[Category:Item]]|limit=2|limit=1000 and SMW honours the last one. Verified live (returned 1000, not 2). Not changed here.

- SMW serialises an empty ask result set as a JSON array, not an object,
  so any zero-result query raised AttributeError instead of returning []
- normalise the payload once via _ask_results_as_dict()
- warn when a query hits its limit, which silently truncated results
- warn when entries are dropped by the exists != "1" filter
- refs #145, #111
@github-actions

Copy link
Copy Markdown
Contributor

Release preview

Merging this PR would release v2.0.3 (current: v2.0.2).

Changelog preview (truncated)
## v2.0.3 (2026-08-31)

### Bug Fixes

- Handle empty SMW ask results in semantic_search
  ([`e00ca8b`](https://github.com/OpenSemanticLab/osw-python/commit/e00ca8be80eb1575a520b609d5d988604ad59e24))

### Testing

- Rename oold.py to oold_test.py so its tests are collected
  ([`20072a9`](https://github.com/OpenSemanticLab/osw-python/commit/20072a9249cd97126a222c62a70f84e0433343ef))

Preview via python-semantic-release and conventional commits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant