refactor(dedupe): one definition of the location prefetch

[valentijnscholten]

Summary

• Which location relation to prefetch is a property of what Finding.get_locations() reads, not of any individual caller — but it was spelled out separately at every call site, and that drifts in both directions. The hash-recompute paths disagreed about it for months, and the batch dedupe loader prefetched the deprecated endpoints relation, which raises under V3_FEATURE_LOCATIONS on an instance migrated from endpoints.
• Adds location_prefetch_lookups(prefix="") to dojo/location/queries.py, next to vulnerability_id_prefetch, and routes build_candidate_scope_queryset and manage.py dedupe through it.
• The dedupe command's V3 branch prefetched locations — one hop — where get_locations() reads location.url, leaving a query per location reference. It now prefetches the full path like the other call sites, and its two near-identical select_related/prefetch_related blocks collapse into one.
• get_finding_models_for_deduplication is deliberately left alone: fix(dedupe): prefetch locations, not endpoints, in the batch dedupe loader #15508 fixes the crash there and re-baselines the perf query counts that change shifts. I measured that change independently while scoping this PR — +2 queries on each V3 step, and −104 on the second import — which matches fix(dedupe): prefetch locations, not endpoints, in the batch dedupe loader #15508's numbers, so re-deriving them here would only create a conflict. A TODO marks the one-line swap to the helper once it lands.
• unittests/test_dedupe_location_prefetch.py pins the helper (both flag states, the full location.url path, and the relation-prefix form) and asserts both build_candidate_scope_queryset modes use it. Queryset shape only — no fixtures, no timing. 6 tests pass, and test_deduplication_logic still passes at 85.
• The prefix argument exists for callers that page a model reaching the finding through a relation rather than paging Finding itself.

Extension points instead of a fork (second commit)

• Distributions that store extra hash fields or need extra scoping have maintained near-verbatim copies of this command, and a copy is what lets them drift — the vulnerability-id prefetch was fixed in a copy first and had to be fixed again elsewhere months later, and the location prefetch was fixed elsewhere and never reached a copy, leaving it prefetching the deprecated endpoint relation.
• Such a distribution can now subclass this command and override a hook: extra arguments, extra scope and its description, the hash generator, an extra recompute pass, the two batch-dedupe entry points, and product grading. Narrow on purpose — a stale override is a signature mismatch rather than silent drift.
• Behaviour is unchanged; every default does what the code did before. The scope build is unified into one queryset that filters down instead of a parser/no-parser if/else, which is what makes an extra scope filter composable.
• unittests/test_dedupe_command_hooks.py covers both halves: every hook has a working default, and a subclass's hooks are actually reached — including that --dedupe_only skips the extra hash pass, and that both batch paths and grading go through their hooks. This command had no tests at all before. 10 tests, verified locally.

[dryrunsecurity]

This pull request contains a critical finding where a sensitive codepath in 'dojo/finding/deduplication.py' was modified by an author not on the allowed list.

🔴 Configured Sensitive Codepath Modified by Non-Allowed Author in dojo/finding/deduplication.py (drs_66627b32)

Vulnerability	Configured Sensitive Codepath Modified by Non-Allowed Author
Description	File 'dojo/finding/deduplication.py' matches configured sensitive codepath pattern 'dojo/finding/*.py' and was modified by '' (commit 91af03a) who is not in the allowed authors list.

We've notified @mtesauro.

---

Comment to provide feedback on these findings.

Report false positive: @dryrunsecurity fp [FINDING ID] [FEEDBACK]
Report low-impact: @dryrunsecurity nit [FINDING ID] [FEEDBACK]

Example: @dryrunsecurity fp drs_90eda195 This code is not user-facing

All finding details can be found in the DryRun Security Dashboard.