feat: package-repo ownership-evidence signal (CM-1394) by joanagmaia · Pull Request #4576 · linuxfoundation/crowd.dev

joanagmaia · 2026-09-08T17:16:32Z

Summary

Computes an ownership-match signal (matched/unmatched/no_evidence) between a package's declared namespace/maintainers and its linked repo owner, across all 8 declared-writer ecosystems (npm, pypi, cargo, rubygems, packagist, maven, go, nuget)
Persists the signal per package_repos row, feeding CM-1306 scoring's confidence tiering
Emits ingest-time counters (declared_matched, declared_unmatched, declared_no_evidence) per ecosystem batch log
ADR-0022 documents the design

Stacked on #4570 (CM-1393) — review only the CM-1394 commits on top.

cursor · 2026-09-08T17:16:40Z

PR Summary

High Risk
Changes deterministic package→repo confidence and best-repo selection across all ecosystems; TS/SQL matcher drift would skew Cargo scores, and a broad unmatched distribution after rescore could shift downstream aggregates.

Overview
Adds ownership evidence so declared package→repo links can be scored as matched, unmatched, or no_evidence, separating squatting links from legitimate ones via confidence rather than hard filters.

Database: New package_repos.ownership_match column plus SQL helpers package_repo_owner_key / package_repo_owner_match (Cargo’s set-based pipeline). package_repo_confidence now takes ownership_match and applies declared-only penalties (−0.25 unmatched, −0.10 no_evidence); rescoring reads the stored column. A rolling-deploy compat overload defaults missing writers to no_evidence.

Ingestion: Shared matchOwnership() compares normalized namespace/maintainer identities to the repo owner; each declared writer (npm, PyPI, Maven, Go, NuGet, Packagist, Rubygems core, Cargo bulk SQL) persists the result on upsert. Batch logs aggregate declared_matched / declared_unmatched / declared_no_evidence. deps.dev links store no_evidence; after the final merge chunk, non-GitHub rows with a competing GitHub sibling are rescored.

Platform: DAL claim params, conflict updates, and Tinybird packageRepos gain ownershipMatch; ADR-0022 documents the design.

^{Reviewed by Cursor Bugbot for commit 6e2f354. Bugbot is set up for automated code reviews on this repo. Configure here.}

Introduces the ownership_match column on package_repos and folds it into package_repo_confidence scoring: unmatched maintainer/namespace evidence on a declared link drops confidence by 0.25, missing evidence by 0.10, on top of the existing secondary-signal penalty. Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

Shared matchOwnership()/repoOwnerFromCanonical() helpers used by every ecosystem writer to compare a package's namespace/maintainers against the linked repo's owner and classify the result as matched, unmatched, or no_evidence. Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

cursor

Stale Bugbot comment from a previous run.

…-1394) Every ecosystem's declared-repo-link write (cargo, deps.dev, go, maven, npm, nuget, packagist, pypi) now computes ownershipMatch via the shared matchOwnership() util and persists it alongside the link, feeding the new package_repo_confidence penalty. Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

…CM-1394) Parse the authors field already present in the RubyGems gem response instead of adding a second fetchOwners call to the core loop, which runs over every rubygems package. Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

Copilot

Pull request overview

This PR extends the packages ingestion + scoring pipeline to compute and persist an ownership-evidence signal (matched / unmatched / no_evidence) for package→repo links, and to apply the corresponding declared-link score penalties via a widened package_repo_confidence() function. It also adds per-batch ingest counters and documents the design in ADR-0022.

Changes:

Add ownership_match handling end-to-end: writer computation in packages_worker, persistence in package_repos, and conflict-resolution support in the DAL.
Update confidence scoring: new package_repo_confidence() signature includes ownership evidence and applies declared-only penalties; deps.dev ingestion adds a rescore pass for cross-chunk competing-GitHub cases.
Add tests (Vitest for matcher + DAL scoring/integration updates) and document the decision (ADR-0022).

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
services/libs/data-access-layer/src/packages/repos.ts	Upserts now persist `ownership_match` and score via updated confidence call.
services/libs/data-access-layer/src/packages/repoConfidenceScoring.integration.test.ts	Updates integration scoring tests for new function signature and ownership penalties.
services/libs/data-access-layer/src/packages/repoConfidence.ts	Adds `PackageRepoOwnershipMatch`, parameter plumbing, and conflict-update handling.
services/libs/data-access-layer/src/packages/repoConfidence.test.ts	Updates unit tests for new claim defaults and SQL call generation.
services/libs/data-access-layer/src/osspckgs/types.ts	Adds note pointing to the new claim/scoring types location.
services/libs/data-access-layer/src/osspckgs/sqlFragments.ts	Updates best-repo-link ordering comment for confidence offset provenance.
services/apps/packages_worker/src/utils/ownershipMatch.ts	Implements ownership matching + counters and repo-owner extraction.
services/apps/packages_worker/src/utils/tests/ownershipMatch.test.ts	Adds unit tests for ownership matching and repo-owner extraction.
services/apps/packages_worker/src/rubygems/types.ts	Extends batch result shape to include declared ownership counters; adds authors field.
services/apps/packages_worker/src/rubygems/runRubyGemsCriticalLoop.ts	Initializes batch counters for RubyGems critical loop results.
services/apps/packages_worker/src/rubygems/runRubyGemsCoreLoop.ts	Computes ownership match from authors + repo owner; bumps counters per processed package.
services/apps/packages_worker/src/rubygems/normalize.ts	Parses `authors` into a normalized string array for ownership evidence.
services/apps/packages_worker/src/pypi/upsertProject.ts	Computes ownership match from maintainers vs repo owner and persists it on link upsert.
services/apps/packages_worker/src/pypi/activities.ts	Returns ownership match from ingest and logs per-batch counters.
services/apps/packages_worker/src/packagist/upsertPackageInfo.ts	Computes ownership match (vendor/maintainers vs repo owner) and persists it.
services/apps/packages_worker/src/packagist/activities.ts	Threads ownership match through ingest path and logs per-batch counters.
services/apps/packages_worker/src/packagist/tests/persistPackageInfo.test.ts	Updates unit tests to expect `ownershipMatch` persisted on repo link upserts.
services/apps/packages_worker/src/packagist/tests/ingest.test.ts	Updates mocks/expectations for new return shape including ownership match.
services/apps/packages_worker/src/nuget/types.ts	Extends NuGet batch result to include declared ownership counters.
services/apps/packages_worker/src/nuget/runNuGetEnrichmentLoop.ts	Computes ownership match (owners/authors vs repo owner), persists it, bumps counters.
services/apps/packages_worker/src/npm/upsertPackage.ts	Computes ownership match (scope/maintainers vs repo owner) and persists it.
services/apps/packages_worker/src/npm/activities.ts	Returns ownership match from ingest and logs per-batch counters.
services/apps/packages_worker/src/maven/runMavenEnrichmentLoop.ts	Computes ownership match from groupId + dev usernames, persists it, aggregates counters.
services/apps/packages_worker/src/go/activities.ts	Adds Go-specific owner extraction (module-path host gate + repo-owner fallback) and counters.
services/apps/packages_worker/src/deps-dev/workflows/ingestRepos.ts	Adds final-chunk rescore of non-GitHub links once competing GitHub repos exist.
services/apps/packages_worker/src/cargo/types.ts	Extends Cargo enrichment result to include declared ownership counters.
services/apps/packages_worker/src/cargo/normalizeRepos.ts	Carries repo owner in staging to support SQL-side matcher.
services/apps/packages_worker/src/cargo/enrich.ts	Uses SQL matcher to set `ownership_match`, returns ownership counts.
docs/adr/README.md	Adds ADR-0022 to the index table.
docs/adr/0022-package-repo-ownership-evidence.md	Documents the ownership-evidence design, normalization rules, and per-ecosystem evidence.
backend/src/osspckgs/migrations/V1788393600__package_repo_owner_match.sql	Adds `ownership_match` column and SQL normalizer/matcher functions for Cargo pipeline parity.
backend/src/osspckgs/migrations/V1788393601__no_evidence_ownership_penalty.sql	Widens `package_repo_confidence()` to include ownership evidence and updates rescore procedure.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…osystems (CM-1394) Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 3 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 6f6a6db. Configure here.}

Copilot

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 3 comments.

Fixes the DROP FUNCTION dependency failure (9-arg compat overload from V1788307300 still referenced the 10-arg signature), restores a rolling-deploy compat overload for the dropped 10-arg signature, restores the verified_at monotonic bump in both rescore paths so Tinybird's ReplacingMergeTree doesn't drop concurrent updates, adds ownership_match to the Tinybird packageRepos datasource and to the upsertPackageRepo audit diff, stops RubyGems from feeding free-text author names into ownership matching (ADR-0022 says that loop stays no_evidence), and fixes a stale migration reference in a comment. Also fixes prettier formatting drift in npm/pypi/packagist writers and a forbidden non-null assertion in ownershipMatch.ts that were failing the lint-format-services CI check. Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

Copilot

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

services/apps/packages_worker/src/maven/runMavenEnrichmentLoop.ts:30

OwnershipEvidence is a type-only symbol, but it’s imported via a value import. If the TS build preserves value imports, this can trigger a runtime ESM error (missing named export). Split this into an import type for OwnershipEvidence and a value import for the functions.
services/apps/packages_worker/src/nuget/types.ts:2
These imports are only used as types (interface extension and type annotations). Using value imports can cause runtime ESM errors if the compiler preserves them (e.g. “module ... does not provide an export named ...”). Switch to import type for both.
services/apps/packages_worker/src/rubygems/types.ts:2
These imports are only used as types (interface extension and type annotations). Using value imports can cause runtime ESM errors if the compiler preserves them (e.g. “module ... does not provide an export named ...”). Switch to import type for both.


 describe('packageRepoLinkClaimParams', () => {
-  it('defaults provenance and signal for claims that carry neither', () => {
+  it('defaults the signals CM-1393 and CM-1394 have not started writing yet', () => {


Copilot AI balanced review requested due to automatic review settings September 8, 2026 17:16

Copilot started reviewing on behalf of joanagmaia September 8, 2026 17:17 View session

joanagmaia added 2 commits September 8, 2026 18:19

cursor Bot reviewed Sep 8, 2026

View reviewed changes

Comment thread backend/src/osspckgs/migrations/V1788393600__package_repo_owner_match.sql

Comment thread services/apps/packages_worker/src/rubygems/runRubyGemsCoreLoop.ts

Comment thread services/apps/packages_worker/src/deps-dev/workflows/ingestRepos.ts

joanagmaia added 3 commits September 8, 2026 18:21

docs: add ADR-0022 for package-repo ownership-evidence signal (CM-1394)

9fb84f5

Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

Copilot AI reviewed Sep 8, 2026

View reviewed changes

Comment thread backend/src/osspckgs/migrations/V1788393601__no_evidence_ownership_penalty.sql

Comment thread services/libs/data-access-layer/src/osspckgs/sqlFragments.ts Outdated

feat: emit ingest-time ownership-match counters across all package ec…

6f6a6db

…osystems (CM-1394) Signed-off-by: Joana Maia <jmaia@contractor.linuxfoundation.org>

Copilot AI review requested due to automatic review settings September 8, 2026 17:37

joanagmaia force-pushed the feat/CM-1394-ownership-evidence branch from fb76cd7 to 6f6a6db Compare September 8, 2026 17:37

Copilot started reviewing on behalf of joanagmaia September 8, 2026 17:38 View session

cursor Bot reviewed Sep 8, 2026

View reviewed changes

Comment thread backend/src/osspckgs/migrations/V1788393601__no_evidence_ownership_penalty.sql

Comment thread backend/src/osspckgs/migrations/V1788393601__no_evidence_ownership_penalty.sql Outdated

Copilot AI reviewed Sep 8, 2026

View reviewed changes

Comment thread backend/src/osspckgs/migrations/V1788393601__no_evidence_ownership_penalty.sql

Comment thread services/libs/data-access-layer/src/packages/repos.ts

Comment thread docs/adr/0022-package-repo-ownership-evidence.md

Copilot AI review requested due to automatic review settings September 8, 2026 18:01

Copilot started reviewing on behalf of joanagmaia September 8, 2026 18:01 View session

Copilot AI reviewed Sep 8, 2026

View reviewed changes

Comment thread services/libs/data-access-layer/src/packages/repoConfidence.test.ts

describe('packageRepoLinkClaimParams', () => {

it('defaults provenance and signal for claims that carry neither', () => {

it('defaults the signals CM-1393 and CM-1394 have not started writing yet', () => {

Navigation Menu

Uh oh!

feat: package-repo ownership-evidence signal (CM-1394) - #4576

feat: package-repo ownership-evidence signal (CM-1394)#4576
joanagmaia wants to merge 7 commits into
feat/CM-1393-secondary-manifest-signalfrom
feat/CM-1394-ownership-evidence

joanagmaia commented Sep 8, 2026

Uh oh!

cursor Bot commented Sep 8, 2026 •

edited

Loading

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

joanagmaia commented Sep 8, 2026

Summary

Uh oh!

cursor Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cursor Bot commented Sep 8, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading