Skip to content

ATLAS-5381: Handle empty/corrupted reference vertices during entity delete and purge traversal - #740

Open
sheetalshah1007 wants to merge 2 commits into
apache:masterfrom
sheetalshah1007:ATLAS-5381
Open

ATLAS-5381: Handle empty/corrupted reference vertices during entity delete and purge traversal#740
sheetalshah1007 wants to merge 2 commits into
apache:masterfrom
sheetalshah1007:ATLAS-5381

Conversation

@sheetalshah1007

@sheetalshah1007 sheetalshah1007 commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes delete and purge failures when an owned or related reference vertex is empty or corrupt. The vertex and relationship edge remain, but required properties such as __typeName and __guid are missing.

Error fixed: ATLAS-400-00-014: Type ENTITY with name null does not exist

This graph state is already handled on the read path, where broken relationship references can appear as null in arrays such as relationshipAttributes.columns (ATLAS-4605). Before this fix, the same state could cause DELETE or PURGE to fail.

This PR makes delete traversal resilient by skipping unreadable reference vertices and logging a warning. Valid vertices continue to be processed normally.

Changes in DeleteHandlerV1

isUnreadableEntityVertex() — new

Adds a common check for vertices that cannot be safely read as Atlas entities.

A vertex is considered unreadable when it is:

  • null or already removed
  • empty / has no properties
  • missing __guid
  • missing __typeName

This check is used across the delete traversal paths to avoid passing invalid vertices to entity-header or type-resolution logic.

resolveEntityType() — new

Safely resolves the entity type after validating the vertex.

If the vertex is unreadable or its type is unknown, the method returns null instead of allowing type lookup to fail. This protects callers that need the entity type for dataset/process-specific delete handling.

getOwnedVertices() — modified

This is the primary failure path for the original issue.

When traversing owned references such as a table's columns, unreadable or unknown-type vertices are now skipped with a warning instead of throwing ATLAS-400-00-014.

Valid owned vertices and the parent entity continue through the normal delete flow.

accumulateDeletionCandidates() — modified

Uses resolveEntityType() when determining the type of the root vertex.

If the root vertex cannot be resolved, dataset/process-specific processing is skipped rather than failing with an exception. The vertex can still be included in deletion candidates so cleanup can proceed.

addUpstreamProcessEntities() — modified

When deleting a dataset, Atlas can traverse linked process vertices.

Unreadable process vertices are now skipped before further processing, allowing the dataset delete to continue.

getColumnLineageEntities() — modified

When deleting a process, Atlas can traverse column-lineage relationships.

Unreadable process or lineage endpoint vertices are now skipped, preventing a corrupt lineage reference from blocking the delete.

deleteEdgeReference() — modified

When removing an edge, Atlas may update the entity at the other endpoint.

If that endpoint is unreadable, the inverse entity update is skipped. The edge deletion itself can continue without failing.

Tests

Four regression tests were added to cover the affected delete paths:

Test Corrupt vertex Coverage
testDeleteTableWithHollowOwnedColumnVertex Owned column getOwnedVertices() and owned-edge cleanup
testDeleteOutputTableWhenProcessVertexIsHollow Upstream process addUpstreamProcessEntities()
testDeleteProcessSkipsHollowColumnLineageVertex Column-lineage endpoint getColumnLineageEntities()
testDeleteSubordinateWhenManagerVertexIsHollow Manager vertex deleteEdgeReference() inverse update

Each test covers a separate scenario and removes the properties from the reference vertex to reproduce the corrupted graph state.

Existing ATLAS-4766 and ATLAS-5317 purge/resilience tests continue to provide coverage for related delete and purge scenarios.

Expected Outcome

Scenario Before After
Delete entity with an empty owned reference Delete fails with ATLAS-400-00-014 Delete completes; bad reference is skipped
Purge traversal encounters an empty reference Candidate expansion can fail Expansion continues
Delete encounters an empty process/lineage vertex Delete can fail Bad vertex is skipped
Edge deletion encounters an empty inverse endpoint Inverse update can fail Edge deletion continues; inverse update is skipped
GET encounters a broken reference May return null in relationship arrays No change

Known Limitations

  • Empty reference vertices are not repaired.
  • GET may continue to show null relationship entries until the affected entity/edges are cleaned up.
  • Vertices with unknown types may remain orphaned and require separate graph cleanup.
  • This change does not address how empty reference vertices are created.

…dlerV1Test to cover addUpstreamProcessEntities, getColumnLineageEntities, and deleteEdgeReference
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants