feat: allow source and destination clusters to overlap by levkk · Pull Request #1473 · pgdogdev/pgdog

levkk · 2026-09-01T21:07:02Z

Allow source and destination clusters to overlap. Matching databases in the destination cluster are ignored and data is only routed to shards not specified in the source cluster. This little hack allows us to:

Sync omni tables to new shards added to an existing cluster
Copy rows from existing nodes to new nodes with new sharding map

Example

[[databases]]
name = "source"
host = "10.0.0.0"

[[databases]]
name = "destination"
host = "10.0.0.0" # Same host as source!
shard = 0

[[databases]]
name = "destination"
host = "10.0.0.1"
shard = 1

[[sharded_tables]]
database = "destination"
column = "tenant_id"

[[sharded_tables.mapping]]
values = [1, 2, 3]
shard = 0

[[sharded_tables.mapping]]
values = [4]
shard = 1

Using data-sync, we can copy:

all omni tables
all rows matching tenant_id = 4

from source to destination, without touching the original node.

codecov · 2026-09-01T21:12:57Z

Codecov Report

❌ Patch coverage is 90.58824% with 16 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...src/backend/replication/logical/subscriber/copy.rs	69.69%	10 Missing ⚠️
...c/backend/replication/logical/subscriber/stream.rs	89.58%	5 Missing ⚠️
.../replication/logical/subscriber/duplicate_check.rs	90.90%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

levkk · 2026-09-02T17:25:29Z

Next on the TODO:

~~1. Add an --omnisharded-only flag to copy-data (and admin DB command)~~ Use CREATE PUBLICATION ... FOR TABLES to only sync omni tables instead.
2. Add a way to build a temporary lookup table and atomically swap it during cutover

meskill · 2026-09-03T14:09:34Z

+        let overlap_check = OverlappingShardsCheck::new(&self.source);

-        for shard in self.cluster.shards() {
-            let primary = shard
-                .pools_with_roles()
-                .iter()
-                .find(|(r, _)| r == &Role::Primary)
-                .ok_or(Error::NoPrimary)?
-                .1
-                .standalone(ConnectReason::Replication)
-                .await?;
-            conns.push(primary);
+        let dest_shards = self.dest.shards();
+
+        if dest_shards.is_empty() {
+            return Err(Error::DestinationNoShards);
+        }
+
+        for (shard_number, shard) in self.dest.shards().iter().enumerate() {
+            if overlap_check.overlaps(shard)? {
+                warn!(
+                    "skipping replication to shard {} because it overlaps with source cluster",
+                    shard.primary_address()?
+                );
+                continue;
+            }
+
+            let primary = shard.primary_standalone(ConnectReason::Replication).await?;
+            conns.push((shard_number, primary));
+        }
+
+        if conns.is_empty() {
+            return Err(Error::SourceDestinationIdentical);
        }


this is very similar code how it's done in the copy.rs, we may consider to put this inside OverlappingShardsCheck and reuse

meskill · 2026-09-03T14:16:44Z

+    // Connect to all shards.
    pub(crate) async fn connect(&mut self) -> Result<(), Error> {
-        let mut conns: Vec<Server> = vec![];
+        let mut conns = vec![];


oh, when the len of conns do not correspond to the len of destination shards it'll be more error prone if for any reason we'll use the conns index instead of shard_number inside for shard identification. I'll put the exact places right now what I found with this problem, but I wonder if we should cover it somehow on the type level:

use wrapper structure for conns

create empty Conn instead that do nothing to have the same len

maybe operate on Shard or similar instead of passing shard_number along the data

meskill · 2026-09-03T14:22:13Z

        }

        let partition = self.partition;
        let n_conns = self.connections.len();


here should be the dest.shards().len() I believe instead, since this relies on the behavior of shard routing that sees all shards

meskill · 2026-09-03T14:34:54Z


-        self.connections = servers;
+        for (shard_number, shard) in destination_shards.iter().enumerate() {
+            if overlap_check.overlaps(shard)? {


I wonder is we should use the Shard type or similar everywhere instead of tuples (shard_number, smth). There is a number already inside the Shard and we could simplify it in places like this and maybe reuse instead of adding shard_number to other structs

meskill · 2026-09-03T14:39:02Z

+                warn!(
+                    "skipping data sync to {} because it is part of the source cluster",
+                    shard.primary_address()?,
+                );
+                continue;


some edge case to validate maybe: check if the shard number is different in source/destination cluster, just in case

meskill · 2026-09-03T14:40:42Z

            let cancel = cancel.clone();
            handles.push(tasks::spawn("parallel sync manager", async move {
                let manager = ParallelSyncManager::new(tables, replicas, source, dest)?;
                let tables = manager.run(cancel).await?;


ParallelSync looks like using all the shards for destination_has_rows check, so for any error we'll stop retrying since we'll find the entries on the existing shards

meskill · 2026-09-03T14:47:33Z

Sounds like to this to work some invariants should hold, because if we change the number of shards the routing will expect the data be reorganized differently on the destination comparing to source and since we leave the data on source shards as is we should guarantee that the routing will work the same way as it was before.
I wonder if we can check this somehow and prevent issues? That relies on sharded_tables.mapping configuration, but I'm not sure we can use it to do the validation upfront if not only the configuration is exhaustive. Maybe we should abort in case we identified rerouting to another existing shard while doing the copy?

meskill · 2026-09-03T14:47:59Z

This should have some integration test

feat: allow source and destination clusters to overlap

e57cf06

levkk marked this pull request as draft September 1, 2026 21:08

fix test

81b514d

levkk requested a review from meskill September 2, 2026 17:23

levkk marked this pull request as ready for review September 2, 2026 17:24

warn

7033c39

meskill reviewed Sep 3, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat: allow source and destination clusters to overlap - #1473

feat: allow source and destination clusters to overlap#1473
levkk wants to merge 3 commits into
mainfrom
levkk-omni-only-sync

levkk commented Sep 1, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Sep 1, 2026 •

edited

Loading

Uh oh!

levkk commented Sep 2, 2026 •

edited

Loading

Uh oh!

meskill Sep 3, 2026

Uh oh!

meskill Sep 3, 2026

Uh oh!

meskill Sep 3, 2026

Uh oh!

meskill Sep 3, 2026

Uh oh!

meskill Sep 3, 2026

Uh oh!

meskill Sep 3, 2026

Uh oh!

meskill commented Sep 3, 2026

Uh oh!

meskill commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

levkk commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Example

Uh oh!

codecov Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

levkk commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

meskill Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

meskill Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

meskill Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

meskill Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

meskill Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

meskill Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

meskill commented Sep 3, 2026

Uh oh!

meskill commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

levkk commented Sep 1, 2026 •

edited

Loading

codecov Bot commented Sep 1, 2026 •

edited

Loading

levkk commented Sep 2, 2026 •

edited

Loading