Re: PgSQL 14 - Logical Rep - Single table multiple publications?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 02/08/22, Robert Blayzor (rblayzor.bulk@xxxxxxxx) wrote:
> Is it possible to have a single subscriber table contact multiple publishers
> and just insert all of the data into a single table on the subscriber? ie:
> merge type replication. There are no primary/FK constraints, etc.  The
> records are just time based audit log type data...

Your use case meets, I think, the third "typical use case" listed at
https://www.postgresql.org/docs/current/logical-replication.html, namely
"Consolidating multiple databases into a single one (for example for
analytical purposes)."

I've just been testing aggregating all the data in one schema across 300
publisher databases into 5 subscriber schemas on two Postgresql 14 clusters on
the same machine. Each of 60 publisher tables are aggregating into a
single table on the subscriber.

Special care must be taken with the "replica identity" of published
tables, as set out at
https://www.postgresql.org/docs/current/logical-replication-publication.html.
For example, you may need a unique identifying column for each source
table in addition to the normal row identifier to differentiate *this*
table's id 1 row from the *other* table's id 1 row, otherwise the
subscriber won't be able to identify the row to delete if this table's
id 1 row is deleted (for example).

Although this seems to work fine with native replication, the pglogical
extension has more knobs. For instance, the
pglogical.wait_for_subscription_sync_complete function is useful to ensure that
sync finishes when part of a migration.

Rory





[Postgresql General]     [Postgresql PHP]     [PHP Users]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Yosemite]

  Powered by Linux