From 464824323e57dc4b397e8b05854d779908b55304 Mon Sep 17 00:00:00 2001 From: Amit Kapila Date: Thu, 3 Sep 2020 07:54:07 +0530 Subject: Add support for streaming to built-in logical replication. To add support for streaming of in-progress transactions into the built-in logical replication, we need to do three things: * Extend the logical replication protocol, so identify in-progress transactions, and allow adding additional bits of information (e.g. XID of subtransactions). * Modify the output plugin (pgoutput) to implement the new stream API callbacks, by leveraging the extended replication protocol. * Modify the replication apply worker, to properly handle streamed in-progress transaction by spilling the data to disk and then replaying them on commit. We however must explicitly disable streaming replication during replication slot creation, even if the plugin supports it. We don't need to replicate the changes accumulated during this phase, and moreover we don't have a replication connection open so we don't have where to send the data anyway. Author: Tomas Vondra, Dilip Kumar and Amit Kapila Reviewed-by: Amit Kapila, Kuntal Ghosh and Ajin Cherian Tested-by: Neha Sharma, Mahendra Singh Thalor and Ajin Cherian Discussion: https://postgr.es/m/688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com --- doc/src/sgml/monitoring.sgml | 16 ++++++++++++++++ doc/src/sgml/ref/alter_subscription.sgml | 5 +++-- doc/src/sgml/ref/create_subscription.sgml | 11 +++++++++++ 3 files changed, 30 insertions(+), 2 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index d973e1149aa..673a0e73e45 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -1509,6 +1509,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser WALWrite Waiting for a write to a WAL file. + + LogicalChangesRead + Waiting for a read from a logical changes file. + + + LogicalChangesWrite + Waiting for a write to a logical changes file. + + + LogicalSubxactRead + Waiting for a read from a logical subxact file. + + + LogicalSubxactWrite + Waiting for a write to a logical subxact file. + diff --git a/doc/src/sgml/ref/alter_subscription.sgml b/doc/src/sgml/ref/alter_subscription.sgml index 81c4e70cdf4..a1666b370be 100644 --- a/doc/src/sgml/ref/alter_subscription.sgml +++ b/doc/src/sgml/ref/alter_subscription.sgml @@ -165,8 +165,9 @@ ALTER SUBSCRIPTION name RENAME TO < . See there for more information. The parameters that can be altered are slot_name, - synchronous_commit, and - binary. + synchronous_commit, + binary, and + streaming. diff --git a/doc/src/sgml/ref/create_subscription.sgml b/doc/src/sgml/ref/create_subscription.sgml index cdb22c54fea..b7d7457d004 100644 --- a/doc/src/sgml/ref/create_subscription.sgml +++ b/doc/src/sgml/ref/create_subscription.sgml @@ -228,6 +228,17 @@ CREATE SUBSCRIPTION subscription_name + + streaming (boolean) + + + Specifies whether streaming of in-progress transactions should + be enabled for this subscription. By default, all transactions + are fully decoded on the publisher, and only then sent to the + subscriber as a whole. + + + -- cgit v1.2.3