Re: PostgreSQL logical replication depends on WAL segments?

[Achilleas Mantzios <achill@xxxxxxxxxxxxxxxxxxxxx>]

On 22/1/19 6:01 μ.μ., Josef Machytka wrote:

https://www.postgresql.org/docs/11/logical-replication-architecture.html

Thanks, I see... So if I understand it correctly - since I have quite big partitions like ~30 GB each in one parent table and from ~1GB to ~5 GB in several others I presume I had to set wal_keep_segments to some really high number and stop our security cronjob cleaning old WAL segments (because we already had some problems with almost full disk due to old WAL segments)

I asked you if you had any cron jobs messing around with your WALs. The WALs in pg_wal dir is PostgreSQL's business to manage, not your scripts. Dont do that. Defining wal_keep_segments to a very large value or using replication slots (as with logical repl) won't help if your crons are deleting the WALs.
Now what you have to do :
- read up about the basic concepts of pgsql administration : https://www.postgresql.org/docs/11/admin.html
- understand how postgresql replication slots work (logical repl is based on those)
- understand how WAL archiving works
- understand how streaming replication works
- understand how logical replication works
- understand how the various runtime configuration params act on the number of WALs kept

until the whole transfer of snapshot is done. Because only after the whole snapshot is transferred logical replication workers start to transfer WAL logs reflecting changes done from the moment snapshot was taken...

jm

-- 
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt