Greg, I think you might be onto something here. your suggestion prompts
to get rid off --remove-sent-files in my rsync command, and clean up wal
files from master by some other means, this way even if master's rsync
picks partial file on the first run, it should update and complete file
once it's finished being copied by archive_command to my buffer directory.
Thanks! I'll get this in asap, if this does not solve the problem I'll
re-post
V.
Greg Smith wrote:
On Sat, 17 May 2008, Ioannis Tambouras wrote:
The archive command tests if the wal segment exists and is a file,
but it does not check if the file is still being written.
That's because it doesn't have to; the archive command doesn't get
called until the writing is done.
I don't have sources of pg_standby near me, but I remember in the
C code checks for complete segment sizes.
That's on the receiving side, to make sure it's not trying to process
files that haven't finished copying to the standby yet. You don't
have to do any of that in the archive_command.
Anyway, back to the original question:
archive_command = 'test ! -f /usr/local/wal_archive_local/%f && cp %p
/usr/local/wal_archive_local/%f'
archive files are then moved on master to standby every other minute:
rsync -aq --remove-sent-files /usr/local/wal_archive_local/
slave::wal_archive/
I don't see any mechanism here to keep rsync from copying over partial
files to the standby before they've finished copying to the
wal_archive_local directory. That's my guess for where the small
files are coming from, rsync before the cp is done. If you're going
to buffer in a transfer directory, you need some sort of test or
locking to make sure the file is complete with exactly 16MB before it
gets rsync'd over. I suspect no amount of poking at the standby will
root out the issue because it's happening on the primary.
--
* Greg Smith gsmith@xxxxxxxxxxxxx http://www.gregsmith.com Baltimore, MD
--
________________________________________
Vladimir (Vlad) Kosilov
Senior Systems Administrator
Contigo Systems Inc.
604.683.3106 (phone)
604.648.9886 (fax)
vkosilov@xxxxxxxxxxx
www.contigo.com
________________________________________
