On 1/12/24 14:03, David Ventimiglia wrote:
Reply to list also.
Ccing list
Hi Adrian,
I left out the creation of the slot both from the SO question and from
this mailing list question, because I believe it's a red herring. I run
into the same problem with the default output plugin as I do with the
wal2json plugin. However, the problem is a little different from what I
described. It turns out it's not output redirection from pg_recvlogical
sending to stdout that fails. Rather, it's output redirection from
pg_recvlogical sending to stdout and then passed through a pipeline that
fails. Or something like that.
This works. The sample.txt file is non-empty.
pg_recvlogical -d postgres --slot=test --create-slot
pg_recvlogical -d postgres -n --slot=test --start -f - > sample.txt
This does not work. The sample.txt file is empty.
pg_recvlogical -d postgres --slot=test --create-slot
pg_recvlogical -d postgres -n --slot=test --start -f - | awk
'{print}' > sample.txt
Weirdly, this works. The sample.txt file is non-empty.
pg_recvlogical -d postgres --slot=test --create-slot
pg_recvlogical -d postgres -n --slot=test --start -f - | cat >
sample.txt
FWIW, this is demonstrated in this screen-cast
<https://asciinema.org/a/631166>. I suspect it has something to do with
the flushing of buffers, as other people have suggested.
Thanks!
David
On Fri, Jan 12, 2024 at 1:48 PM Adrian Klaver <adrian.klaver@xxxxxxxxxxx
<mailto:adrian.klaver@xxxxxxxxxxx>> wrote:
On 1/12/24 11:34, David Ventimiglia wrote:
> Hello! How do I redirect logical decoding output from the
PostgreSQL CLI
> tool |pg_recvlogical| either to a file or to another command via
a pipe?
> I ask because when I try the obvious, no output is recorded or sent:
>
> |pg_recvlogical -d postgres --slot test --start -f - >>
sample.jsonl |
>
> Lest there be any confusion, I already created the slot in an
earlier
> step. Moreover, I can verify that if I omit the output
redirection |>>
> sample| then it does work, insofar as it emits the expected change
> events when I perform DML in another terminal window. When I
include the
> redirection (or alternatively, set up a pipeline), then nothing
happens.
You left out the important part of the SO question:
pg_recvlogical -d postgres --slot test --create-slot -P wal2json
I can get:
pg_recvlogical -d postgres --slot test --start -f - >> sample.jsonl
to work when I do:
pg_recvlogical -d postgres --slot test --create-slot
wal2json seems to be the issue.
>
> Note that I am aware of the option to pass a filename to the -f
switch
> to write to a file. That works, but it's not what I'm after
because it
> doesn't help update my mental model of how this is supposed to work.
> Based on my current (flawed) mental model built up from command line
> experience with other tools, this /should/ work. I should be
able to
> send the output to stdout and then redirect it to a file. It
surprises
> me that I cannot.
>
> Anyway, thanks!
>
> Best,
>
> David
>
--
Adrian Klaver
adrian.klaver@xxxxxxxxxxx <mailto:adrian.klaver@xxxxxxxxxxx>
--
Adrian Klaver
adrian.klaver@xxxxxxxxxxx
