Re: Splitting large message written to stdout, explanation?

[Lennart Poettering <lennart@xxxxxxxxxxxxxx>]

On Mo, 22.05.23 15:58, Virendra Negi (virendra.negi@xxxxxxxxxxxxxxxxxxxx) wrote:

> I'm not sure how Systemd was handling this, but my assumption is that
> systemd redirects STDOUT , STDERR to  /*dev/log *and then systemd would
> pick that up and write to the respective file based. Given I found no help
> with rsyslog to deal with the large size log message (which are few in
> number) I looked at the journald conf.

"Standard{Output|Error}=syslog" is legacy. It's identical to
"Standard{Output|Error}=journal", and that's the default anyway. Hence
these two lines are entirely unnecessary, you can drop them without
change in behaviour

The journal daemon picks up the logs from stdout/stderr of various
services, from syslog, form the native journal protocol and writes it
to the journal files.

I have no idea about rsyslog and your distro, but secondary logging
services have two way to get ahold of the log data once journald
picked it up: they can listen on some AF_UNIX that systemd forwards
all mentioned log data. This is mostly a compat feature since it only
covers log data "as it happens", and that means not early boot/late
shutdown stuff. It also doesn't do structured loggic. The other way is
to simply read the data from journal files as the are updated, using
the files as a "live" transport, with the nice functionality that
secondary logging services can easily catch up with what happened
while they weren't running. And you get full structured data. I know
that RHEL configures rsyslog that way, but I think rsyslog upstream
used to be hostile to such an approach, so no idea, if that ever was
merged upstream.

> As mentioned you can use the _LINE_BREAK= field to reassemble the
> > lines. But seriously, if you are logging megabytes of data in single
> > log messages you are doing things wrong. Rivisit what you are doing
> > there, you are trying to hammer a square log message into a round log
> > transport. Bad idea.
>
> @Lennart How? JFI, this is what the split message of a large log message
> looks like.

Well, I think rsyslog has no idea about the journal's structured
logging, because it lives in its own world. It won't see the
_LINE_BREAK= structured logging. Hence you cannot reasonably
reassamble I guess, the info is simply lost once rsyslog takes over.

Lennart

--
Lennart Poettering, Berlin