[ANNOUNCE] jio is an experimental systemd-journald journal file tool utilizing io_uring

[Vito Caputo <vcaputo@xxxxxxxxxxx>]

Hello systemd-devel,

Recent discussion here about journal space consumption happened to
occur while I was exploring use of the new io_uring linux kernel
interface in combination with journal files.

What began as a curiosity about this new kernel interface, and what it
would be like to program journal file processing with some kind of
continuation-passing style C code bolted onto it, evolved into
something already providing new visibility into journal-file space
utilization not AFAIK currently offered by systemd's journalctl (not
that it couldn't be added there too).

I've called this program jio, pronounced "jai-oh".

At this time jio implements three basic functions:

1. `jio report usage`

    READ-ONLY

    Measures and reports space actually used by objects in all
    accessible journal files, classified by object type.


2. `jio report tail-waste`

    READ-ONLY

    Measures and reports unused space allocated to the tail ends of
    all journal files, classified by journal state.  Journal state
    being Online, Offline, or Archived.


3. `jio reclaim tail-waste`

    READ-WRITE  **MAKE BACKUPS!**

    Reclaim tail-waste from Archived journal files through truncation.


It'd be appreciated if interested parties would clone the repo and
give some success/fail feedback on compilation and maybe even running
the READ-ONLY report operations on real systems.  You might learn
something about your journals, and I might fix some important
breakages/incompatibilities.

I have more ideas I'd like to implement in jio, but this seemed
already worth sharing since it can give fresh insight into where your
journal space is going.


Clone jio recursively via git (there are submodules):

 `git clone --recursive git://git.pengaru.com/jio`

Or browse the code on the web:

 https://git.pengaru.com/cgit/jio/.git


It's a classic autotools bootstrap, configure, make dance.  It's a
standalone program that runs fine in-tree.  The included README has
more details.

You will need a recent kernel with working and enabled io_uring, and
liburing.  I'm personally using kernel version 5.9, and should warn I
experienced hard io_uring lockups on 5.9-rc2 early in this excursion,
but 5.9 has proven stable.

I'd like to eventually get to where jio can perform the journal
searching equivalent of `systemctl status unit` and explore how async
IO w/io_uring or simply alternative implementations may help improve
performance in this area.

Anyone interested in collaborating on this or sponsoring work in this
area feel free to contact me.

Cheers,
Vito Caputo
_______________________________________________
systemd-devel mailing list
systemd-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/systemd-devel