Re: Question about wal_compression and what to expect

[Jeff Janes <jeff.janes@xxxxxxxxx>]

On Wed, Jun 21, 2023 at 9:51 AM Sean O'Grady <sean@xxxxxxxxxxx> wrote:

Yes, my question was essentially do I need to separately compress WAL files for storage/archiving even with wal_compression set. So, I think you've answered my question.

The internal compression has to compress each record in isolation, which is not nearly as good as compressing the entire file as there are patterns that exist between records not just within records.  So yes, you can still usefully compress the files for archival.  But if that is your main goal, then maybe you should turn off wal_compression, as it actually interferes with the whole-file compression.  Compressing just once at the whole-file level gives me better overall compression than compression both internally and then whole-file.

You should test this for yourself in your own data, as compressibility is not uniform over all work loads.  But you need to make sure you do the test correctly, compressing the same amount of "work" for each set-up.  When you have wal_compression on, more "work" should fit into each WAL file, so just compressing the same number of WAL files will not give you a fair comparison.

Cheers,

Jeff