On Tue, Nov 26, 2024 at 6:09 PM Sergey Senozhatsky <senozhatsky@xxxxxxxxxxxx> wrote: > > On (24/11/22 11:25), Barry Song wrote: > > When large folios are compressed at a larger granularity, we observe > > a notable reduction in CPU usage and a significant improvement in > > compression ratios. > > > > This patchset enhances zsmalloc and zram by adding support for dividing > > large folios into multi-page blocks, typically configured with a > > 2-order granularity. Without this patchset, a large folio is always > > divided into `nr_pages` 4KiB blocks. > > > > The granularity can be set using the `ZSMALLOC_MULTI_PAGES_ORDER` > > setting, where the default of 2 allows all anonymous THP to benefit. > > I can't say that I'm in love with this part. > > Looking at zsmalloc stats, your new size-classes are significantly > further apart from each other than our tradition size classes. > For example, with ZSMALLOC_CHAIN_SIZE of 10, some size-classes are > more than 400 (that's almost 10% of PAGE_SIZE) bytes apart > > // stripped > 344 9792 > 348 10048 > 351 10240 > 353 10368 > 355 10496 > 361 10880 > 368 11328 > 370 11456 > 373 11648 > 377 11904 > 383 12288 > 387 12544 > 390 12736 > 395 13056 > 400 13376 > 404 13632 > 410 14016 > 415 14336 > > Which means that every objects of size, let's say, 10881 will > go into 11328 size class and have 447 bytes of padding between > each object. > > And with ZSMALLOC_CHAIN_SIZE of 8, it seems, we have even larger > padding gaps: > > // stripped > 348 10048 > 351 10240 > 353 10368 > 361 10880 > 370 11456 > 373 11648 > 377 11904 > 383 12288 > 390 12736 > 395 13056 > 404 13632 > 410 14016 > 415 14336 > 418 14528 > 447 16384 > > E.g. 13632 and 13056 are more than 500 bytes apart. > > > swap-out time(ms) 68711 49908 > > swap-in time(ms) 30687 20685 > > compression ratio 20.49% 16.9% > > These are not the only numbers to focus on, really important metrics > are: zsmalloc pages-used and zsmalloc max-pages-used. Then we can > calculate the pool memory usage ratio (the size of compressed data vs > the number of pages zsmalloc pool allocated to keep them). To address this, we plan to collect more data and get back to you afterwards. From my understanding, we still have an opportunity to refine the CHAIN SIZE? Essentially, each small object might cause some waste within the original PAGE_SIZE. Now, with 4 * PAGE_SIZE, there could be a single instance of waste. If we can manage the ratio, this could be optimized? > > More importantly, dealing with internal fragmentation in a size-class, > let's say, of 14528 will be a little painful, as we'll need to move > around 14K objects. > > As, for the speed part, well, it's a little unusual to see that you > are focusing on zstd. zstd is slower than any from the lzX family, > sort of a fact, zstsd sports better compression ratio, but is slower. > Do you use zstd in your smartphones? If speed is your main metrics, Yes, essentially, zstd is too slow. However, with mTHP and this patch set, the swap-out/swap-in bandwidth has significantly improved. As a result, we are now using zstd directly on phones with two zRAM devices: zRAM0: swap-out/swap-in small folios using lz4; zRAM1: swap-out/swap-in large folios using zstd Without large folios, the latency of zstd for small folios is unacceptable, which is why zRAM0 uses lz4. On the other hand, zRAM1 strikes a balance by combining the acceptable speed of large folios with the memory savings provided by zstd. > another option might be to just use a faster algorithm and then utilize > post-processing (re-compression with zstd or writeback) for memory > savings? The concern lies in power consumption, as re-compression would require decompressing LZ4 and recompressing it into Zstd. Mobile phones are particularly sensitive to both power consumption and standby time. On the other hand, I don’t see any conflict between recompression and the large block compression proposed by this patchset. Even during recompression, the advantages of large block compression can be utilized to enhance speed. Writeback is another approach we are exploring. The main concern is that it might require swapping in data from backend block devices. We need to ensure that only truly cold data is stored there; otherwise, it could significantly impact app launch times when an app transitions from the background to the foreground. > > Do you happen to have some data (pool memory usage ratio, etc.) for > lzo or lzo-rle, or lz4? TBH, I don't, because the current use case involves using zstd for large folios, which is our main focus. We are not using lzo or lz4 for large folios, but I can definitely collect some data on that. Thanks Barry
