On 07/20/2010 07:58 PM, Dan Magenheimer wrote: >> On 07/20/2010 01:27 AM, Dan Magenheimer wrote: >>>> We only keep pages that compress to PAGE_SIZE/2 or less. Compressed >>>> chunks are >>>> stored using xvmalloc memory allocator which is already being used >> by >>>> zram >>>> driver for the same purpose. Zero-filled pages are checked and no >>>> memory is >>>> allocated for them. >>> >>> I'm curious about this policy choice. I can see why one >>> would want to ensure that the average page is compressed >>> to less than PAGE_SIZE/2, and preferably PAGE_SIZE/2 >>> minus the overhead of the data structures necessary to >>> track the page. And I see that this makes no difference >>> when the reclamation algorithm is random (as it is for >>> now). But once there is some better reclamation logic, >>> I'd hope that this compression factor restriction would >>> be lifted and replaced with something much higher. IIRC, >>> compression is much more expensive than decompression >>> so there's no CPU-overhead argument here either, >>> correct? >> >> Its true that we waste CPU cycles for every incompressible page >> encountered but still we can't keep such pages in RAM since this >> is what host wanted to reclaim and we can't help since compression >> failed. Compressed caching makes sense only when we keep highly >> compressible pages in RAM, regardless of reclaim scheme. >> >> Keeping (nearly) incompressible pages in RAM probably makes sense >> for Xen's case where cleancache provider runs *inside* a VM, sending >> pages to host. So, if VM is limited to say 512M and host has 64G RAM, >> caching guest pages, with or without compression, will help. > > I agree that the use model is a bit different, but PAGE_SIZE/2 > still seems like an unnecessarily strict threshold. For > example, saving 3000 clean pages in 2000*PAGE_SIZE of RAM > still seems like a considerable space savings. And as > long as the _average_ is less than some threshold, saving > a few slightly-less-than-ideally-compressible pages doesn't > seem like it would be a problem. For example, IMHO, saving two > pages when one compresses to 2047 bytes and the other compresses > to 2049 bytes seems just as reasonable as saving two pages that > both compress to 2048 bytes. > > Maybe the best solution is to make the threshold a sysfs > settable? Or maybe BOTH the single-page threshold and > the average threshold as two different sysfs settables? > E.g. throw away a put page if either it compresses poorly > or adding it to the pool would push the average over. > Considering overall compression average instead of bothering about individual page compressibility seems like a good point. Still, I think storing completely incompressible pages isn't desirable. So, I agree with the idea of separate sysfs tunables for average and single-page compression thresholds with defaults conservatively set to 50% and PAGE_SIZE/2 respectively. I will include these in "v2" patches. Thanks, Nitin -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxxx For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>