On Mon, 6 Feb 2023 at 14:31, Amir Goldstein <amir73il@xxxxxxxxx> wrote: > > > > > My little request again, could you help benchmark on your real workload > > > > rather than "ls -lR" stuff? If your hard KPI is really what as you > > > > said, why not just benchmark the real workload now and write a detailed > > > > analysis to everyone to explain it's a _must_ that we should upstream > > > > a new stacked fs for this? > > > > > > > > > > I agree that benchmarking the actual KPI (boot time) will have > > > a much stronger impact and help to build a much stronger case > > > for composefs if you can prove that the boot time difference really matters. > > > > > > In order to test boot time on fair grounds, I prepared for you a POC > > > branch with overlayfs lazy lookup: > > > https://github.com/amir73il/linux/commits/ovl-lazy-lowerdata > > > > Sorry about being late to the party... > > > > Can you give a little detail about what exactly this does? > > > > Consider a container image distribution system, with base images > and derived images and instruction on how to compose these images > using overlayfs or other methods. > > Consider a derived image L3 that depends on images L2, L1. > > With the composefs methodology, the image distribution server splits > each image is split into metadata only (metacopy) images M3, M2, M1 > and their underlying data images containing content addressable blobs > D3, D2, D1. > > The image distribution server goes on to merge the metadata layers > on the server, so U3 = M3 + M2 + M1. > > In order to start image L3, the container client will unpack the data layers > D3, D2, D1 to local fs normally, but the server merged U3 metadata image > will be distributed as a read-only fsverity signed image that can be mounted > by mount -t composefs U3.img (much like mount -t erofs -o loop U3.img). > > The composefs image format contains "redirect" instruction to the data blob > path and an fsverity signature that can be used to verify the redirected data > content. > > When composefs authors proposed to merge composefs, Gao and me > pointed out that the same functionality can be achieved with minimal changes > using erofs+overlayfs. > > Composefs authors have presented ls -lR time and memory usage benchmarks > that demonstrate how composefs performs better that erofs+overlayfs in > this workload and explained that the lookup of the data blobs is what takes > the extra time and memory in the erofs+overlayfs ls -lR test. > > The lazyfollow POC optimizes-out the lowerdata lookup for the ls -lR > benchmark, so that composefs could be compared to erofs+overlayfs. Got it, thanks. > > To answer Alexander's question: > > > Cool. I'll play around with this. Does this need to be an opt-in > > option in the final version? It feels like this could be useful to > > improve performance in general for overlayfs, for example when > > metacopy is used in container layers. > > I think lazyfollow could be enabled by default after we hashed out > all the bugs and corner cases and most importantly remove the > POC limitation of lower-only overlay. > > The feedback that composefs authors are asking from you > is whether you will agree to consider adding the "lazyfollow > lower data" optimization and "fsverity signature for metacopy" > feature to overlayfs? > > If you do agree, the I think they should invest their resources > in making those improvements to overlayfs and perhaps > other improvements to erofs, rather than proposing a new > specialized filesystem. Lazy follow seems to make sense. Why does it need to be optional? Does it have any advantage to *not* do lazy follow? Not sure I follow the fsverity requirement. For overlay+erofs case itsn't it enough to verify the erofs image? Thanks, Miklos > > Thanks, > Amir.
