On 2023/3/7 16:21, Alexander Larsson wrote:
On Mon, Mar 6, 2023 at 5:17 PM Gao Xiang <hsiangkao@xxxxxxxxxxxxxxxxx> wrote:
I tested the performance of "ls -lR" on the whole tree of
cs9-developer-rootfs. It seems that the performance of erofs (generated
from mkfs.erofs) is slightly better than that of composefs. While the
performance of erofs generated from mkfs.composefs is slightly worse
that that of composefs.
I suspect that the reason for the lower performance of mkfs.composefs
is the added overlay.fs-verity xattr to all the files. It makes the
image larger, and that means more i/o.
Actually you could move overlay.fs-verity to EROFS shared xattr area (or
even overlay.redirect but it depends) if needed, which could save some
I/Os for your workloads.
shared xattrs can be used in this way as well if you care such minor
difference, actually I think inlined xattrs for your workload are just
meaningful for selinux labels and capabilities.
Really? Could you expand on this, because I would think it will be
sort of the opposite. In my usecase, the erofs fs will be read by
overlayfs, which will probably access overlay.* pretty often. At the
very least it will load overlay.metacopy and overlay.redirect for
every lookup.
Really. In that way, it will behave much similiar to composefs on-disk
arrangement now (in composefs vdata area).
Because in that way, although an extra I/O is needed for verification,
and it can only happen when actually opening the file (so "ls -lR" is
not impacted.) But on-disk inodes are more compact.
All EROFS xattrs will be cached in memory so that accessing
overlay.* pretty often is not greatly impacted due to no real I/Os
(IOWs, only some CPU time is consumed).
I guess it depends on how the verity support in overlayfs would work.
If it delays access to overlay.verity until open time, then it would
make sense to move it to the shared area.
I think it could be just like what composefs does, it's not hard to
add just new dozen lines to overlayfs like:
static int cfs_open_file(struct inode *inode, struct file *file)
{
...
/* If metadata records a digest for the file, ensure it is there
* and correct before using the contents.
*/
if (cino->inode_data.has_digest &&
fsi->verity_check >= CFS_VERITY_CHECK_IF_SPECIFIED) {
...
res = fsverity_get_digest(d_inode(backing_dentry),
verity_digest, &verity_algo);
if (res < 0) {
pr_warn("WARNING: composefs backing file '%pd' has no fs-verity digest\n",
backing_dentry);
return -EIO;
}
if (verity_algo != HASH_ALGO_SHA256 ||
memcmp(cino->inode_data.digest, verity_digest,
SHA256_DIGEST_SIZE) != 0) {
pr_warn("WARNING: composefs backing file '%pd' has the wrong fs-verity digest\n",
backing_dentry);
return -EIO;
}
...
}
...
}
Is this stacked fsverity feature really hard?
Thanks,
Gao Xiang
