On Tue, Dec 17, 2024 at 05:28:24PM -0800, Sarthak Kukreti wrote:
> On Sun, Dec 15, 2024 at 10:39 AM Ojaswin Mujoo <ojaswin@xxxxxxxxxxxxx> wrote:
> >
> > On Thu, Dec 12, 2024 at 02:49:58PM -0800, Sarthak Kukreti wrote:
> > > Add a new flag to add support for fixed goal allocations in
> > > ext_falloc_helper. For fixed goal allocations, omit merging extents and
> > > return an error unless the exact extent is found.
> > >
> > > Use case:
> > > On ChromiumOS, we'd like to add the capability of resetting a filesystem
> > > while preserving a set of files in-place. This will be used during
> > > filesystem reset flows where everything apart from select files (which
> > > contain system applications) should be removed: the combined size of the
> > > files can exceed the amount of available space in other
> > > partitions/memory. The reset process will look something like:
> > >
> > > 1. Reset code dumps the FIEMAP of the set of preserved files into a
> > > file.
> > > 2. Mkfs.ext4 is called on the filesystem with -E nodiscard.
> > > 3. Post mkfs, the reset code will utilize ext2fs_fallocate w/
> > > EXT2_FALLOCATE_FIXED_GOAL | EXT2_FALLOCATE_FORCE_INIT on the extent list
> > > created in step 1.
> >
> > Hey Sarthak,
> >
> > On the e2fsprogs side, the change looks straight forward enough and
> > irrespective of the use case having FIXED GOAL for fallocate makes sense
> > to me. While you are at it, I would just request you to fix the comment
> > above ext2fs_new_range():
> >
> > /*
> > * Starting at _goal_, scan around the filesystem to find a run of free blocks
> > * that's at least _len_ blocks long. Possible flags:
> > - * - EXT2_NEWRANGE_EXACT_GOAL: The range of blocks must start at _goal_.
> > + * - EXT2_NEWRANGE_FIXED_GOAL: The range of blocks must start at _goal_.
> > * - EXT2_NEWRANGE_MIN_LENGTH: do not return a allocation shorter than _len_.
> > * - EXT2_NEWRANGE_ZERO_BLOCKS: Zero blocks pblk to pblk+plen before returning.
> >
> Sure, let me add and send that as a quick v2.
>
> > That being said, the usecase seems interesting to me and I have a few
> > questions about it:
> >
> > 1. So if i understand correctly, after mkfs your tool will essentially
> > handcraft the FS by using lib/ext2fs helpers to fallocate the exact
> > physical blocks where your files are supposed to be on disk. I believe
> > you'd also need to recreate inodes/xattrs etc for the files to make sure
> > they are identical after mkfs?
> >
> Correct, the restore tool will ensure that inode, attrs and xattrs are
> regenerated.
>
> > 2. I'm assuming you don't expect the underlying storage medium to change
> > across this reset and hence using the same physical block works?
> >
> > 3. I wonder if there are any other ways of doing this without having to
> > handcraft the FS in this way. It just seems a bit fragile.
> >
> Yes, the underlying block device remains the same. The preservation
> mechanism is primarily intended for system files with attached integrity data;
> more specifically, we use dm-verity to validate the integrity of these
> files before
> use post-reset.
Got it, maybe fs-verity could be of help here as well to verify things
at file level, however I believe that will have it's own challenges :)
Anyways, thanks for the details!
Regards,
ojaswin
>
> Cheers
> Sarthak
>
> > Regards,
> > ojaswin
> >
> > >
> > > Signed-off-by: Sarthak Kukreti <sarthakkukreti@xxxxxxxxxx>
> > > ---
> > > lib/ext2fs/ext2fs.h | 3 ++-
> > > lib/ext2fs/fallocate.c | 21 +++++++++++++++++++--
> > > 2 files changed, 21 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> > > index 6e87829f..313c5981 100644
> > > --- a/lib/ext2fs/ext2fs.h
> > > +++ b/lib/ext2fs/ext2fs.h
> > > @@ -1446,7 +1446,8 @@ extern errcode_t ext2fs_decode_extent(struct ext2fs_extent *to, void *from,
> > > #define EXT2_FALLOCATE_FORCE_INIT (0x2)
> > > #define EXT2_FALLOCATE_FORCE_UNINIT (0x4)
> > > #define EXT2_FALLOCATE_INIT_BEYOND_EOF (0x8)
> > > -#define EXT2_FALLOCATE_ALL_FLAGS (0xF)
> > > +#define EXT2_FALLOCATE_FIXED_GOAL (0x10)
> > > +#define EXT2_FALLOCATE_ALL_FLAGS (0x1F)
> > > errcode_t ext2fs_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
> > > struct ext2_inode *inode, blk64_t goal,
> > > blk64_t start, blk64_t len);
> > > diff --git a/lib/ext2fs/fallocate.c b/lib/ext2fs/fallocate.c
> > > index 5cde7d5c..20aa9c9f 100644
> > > --- a/lib/ext2fs/fallocate.c
> > > +++ b/lib/ext2fs/fallocate.c
> > > @@ -103,7 +103,7 @@ static errcode_t ext_falloc_helper(ext2_filsys fs,
> > > blk64_t alloc_goal)
> > > {
> > > struct ext2fs_extent newex, ex;
> > > - int op;
> > > + int op, new_range_flags = 0;
> > > blk64_t fillable, pblk, plen, x, y;
> > > blk64_t eof_blk = 0, cluster_fill = 0;
> > > errcode_t err;
> > > @@ -132,6 +132,9 @@ static errcode_t ext_falloc_helper(ext2_filsys fs,
> > > max_uninit_len = EXT_UNINIT_MAX_LEN & ~EXT2FS_CLUSTER_MASK(fs);
> > > max_init_len = EXT_INIT_MAX_LEN & ~EXT2FS_CLUSTER_MASK(fs);
> > >
> > > + if (flags & EXT2_FALLOCATE_FIXED_GOAL)
> > > + goto no_implied;
> > > +
> > > /* We must lengthen the left extent to the end of the cluster */
> > > if (left_ext && EXT2FS_CLUSTER_RATIO(fs) > 1) {
> > > /* How many more blocks can be attached to left_ext? */
> > > @@ -605,12 +608,15 @@ no_implied:
> > > max_extent_len = max_uninit_len;
> > > newex.e_flags = EXT2_EXTENT_FLAGS_UNINIT;
> > > }
> > > +
> > > + if (flags & EXT2_FALLOCATE_FIXED_GOAL)
> > > + new_range_flags = EXT2_NEWRANGE_FIXED_GOAL | EXT2_NEWRANGE_MIN_LENGTH;
> > > pblk = alloc_goal;
> > > y = range_len;
> > > for (x = 0; x < y;) {
> > > cluster_fill = newex.e_lblk & EXT2FS_CLUSTER_MASK(fs);
> > > fillable = min(range_len + cluster_fill, max_extent_len);
> > > - err = ext2fs_new_range(fs, 0, pblk & ~EXT2FS_CLUSTER_MASK(fs),
> > > + err = ext2fs_new_range(fs, new_range_flags, pblk & ~EXT2FS_CLUSTER_MASK(fs),
> > > fillable,
> > > NULL, &pblk, &plen);
> > > if (err)
> > > @@ -681,6 +687,16 @@ static errcode_t extent_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
> > > if (err)
> > > return err;
> > >
> > > + /*
> > > + * For fixed goal allocations, let the allocations fail iff we can't
> > > + * find the exact goal extent.
> > > + */
> > > + if (flags & EXT2_FALLOCATE_FIXED_GOAL) {
> > > + err = ext_falloc_helper(fs, flags, ino, inode, handle, NULL,
> > > + NULL, start, len, goal);
> > > + goto errout;
> > > + }
> > > +
> > > /*
> > > * Find the extent closest to the start of the alloc range. We don't
> > > * check the return value because _goto() sets the current node to the
> > > @@ -796,6 +812,7 @@ errout:
> > > * - EXT2_FALLOCATE_FORCE_INIT: Create only initialized extents.
> > > * - EXT2_FALLOCATE_FORCE_UNINIT: Create only uninitialized extents.
> > > * - EXT2_FALLOCATE_INIT_BEYOND_EOF: Create extents beyond EOF.
> > > + * - EXT2_FALLOCATE_FIXED_GOAL: Ensure range starts at goal.
> > > *
> > > * If neither FORCE_INIT nor FORCE_UNINIT are specified, this function will
> > > * try to expand any extents it finds, zeroing blocks as necessary.
> > > --
> > > 2.47.0.rc1.288.g06298d1525-goog
>
>
>
> --
> Sarthak Kukreti | Software Engineer | sarthakkukreti@xxxxxxxxxx | 650-203-5572
