On Sun, Dec 15, 2024 at 10:39 AM Ojaswin Mujoo <ojaswin@xxxxxxxxxxxxx> wrote:
>
> On Thu, Dec 12, 2024 at 02:49:58PM -0800, Sarthak Kukreti wrote:
> > Add a new flag to add support for fixed goal allocations in
> > ext_falloc_helper. For fixed goal allocations, omit merging extents and
> > return an error unless the exact extent is found.
> >
> > Use case:
> > On ChromiumOS, we'd like to add the capability of resetting a filesystem
> > while preserving a set of files in-place. This will be used during
> > filesystem reset flows where everything apart from select files (which
> > contain system applications) should be removed: the combined size of the
> > files can exceed the amount of available space in other
> > partitions/memory. The reset process will look something like:
> >
> > 1. Reset code dumps the FIEMAP of the set of preserved files into a
> > file.
> > 2. Mkfs.ext4 is called on the filesystem with -E nodiscard.
> > 3. Post mkfs, the reset code will utilize ext2fs_fallocate w/
> > EXT2_FALLOCATE_FIXED_GOAL | EXT2_FALLOCATE_FORCE_INIT on the extent list
> > created in step 1.
>
> Hey Sarthak,
>
> On the e2fsprogs side, the change looks straight forward enough and
> irrespective of the use case having FIXED GOAL for fallocate makes sense
> to me. While you are at it, I would just request you to fix the comment
> above ext2fs_new_range():
>
> /*
> * Starting at _goal_, scan around the filesystem to find a run of free blocks
> * that's at least _len_ blocks long. Possible flags:
> - * - EXT2_NEWRANGE_EXACT_GOAL: The range of blocks must start at _goal_.
> + * - EXT2_NEWRANGE_FIXED_GOAL: The range of blocks must start at _goal_.
> * - EXT2_NEWRANGE_MIN_LENGTH: do not return a allocation shorter than _len_.
> * - EXT2_NEWRANGE_ZERO_BLOCKS: Zero blocks pblk to pblk+plen before returning.
>
Sure, let me add and send that as a quick v2.
> That being said, the usecase seems interesting to me and I have a few
> questions about it:
>
> 1. So if i understand correctly, after mkfs your tool will essentially
> handcraft the FS by using lib/ext2fs helpers to fallocate the exact
> physical blocks where your files are supposed to be on disk. I believe
> you'd also need to recreate inodes/xattrs etc for the files to make sure
> they are identical after mkfs?
>
Correct, the restore tool will ensure that inode, attrs and xattrs are
regenerated.
> 2. I'm assuming you don't expect the underlying storage medium to change
> across this reset and hence using the same physical block works?
>
> 3. I wonder if there are any other ways of doing this without having to
> handcraft the FS in this way. It just seems a bit fragile.
>
Yes, the underlying block device remains the same. The preservation
mechanism is primarily intended for system files with attached integrity data;
more specifically, we use dm-verity to validate the integrity of these
files before
use post-reset.
Cheers
Sarthak
> Regards,
> ojaswin
>
> >
> > Signed-off-by: Sarthak Kukreti <sarthakkukreti@xxxxxxxxxx>
> > ---
> > lib/ext2fs/ext2fs.h | 3 ++-
> > lib/ext2fs/fallocate.c | 21 +++++++++++++++++++--
> > 2 files changed, 21 insertions(+), 3 deletions(-)
> >
> > diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> > index 6e87829f..313c5981 100644
> > --- a/lib/ext2fs/ext2fs.h
> > +++ b/lib/ext2fs/ext2fs.h
> > @@ -1446,7 +1446,8 @@ extern errcode_t ext2fs_decode_extent(struct ext2fs_extent *to, void *from,
> > #define EXT2_FALLOCATE_FORCE_INIT (0x2)
> > #define EXT2_FALLOCATE_FORCE_UNINIT (0x4)
> > #define EXT2_FALLOCATE_INIT_BEYOND_EOF (0x8)
> > -#define EXT2_FALLOCATE_ALL_FLAGS (0xF)
> > +#define EXT2_FALLOCATE_FIXED_GOAL (0x10)
> > +#define EXT2_FALLOCATE_ALL_FLAGS (0x1F)
> > errcode_t ext2fs_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
> > struct ext2_inode *inode, blk64_t goal,
> > blk64_t start, blk64_t len);
> > diff --git a/lib/ext2fs/fallocate.c b/lib/ext2fs/fallocate.c
> > index 5cde7d5c..20aa9c9f 100644
> > --- a/lib/ext2fs/fallocate.c
> > +++ b/lib/ext2fs/fallocate.c
> > @@ -103,7 +103,7 @@ static errcode_t ext_falloc_helper(ext2_filsys fs,
> > blk64_t alloc_goal)
> > {
> > struct ext2fs_extent newex, ex;
> > - int op;
> > + int op, new_range_flags = 0;
> > blk64_t fillable, pblk, plen, x, y;
> > blk64_t eof_blk = 0, cluster_fill = 0;
> > errcode_t err;
> > @@ -132,6 +132,9 @@ static errcode_t ext_falloc_helper(ext2_filsys fs,
> > max_uninit_len = EXT_UNINIT_MAX_LEN & ~EXT2FS_CLUSTER_MASK(fs);
> > max_init_len = EXT_INIT_MAX_LEN & ~EXT2FS_CLUSTER_MASK(fs);
> >
> > + if (flags & EXT2_FALLOCATE_FIXED_GOAL)
> > + goto no_implied;
> > +
> > /* We must lengthen the left extent to the end of the cluster */
> > if (left_ext && EXT2FS_CLUSTER_RATIO(fs) > 1) {
> > /* How many more blocks can be attached to left_ext? */
> > @@ -605,12 +608,15 @@ no_implied:
> > max_extent_len = max_uninit_len;
> > newex.e_flags = EXT2_EXTENT_FLAGS_UNINIT;
> > }
> > +
> > + if (flags & EXT2_FALLOCATE_FIXED_GOAL)
> > + new_range_flags = EXT2_NEWRANGE_FIXED_GOAL | EXT2_NEWRANGE_MIN_LENGTH;
> > pblk = alloc_goal;
> > y = range_len;
> > for (x = 0; x < y;) {
> > cluster_fill = newex.e_lblk & EXT2FS_CLUSTER_MASK(fs);
> > fillable = min(range_len + cluster_fill, max_extent_len);
> > - err = ext2fs_new_range(fs, 0, pblk & ~EXT2FS_CLUSTER_MASK(fs),
> > + err = ext2fs_new_range(fs, new_range_flags, pblk & ~EXT2FS_CLUSTER_MASK(fs),
> > fillable,
> > NULL, &pblk, &plen);
> > if (err)
> > @@ -681,6 +687,16 @@ static errcode_t extent_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
> > if (err)
> > return err;
> >
> > + /*
> > + * For fixed goal allocations, let the allocations fail iff we can't
> > + * find the exact goal extent.
> > + */
> > + if (flags & EXT2_FALLOCATE_FIXED_GOAL) {
> > + err = ext_falloc_helper(fs, flags, ino, inode, handle, NULL,
> > + NULL, start, len, goal);
> > + goto errout;
> > + }
> > +
> > /*
> > * Find the extent closest to the start of the alloc range. We don't
> > * check the return value because _goto() sets the current node to the
> > @@ -796,6 +812,7 @@ errout:
> > * - EXT2_FALLOCATE_FORCE_INIT: Create only initialized extents.
> > * - EXT2_FALLOCATE_FORCE_UNINIT: Create only uninitialized extents.
> > * - EXT2_FALLOCATE_INIT_BEYOND_EOF: Create extents beyond EOF.
> > + * - EXT2_FALLOCATE_FIXED_GOAL: Ensure range starts at goal.
> > *
> > * If neither FORCE_INIT nor FORCE_UNINIT are specified, this function will
> > * try to expand any extents it finds, zeroing blocks as necessary.
> > --
> > 2.47.0.rc1.288.g06298d1525-goog
--
Sarthak Kukreti | Software Engineer | sarthakkukreti@xxxxxxxxxx | 650-203-5572
