Re: UBIFS quota support

[Jan Kara <jack@xxxxxxx>]

Hello,

On Fri 25-01-19 10:21:56, Sascha Hauer wrote:
> On Wed, Jan 23, 2019 at 04:47:43PM +0100, Jan Kara wrote:
> > sorry for not getting to you earlier, this email got burried in my inbox...
> > 
> > On Wed 23-01-19 00:07:12, Richard Weinberger wrote:
> > > On Thu, Jan 10, 2019 at 12:45 PM Sascha Hauer <s.hauer@xxxxxxxxxxxxxx> wrote:
> > > > I'm currently working on resurrecting the UBIFS quota patches posted back in
> > > > 2015 by Dongsheng Yang, last posted here:
> > > >
> > > > http://lists.infradead.org/pipermail/linux-mtd/2015-September/061812.html
> > > >
> > > > First of all I think work stopped there, there is no newer UBIFS quota
> > > > support I am missing, right?
> > > >
> > > > One problem with this series was that the quotactl systemcall expects a
> > > > path to a block device. UBIFS doesn't work on a block device but on a
> > > > character device instead.
> > > > The solution in this series was to pass the path to the cdev in
> > > > quotactl.  A struct cdev * member was added to struct super_block which
> > > > was used to identify the superblock for a given cdev. This approach was
> > > > rejected by Christoph ("I don't think the cdev has any business in core
> > > > VFS code.").  Apart from that UBIFS can not only be mounted with a path
> > > > to the character device (mount -t ubifs /dev/ubix_y /mnt) but also in
> > > > the form ubix:volname (mount -t ubifs ubix:volname /mnt) in which case
> > > > userspace doesn't have any valid path it could pass in quotactl.
> > > >
> > > > An idea out of this would be to allow to pass the mountpoint instead of
> > > > the path to the block device in quotactl which would work with nfs or
> > > > even tmpfs aswell. Would that be acceptable? Any other ideas?
> > 
> > So after some thought, yes, I think that passing mount point as a specifier
> > identifying a block device will be OK.
> > 
> > > *kind ping*
> > > 
> > > Jan, another thing Sascha and I are not sure about, what are the
> > > consistency constraints of the quota file?
> > > If I read the code correctly, quota just writes to the quota file and
> > > assumes that the file system makes sure about consistency. Either by fsck
> > > fixing the quota file or having a data journal for the quota file.
> > 
> > Essentially yes but it depends on how exactly you decide to implement quota
> > files. First let me explain to you some details about how quota subsystem
> > works.
> > 
> > When quota structure (struct dquot) for some user gets
> > first attached to some inode we call ->acquire_dquot callback from
> > dquot_operations. This is responsible for allocating necessary disk space
> > for the structure (if not already allocated) and otherwise making sure that
> > the quota information can be easily stored later. Also it should fill in
> > current quota information if the structure for given ID already exists.
> 
> UBIFS has no idea of transactions and works a little different from
> ext4. In UBIFS all filesystem data and metadata are stored in the leaf
> nodes of a b+tree. These leaf nodes are constantly written to flash and
> would be enough to reconstruct the FS. From time to time UBIFS does a
> commit and writes the index nodes to flash. During recovery the tree can
> be read from the index nodes from the last commit. The remaining leaf
> nodes that were written after the last commit are then scanned and added
> to the tree during replay.
> 
> Quota seems to work in the way that it has callbacks into the FS to read
> and update dqblks. This is not very suitable for UBIFS. Instead it would

Yes, generally it works by requesting loading / storing of quota
information for a particular user from the filesystem.

> be nicer to read the full quota data from flash and hand it over to
> quota. When UBIFS does a commit it would then request a consistent view
> of the quota data and write it back to flash. During replay of an
> uncleanly mounted FS UBIFS could read the quota data from the last
> commit and update it with the remaining leaf nodes that need to be
> replayed anyway.

Well, I don't think writing all quota data for each commit is a good
design. That will write out lot of unnecessary stuff that didn't change
since last time. It is like if you rewrite the whole file to update
one block in it... Similarly loading all quota data doesn't look great for
performance. There can be thousands or tens of thousands different users of
the filesystem (yes, I've seen such deployments) and you don't want to read
and keep in memory information you don't currently need. But I guess UBIFS
is targetted at smaller deployments?

Anyway, what is easily doable is that you would just ignore requests for
update from quota subsystem (just let quota subsystem mark dquot as dirty)
and then on commit call dquot_writeback_dquots() that will call
->write_dquot callback for each dquot that is dirty. That way you'd get
full dump of dquots that changed since last commit. You'd need to somehow
make sure this gets merged with information from previous commit. Then on
crash recovery you'd load quota information from commit and update it with
the changes that happened since that last commit.

> In the latest UBIFS quota patchset storing the quota data is not safe
> for unclean remounts, the data will become inconsistent with the actual
> filesystem state.
> 
> Thinking about it storing the quota data is completely optional. In UBIFS
> it's easy to retrieve the quota data by scanning the index tree during mount
> time. The quota limits could be set during runtime by some init script.
> For our usecase I think there's no need to store the limits in the FS
> itself. With this I could turn the UBIFS quota patches into something
> useful and consistently working with relatively little effort. Storing
> the quota data on flash needs some discussion how to integrate properly
> into UBIFS, but becomes an optimization left for a future exercise.

Honestly loading quota limits by init scripts looks like a hack to me. Note
that quota limits rarely change so you can easily store them separately
from usage information and load them on mount. Since setting of limits does
not have to be crash-safe (well, it needs to keep the quota information in
a state that is recoverable by mount but it doesn't need to coordinate with
any other filesystem operations), I don't think implementing that would be
hard...

								Honza
-- 
Jan Kara <jack@xxxxxxxx>
SUSE Labs, CR

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/