Re: file metadata via fs API

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

On 12/08/2020 20:50, Linus Torvalds wrote:
On Wed, Aug 12, 2020 at 12:34 PM Steven Whitehouse <swhiteho@xxxxxxxxxx> wrote:
The point of this is to give us the ability to monitor mounts from
userspace.
We haven't had that before, I don't see why it's suddenly such a big deal.

The notification side I understand. Polling /proc files is not the answer.

But the whole "let's design this crazy subsystem for it" seems way
overkill. I don't see anybody caring that deeply.

It really smells like "do it because we can, not because we must".

Who the hell cares about monitoring mounts at a kHz frequencies? If
this is for MIS use, you want a nice GUI and not wasting CPU time
polling.

I'm starting to ignore the pull requests from David Howells, because
by now they have had the same pattern for a couple of years now:
esoteric new interfaces that seem overdesigned for corner-cases that
I'm not seeing people clamoring for.

I need (a) proof this is actualyl something real users care about and
(b) way more open discussion and implementation from multiple parties.

Because right now it looks like a small in-cabal of a couple of people
who have wild ideas but I'm not seeing the wider use of it.

Convince me otherwise. AGAIN. This is the exact same issue I had with
the notification queues that I really wanted actual use-cases for, and
feedback from actual outside users.

I really think this is engineering for its own sake, rather than
responding to actual user concerns.

                Linus


I've been hesitant to reply to this immediately, because I can see that somehow there is a significant disconnect between what you expect to happen, and what has actually happened in this case. Have pondered this for a few days, I hope that the best way forward might be to explore where the issues are, with the intention of avoiding a repeat in the future. Sometimes email is a difficult medium for these kinds of communication, and face to face is better, but with the lack of conferences/travel at the moment, that option is not open in the near future.

The whole plan here, leading towards the ability to get a "dump plus updates" view of mounts in the kernel has been evolving over time. It has been discussed at LSF over a number of years [1] and in fact the new mount API which was merged recently - I wonder if this is what you are referring to above as:

I'm starting to ignore the pull requests from David Howells, because
by now they have had the same pattern for a couple of years now

was originally proposed by Al, and also worked on by Miklos[2] in 2017 and others. Community discussion resulted in that becoming a prerequisite for the later notifications/fsinfo work. This was one of the main reasons that David picked it up[3] to work on, but not the only reason. That did also appear to be logical, in that cleaning up the way in which arguments were handled during mount would make it much easier to create future generic code to handle them.

That said, the overall aim here is to solve the problem and if there are better solutions available then I'm sure that everyone is very open to those. I agree very much that monitoring at kHz frequencies is not useful, but at the same time, there are cases which can generate large amounts of mount changes in a very short time period. We want to be reasonably efficient, but not to over-optimise, and sometimes that is a fine line. We also don't want to block mounts if the notifications queue fills up, so some kind of resync operation would be required in the queue overflows. The notifications and fsinfo were designed very much as two sides of the same coin, but submitted separately for ease of review more than anything else.

You recently requested some details of real users for the notifications, and (I assumed) by extension fsinfo too. Ian wrote these emails [4][5] in direct response to your request. That is what we thought you were looking for, so if that isn't not quite what you meant, perhaps you could clarify a bit more. Again, apologies if we've misinterpreted what you were asking for.

You also mention "...it looks like a small in-cabal of a couple of people..." and I hope that it doesn't look that way, it is certainly not our intention. There have been a fair number of people involved, and we've done our best to ensure that the development is guided by the potential users, such as autofs, AFS and systemd. If there are others out there with use cases, and particularly so if the use case is a GUI file manager type application who'd like to get involved, then please do. We definitely want to see involvement from end users, since there is no point in spending a large effort creating something that is then never used. As you pointed that out above, this kind of application was very much part of the original motivation, but we had started with the other users since there were clearly defined use cases that could demonstrate significant performance gains in those cases.

So hopefully that helps to give a bit more background about where we are and how we got here. Where we go next will no doubt depend on the outcome of the current discussions, and any guidance you can give around how we should have better approached this would be very helpful at this stage,

Steve.


[1] https://lwn.net/Articles/718803/

[2] https://lwn.net/Articles/718638/

[3] https://lwn.net/Articles/753473/

[4] https://lkml.org/lkml/2020/6/2/1182

[5] https://lore.kernel.org/linux-fsdevel/8eb2e52f1cbdbb8bcf5c5205a53bdc9aaa11a071.camel@xxxxxxxxxx/





[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [Samba]     [Device Mapper]     [CEPH Development]

  Powered by Linux