On Fri, Jan 26, 2024 at 10:15:53AM -0500, Steven Rostedt wrote: > On Fri, 26 Jan 2024 06:16:38 -0800 > Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> wrote: > > > On Thu, Jan 25, 2024 at 09:40:07PM -0500, Steven Rostedt wrote: > > > On Thu, 25 Jan 2024 17:59:40 -0800 > > > Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> wrote: > > > > > > > > I tried to use kernfs when doing a lot of this and I had issues. I > > > > > don't remember what those were, but I can revisit it. > > > > > > > > You might, as kernfs makes it so that the filesystem structures are > > > > created on demand, when accessed, and then removed when memory pressure > > > > happens. That's what sysfs and configfs and cgroups use quite > > > > successfully. > > > > > > kernfs doesn't look trivial and I can't find any documentation on how > > > to use it. > > > > You have the code :) > > Really Greg? > > I can write what I want to do twice as fast than trying to figure out why > someone else did what they did in their code, unless there's good > documentation on the subject. Sorry, that was snarky, but yes, there is no documentation for kernfs, as it evolved over time with the users of it being converted to use it as it went. I'd suggest looking at how cgroups uses it as odds are that's the simplest way. > > > Should there be work to move debugfs over to kernfs? > > > > Why? Are you seeing real actual memory use with debugfs that is causing > > problems? That is why we made kernfs, because people were seeing this > > in sysfs. > > The reason I brought it up was from Linus's comment about dentries and > inodes should not exist if the file system isn't mounted. That's not the > case with debugfs. My question is, do we want debugfs to not use dentries > as its main handle? In the long run, yes, I want the "handle" that all callers to debugfs to NOT use a dentry, and have been slowly migrating away from allowing debugfs to actually return a dentry to the caller. When that is eventually finished, it will be an opaque "handle" that all users of debugfs has and THEN we can convert debugfs to do whatever it wants to. Again, long-term plans, slowly getting there, if only I had an intern or 10 to help out with it :) But, this is only being driven by my "this feels like the wrong api to use" ideas, and seeing how debugfs returning a dentry has been abused by many subsystems in places, not by any real-world measurements of "debugfs is using up too much memory!" like we have had for sysfs ever since the beginning. If someone comes up with a real workload that shows debugfs is just too slow or taking up too much memory for their systems for functionality that they rely on (that's the kicker), then the movement for debugfs to kernfs would happen much faster as someone would actually have the need to do so. > > Don't change stuff unless you need to, right? > > > > > I could look at it too, but as tracefs, and more specifically eventfs, > > > has 10s of thousands of files, I'm very concerned about meta data size. > > > > Do you have real numbers? If not, then don't worry about it :) > > I wouldn't be doing any of this without real numbers. They are in the > change log of eventfs. > > See commits: > > 27152bceea1df27ffebb12ac9cd9adbf2c4c3f35 > 5790b1fb3d672d9a1fe3881a7181dfdbe741568f Sorry, I mean for debugfs. > > Again, look at kernfs if you care about the memory usage of your virtual > > filesystem, that's what it is there for, you shouldn't have to reinvent > > the wheel. > > Already did because it was much easier than trying to use kernfs without > documentation. I did try at first, and realized it was easier to do it > myself. tracefs was based on top of debugfs, and I saw no easy path to go > from that to kernfs. Perhaps do some digging into history and see how we moved sysfs to kernfs, as originally sysfs looked exactly like debugfs. That might give you some ideas of what to do here. > > And the best part is, when people find issues with scaling or other > > stuff with kernfs, your filesystem will then benifit (lots of tweaks > > have gone into kernfs for this over the past few kernel releases.) > > Code is already done. It would be a huge effort to try to convert it over > to kernfs without even knowing if it will regress the memory issues, which > I believe it would (as the second commit saved 2 megs by getting rid of > meta data per file, which kernfs would bring back). > > So, unless there's proof that kernfs would not add that memory footprint > back, I have no time to waste on it. That's fine, I was just responding to your "do we need a in-kernel way to do this type of thing" and I pointed out that kernfs already does just that. Rolling your own is great, like you did, I'm not saying you have to move to kernfs at all if you don't want to as I'm not the one having to maintain eventfs :) thanks, greg k-h
