On Thu, Jul 29, 2021 at 11:43:21AM +1000, NeilBrown wrote: > On Thu, 29 Jul 2021, Zygo Blaxell wrote: > > > > I'm looking at a few machines here, and if all the subvols are visible to > > 'df', its output would be somewhere around 3-5 MB. That's too much--we'd > > have to hack up df to not show the same btrfs twice...as well as every > > monitoring tool that reports free space...which sounds similar to the > > problems we're trying to avoid. > > Thanks for providing hard data!! How many of these subvols are actively > used (have a file open) a the same time? Most? About half? Just a few?? Between 1% and 10% of the subvols have open fds at any time (not counting bees, which holds an open fd on every subvol most of the time). The ratio is higher (more active) when the machine has less storage or more CPU/RAM: we keep idle subvols around longer if we have lots of free space, or we make more subvols active at the same time if we have lots of RAM and CPU. I don't recall if stat() triggers automount, but most of the subvols are located in a handful of directories. Could a single 'ls -l' command activate all of their automounts? If so, then we'd be hitting those at least once every 15 minutes--these directories are browseable, and an entry point for users. Certainly anything that looks inside those directories (like certain file-browsing user agents that look for icons one level down) will trigger automount as they search children of the subvol root. Some of this might be fixable, like I could probably make bees be more parsimonious with subvol root fds, and I could probably rework reporting scripts to avoid touching anything inside subdirectories, and I could probably rework our subvolume directory layout to avoid accidentally triggering thousands of automounts. But I'd rather not. I'd rather have working NFS and a 15-20 line df output with no new privacy or scalability concerns. > Thanks, > NeilBrown
