On Fri, 30 Jul 2021, Qu Wenruo wrote: > > > > You mean like "du -x"?? Yes. You would lose the misleading illusion > > that there are multiple filesystems. That is one user-expectation that > > would need to be addressed before people opt-in > > OK, forgot it's an opt-in feature, then it's less an impact. The hope would have to be that everyone would eventually opt-in once all issues were understood. > > Really not familiar with NFS/VFS, thus some ideas from me may sounds > super crazy. > > Is it possible that, for nfsd to detect such "subvolume" concept by its > own, like checking st_dev and the fsid returned from statfs(). > > Then if nfsd find some boundary which has different st_dev, but the same > fsid as its parent, then it knows it's a "subvolume"-like concept. > > Then do some local inode number mapping inside nfsd? > Like use the highest 20 bits for different subvolumes, while the > remaining 44 bits for real inode numbers. > > Of-course, this is still a workaround... Yes, it would certainly be possible to add some hacks to nfsd to fix the immediate problem, and we could probably even created some well-defined interfaces into btrfs to extract the required information so that it wasn't too hackish. Maybe that is what we will have to do. But I'd rather not hack NFSD while there is any chance that a more complete solution will be found. I'm not quite ready to give up on the idea of squeezing all btrfs inodes into a 64bit number space. 24bits of subvol and 40 bits of inode? Make the split a mkfs or mount option? Maybe hand out inode numbers to subvols in 2^32 chunks so each subvol (which has ever been accessed) has a mapping from the top 32 bits of the objectid to the top 32 bits of the inode number. We don't need something that is theoretically perfect (that's not possible anyway as we don't have 64bits of device numbers). We just need something that is practical and scales adequately. If you have petabytes of storage, it is reasonable to spend a gigabyte of memory on a lookup table(?). If we can make inode numbers unique, we can possibly leave the st_dev changing at subvols so that "du -x" works as currently expected. One thought I had was to use a strong hash to combine the subvol object id and the inode object id into a 64bit number. What is the chance of a collision in practice :-) Thanks, NeilBrown
