> On Tue, Nov 24, 2020 at 02:15:57PM -0800, Frank Filz wrote: > > How much conversation about re-export has been had at the wider NFS > > community level? I have an interest because Ganesha supports > > re-export via the PROXY_V3 and PROXY_V4 FSALs. We currently don't have > > a data cache though there has been discussion of such, we do have attribute > and dirent caches. > > > > Looking over the wiki page, I have considered being able to specify a > > re-export of a Ganesha export without encapsulating handles. Ganesha > > encapsulates the export_fs handle in a way that could be coordinated > > between the original server and the re-export so they would both > > effectively have the same encapsulation layer. > > In the case the re-export server only servers a single export, I guess you could do > away with the encapsulation. (The only risk I see is that a client of the re-export > server could also access any export of the original server if it could guess > filehandles, which might surprise > admins.) Maybe that'd be useful. Ganesha handles have a minor downside that is a help here if Ganesha was re-exporting another Ganesha server. There is a 16 bit export_id that comes from the export configuration and is part of the handle. We could easily set it up so that if the sysadmin configured it as such, each re-exported Ganesha export would have the same export_id, and then a client handle for export_id 1 would be mirrored to the original server as export_id 1 and the two servers can have the same export permissions and everything. There is some additional stuff we could easily implement in Ganesha to restrict handle manipulation to sneak around export permissions. > Another advantage of not encapsulating filehandles is that clients could more > easily migrate between servers. Yea, with the idea I've been mulling for Ganesha, migration between original server and re-export server would be simple with the same handles. Of course state migration is a whole different ball of wax, but a clustered setup could work just as well as Ganesha's clustered filesystems. On the other hand, re-export with state has a pitfall. If the re-export server crashes, the state is lost on the original server unless we make a protocol change to allow state re-export such that a re-export server crashing doesn't cause state loss. For this reason, I haven't rushed to implement lock state re-export in Ganesha, rather allowing the re-export server to just manage lock state locally. > Cooperating servers could have an agreement on filehandles. And I guess we > could standardize that somehow. Are we ready for that? I'm not sure what > other re-exporting problems there are that I haven't thought of. I'm not sure how far we want to go there, but potentially specific server implementations could choose to be interoperable in a way that allows the handle encapsulation to either be smaller or no extra overhead. For example, if we implemented what I've thought about for Ganesha-Ganesha re-export, Ganesha COULD also be "taught" which portion of the knfsd handle is the filesystem/export identifier, and maintain a database of Ganesha export/filesystem <-> knfsd export/filesystem and have Ganesha re-encapsulate the exportfs/name_to_handle_at portion of the handle. Of course in this case, trivial migration isn't possible since Ganesha will have a different encapsulation than knfsd. Incidentally, I also purposefully made Ganesha's encapsulation different so it never collides with either version of knfsd handles (now if over the course of the past 10 years another handle version has come along...). Frank > --b. > > > I'd love to see some re-export best practices shared among server > > implementations, and also what we can do to improve things when two > > server implementations are interoperating via re-export. -- Linux-cachefs mailing list Linux-cachefs@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cachefs
