> -----Original Message----- > From: Daire Byrne [mailto:daire@xxxxxxxx] > Sent: Wednesday, September 22, 2021 2:48 AM > To: Bruce Fields <bfields@xxxxxxxxxxxx> > Cc: Chuck Lever III <chuck.lever@xxxxxxxxxx>; Linux NFS Mailing List <linux- > nfs@xxxxxxxxxxxxxxx> > Subject: Re: [PATCH] nfs: reexport documentation > > On Tue, 21 Sept 2021 at 17:00, Bruce Fields <bfields@xxxxxxxxxxxx> wrote: > > > > > Any recommended workarounds? Or does this simply mean that > > > administrators need to notify client users to unmount (or at least > > > stop their workloads) before rebooting the proxy? > > > > I think so. > > > > If you don't use any file locking or delegations I suppose you're also > > OK. Delegations might be useful, though. > > > > I'd expect reexport to be useful mainly for data that changes very > > rarely, if that helps. > > > > --b. > > Firstly, it's great to see this documentation and the well maintained wiki page > for something we use in production (a lot) - thanks Bruce! > > I can only draw on our experience to say: > * if the nfs re-export server doesn't crash, we rarely have cause to reboot it. > * we re-export read-only software repositories to WAN/cloud instances (an > ideal use case). > * we also re-export read/write production storage but every client process is > writing unique files - there are no writes to the same > file(s) from any clients of the re-export server. > > We don't use or need crossmnt functionality, but I know from chatting to others > within our industry that the fsid/crossmnt limitation causes them the most grief > and confusion. I think in the case of Netapps, there are similar problems with > trying to re-export a namespace made up of different volumes? > > As noted on the wiki, the only way around that is probably to have a "proxy" > mode (similar to what ganesha does?). I'm glad to see this documentation also. It helps to have a common understanding of the challenges of re-export. Ganesha's proxy mode would be different from what Bruce is proposing due to how Ganesha constructs handles. It adds an export ID to the export configuration which leads to the specific back end (FSAL) that is exporting that handle. It may or may not also encode an fsid into the handle (proxy does so only to the extent the re-exported server put an fsid into the handle, Ganesha's proxy doesn't distinguish between the filesystems on the re-exported server). Ganesha can proxy and serve local file systems because it's proxy isn't a re-export of an NFS mount, but a separate back end module (two actually, one for NFSv3 and one for NFSv4) that talk directly to the proxied server while local file systems resolve handles pretty much the same way knfsd does (use the fsid to find the right filesystem and then use open_by_handle to find the inode within that filesystem - open_by_handle is a user space interface to the same exportfs interface that knfsd uses, with all the same limitations). What Bruce is proposing is a bit more like a proxy mode I have considered for Ganesha that would allow both a proxied Ganesha server and the Ganesha proxy to use the same handles. Basically the export IDs would be shared between the two servers, though even that could be constructed in a way to allow multiple proxied servers (they just would be required to have non-overlapping export IDs) as well as still export local file systems. Frank
