On Fri, 2017-08-18 at 13:15 -0700, Bennett Amodio wrote:
> On Fri, Aug 18, 2017 at 7:57 AM, Trond Myklebust
> <trondmy@xxxxxxxxxxxxxxx> wrote:
> > On Tue, 2017-08-15 at 17:46 -0700, Bennett Amodio wrote:
> > > After seeing Trond’s patches for NFS multipathing on NFSv4.1, we
> > > decided to try using the same concept for NFSv3/4. The primary
> > > issue
> > > we identified was XID collision in the duplicate request cache
> > > (replay
> > > cache) for NFSv3/4. In NFSv3/4, entries are hashed based on XID
> > > instead of the slot ID and sequence ID that NFSv4.1 uses. Since
> > > the
> > > XIDs are generated by the RPC transports, and Trond’s patches
> > > create
> > > multiple transports for multipathing, different transports can
> > > end up
> > > using an overlapping set of XIDs.
> >
> > Why is that a problem? You should end up with connections that show
> > different combinations of source IP+port and/or destination
> > IP+port. It
> > should be trivial to distinguish between XIDs.
>
> Although the Linux NFS server hashes cache entries based on source IP
> and source port as well as XID, this is not a requirement of the
> NFSv3/v4 specification, so NFS server implementations may exist which
> hash only based on source IP and XID. In practice, is this uncommon
> enough that it's not worth addressing?
There is nothing in RFC1813 that gives any direction on how to set up a
duplicate replay cache (DRC). However established practice dictates
that the server should be prepared for duplicate XIDs that originate
from the same IP address.
In particular, if the linux client connects more than once to your
server (e.g. through 2 different IP addresses) it will assume the XIDs
are per connection. Ditto if using UDP.
> > Quite frankly, I do not want to start carving up the XID space,
> > since a
> > 32-bit number is really not that big in these days of 100GigE
> > networks.
>
> This is a good point, and we also think that carving up the XID space
> is not a great solution. If XID collision is a problem, another
> solution could be an atomic XID shared between transports which
> belong
> to the same client.
>
> If there's no problem in the first place, that's even better. We
> thought when you said "I don't feel comfortable subjecting NFSv3/v4
> replay caches to this treatment yet" that you were referring to XID
> collision. Is there another potential issue with multipathing and
> replay caches?
There is the question of what to do when a NIC goes down. Do we fail
over to a different connection or not? The existing practices w.r.t.
DRCs suggest that we cannot do so; for instance the Linux server DRC
would break in that case, leading potentially to issues with non-
idempotent operations that need to be replayed.
--
Trond Myklebust
Linux NFS client maintainer, PrimaryData
trond.myklebust@xxxxxxxxxxxxxxx
��.n��������+%������w��{.n�����{��w��� jg��� �����ݢj����G���� ���j:+v���w�m������w�������h�����٥
