On Thu, Jul 29, 2021 at 12:28 PM Jeff Layton <jlayton@xxxxxxxxxx> wrote: > > And that's exactly what we need for the OpenStack use case. Clients > > are guest VMs, running who knows what. We cannot assume or require > > anything about how they resolve hostnames to addresses. > > > > -- Tom > > > > How are these IP addresses supplied to the tenants in an openstack > cluster? I guess this is using cinder? > > If guests can't rely on DNS resolution, then perhaps cinder could be > taught to hand the guests a random address from a set? Or even better, > hand them more than one address. At least that way they could decide to > pick a different host for their initial mount if there was a problem > contacting the first one they try. > > To reiterate, my main concern is that relying on an ingress controller > limits our options for improving the ability to scale in the future, > since it pretty much means you can't use NFSv4 migration at all. Two thoughts. First, if we want to distribute load across multiple IPs/servers via delegations, then that means we need to provide (ingress with) multiple IP addresses to work with. IIUC the primary reason to do this would be to avoid proxying all of the NFS traffic for a given instance of NFS service through a single IP/node. I suspect that roughly the same thing could also be accomplished with multiple instances of ingress for the same service and round-robin DNS. Second, the primary benefit that ingress provides today (a known, stable, user-provided virtual IP address) is extremely valuable to real users. The alternatives are something like: (1) query a ceph CLI/API to determine which IP to connect to, and if there is a failure, force unmount, (re)query the API, and remount the new IP; (2) manually constrain placement of the NFS service to a specific host with a known IP, and lose NFS service if that host fails. Neither of these are "HA" by any stretch of the imagination. What the cephadm ingress service does now is essentially identical to what kubernetes does, except that the update interval isn't low enough (yet) for lock reclaim to work. You get a known, stable IP, and NFS service can tolerate any cluster host failing. Ingress *also* uses haproxy to distribute load, but that probably isn't necessary for most users. If you don't need scale-out then a single backend NFS daemon is sufficient and haproxy is doing nothing but proxying traffic from the host with the virtual IP to the host where the daemon is running. This is really an implementation detail; we can swap out the implementation to do something else if we decide there are better tools to use... sage As far as the user is concerned, ingress is "on" (and you provide the virtual IP) or it's not. We can modify the implementation in the future to do whatever we like. Currently, ingress provides a few things: 1) Known, stable, user-provided virtual IP address. Regardless of where the nfs ganesha daemons get scheduled, there is one IP address for NFS service and it does not change. _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
