On Thu, Jan 3, 2019 at 1:43 PM Jeff Layton <jlayton@xxxxxxxxxx> wrote:
> > I'd also suggest reversing the object names (if it's not too late for
> > that) so it sorts better. Objects could be named "100/{conf,exports}".
> >
> > To be clear, the purpose of this command is solely to setup the RADOS
> > objects storing the conf/exports for the Ganesha cluster?
> >
>
> Definitely not too late to change the naming format. None of the
> dependent pieces are merged yet, and I haven't written any code for this
> just yet.
>
> I'm curious though -- why would you want to put the unique bit first?
It's just easier to sort/organize when doing rados operations to look
at those objects. It's no big deal either way.
> > > [...]
> > > From there, we'd need to also be able to "link" and "unlink" these
> > > export objects into the config files for each daemon. So if I have a
> > > cluster of 2 servers with nodeids "a" and "b":
> > >
> > > $ ceph nfsexport link --pool=mypool \
> > > --namespace=mynamespace \
> > > --id=100 \
> > > --node=a \
> > > --node=b
> > >
> > > ...with a corresponding "unlink" command. That would append objects
> > > called "conf-a" and "conf-b" with this line:
> >
> > I got lost here. Assuming we're starting two Ganesha servers for a
> > single cluster (active=2), wouldn't they have the same export block
> > (i.e. export-100)? And, wouldn't that export block already be "linked"
> > into the config (conf-100)?
> >
> >
>
> No, it goes something like this:
>
> Each ganesha daemon started by rook gets its own "supplemental" config
> file. So we end up with objects that look like this:
>
> For ganesha node "a" - rados://mypool/mynamespace/conf-a
> For ganesha node "b" - rados://mypool/mynamespace/conf-b
>
> Rook's ganesha CRD creates empty objects if these don't exist when the
> daemon is first started, but we can pre-populate them too.
>
> In most cases, these files will likely be identical across the cluster,
> but allowing for a different config per-node allows us eventually use
> migration to help shift clients off of a server prior to downsizing the
> cluster.
Ah, I misunderstood by thinking there would be a volume config like
"100-conf". I see now that they are configs for each Ganesha node.
> So, the real question is what should go in the conf-* objects?
>
> I was proposing that we put each EXPORT block into separate export-<id>
> objects, and then just add a series of %url directives to the conf-*
> objects to slurp each one in.
>
> That said, there are many ways to set this up so I'm definitely open to
> suggestion here.
The linking makes sense to me now, but only if we actually want to
have a given Ganesha cluster export multiple sub-volumes. Otherwise,
we would just statically set the export include when first writing the
node's config. Or am I missing something?
I'm still leaning towards not multiplexing exports for the Ganesha
clusters but I'd be okay with adding in the machinery for it.
--
Patrick Donnelly
