On Thu, 29 Apr 2010 14:55:23 -0700
Dan Williams <dan.j.williams@xxxxxxxxx> wrote:
> On Wed, Apr 28, 2010 at 7:37 PM, Neil Brown <neilb@xxxxxxx> wrote:
> >> So something like DOMAIN spanning=imsm, to tell mdadm to follow imsm
> >> rules for this domain. Where 'spanning' is policy tag??
> >>
> >
> > Thanks.
> >
> > So we have two different ideas here.
> >
> > 1/ A given set of devices (paths) are all attached to the one controller.
> > 2/ A given array is not allowed to span controllers
> >
> > The first statement is somewhat similar to a statement about sparegroups.
> > It groups devices together is some way.
> >
> > The second is a policy statement, and is metadata specific to some extent.
> > If I create a native-metadata array using the controller, then adding other
> > devices from a different controller is a non-issue. It is only when an
> > IMSM array is created that it is an issue (and then - only if the array is to
> > be used for boot for for multi-boot).
> >
> > So the ARRAY line could have "exclusive-group=foo" where 'foo' is a group
> > name similar to those used for 'spare-group='
> > But that isn't much fun for auto-detect and auto-assembly.
> >
> > Maybe we want to extend the 'auto' line. It gives policy on a per-metadata
> > basis. Maybe:
> >
> > POLICY auto-assemble +1.x -all
> > POLICY same-group imsm
> >
> > where 'same-group' means that all the devices in an array must be in the
> > same spare-group. The 'domain' lines assign spare-groups to devices.
> >
> > Maybe "same-group" could be "same-$tag" so we could have different tags
> > for different metadatas....
> >
> > Is this working for anyone else, or have I lost the plot??
> >
>
> I am not grokking the separate POLICY line, especially for defining
> the spare-migration border because that is already what DOMAIN is
> specifying.
Is it? This is what I'm not yet 100% convinced about.
We seem to be saying:
- A DOMAIN is a set of devices that are handled the same way for
hotplug
- A DOMAIN is a set of devices that define a boundary on spare
migration
and I'm not sure those sets are necessarily isomorphic - though I agree that
they will often be the same.
Does each DOMAIN line define a separate migration boundary so that devices
cannot migrate 'across domains'??
If we were to require that, I would probably want multiple 'path=' words
allowed for a single domain so we can create a union.
>
> Here is what I think we need to allow for simple honoring of platform
> constraints but without needing to expose all the nuances of those
> constraints in config-file syntax... yet.
>
> 1/ Allow path= to take a metadata name this allows the handler to
> identify its known controller ports, alleviating the user from needing
> to track which ports are allowed, especially as it may change over
> time. If someone really wants to see which ports a metadata handler
> cares about we could have a DOMAIN line dumped by --detail-platform
> --brief -e imsm. However for simplicity I would rather just dump:
>
> DOMAIN path=imsm action=spare-same-port spare-migration=imsm
>
So "path=imsm" means "all devices which are attached to a controller which
seems to understand IMSM natively".
What if a system had two such controllers - one on-board and one on a plug-in
card. This might not be possibly for IMSM but would be for DDF.
I presume the default would be that the controllers are separate domains -
would you agree? So the above DOMAIN line would potentially create multiple
'domains' at least for spare-migration.
> 2/ I think we should always block configurations that cross domain
> boundaries. One can always append more path= lines to override this.
I think we all agree on this. Require --force to create an array, or add
devices to an array, where that would cross an established spare-group...
The details are still a bit vague for me but the principle is good.
>
> 3/ The metadata handler may want to restrict/control where spares are
> placed in a domain. To enable interaction with CIM we are looking to
> add a storage-pool id to the metadata. The primary usage of this will
> be to essentially encode a spare-group number in the metadata. This
> seems to require a spare-migration= option to the DOMAIN line. By
> default it is 'all' but it can be set to a metadata-name to let the
> handler apply its internal migration policy.
I'm not following you. Are you talking about subsets of a domain? Subdomains?
Do the storage-pools follow hardware port locations, or dynamic configuration
of individual devices (hence being recorded in metadata).
This is how I think spare migration should work:
Spare migration is controlled entirely by the 'spare-group' attribute.
A spare-group is an attribute of a device. A device may have multiple
spare-group attributes (it might be in multiple groups).
There are two ways a device can be assigned a spare-group.
1/ If an array is tagged with a spare-group= in mdadm.conf then any device
in that array gets that spare-group attribute
2/ If a DOMAIN is tagged with a spare-group attribute then any device
in that domain gets that spare-group attribute
When mdadm --monitor needs to find a hot spare for an array or container
which is degraded, it collects a list of spare-group attributes
for all devices in the array, then finds any device (of suitable size)
that has a spare-group attribute matching any of those.
Possibly a weighting should prefer spare-groups that are more prevalent in
the array, so that if you add a foreign device in an emergency, mdadm won't
feel too free to add other foreign devices (but is still allowed to).
You seem to be suggesting that the spare-group tag could also be specified
by the metadata. I think I'm happy with that.
A DOMAIN line without an explicit spare-group= tag might imply an implicit
spare-group= tag where the spare-group name is some generated string that
is unique to that DOMAIN line.
So all devices in a DOMAIN line are effectively interchangeable, but it is
easy to stretch the migration barrier around multiple domains by giving
them all a matching spare-group tag.
When you create an array, every pair of devices much share a spare-group, or
else one of them must not be in an spare-group. Is that right?
NeilBrown
>
> That should cover everything I would like to expose Comments?
>
> --
> Dan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
