On 04/05/2012 10:38 AM, Bernard Grymonpon wrote:
I assume most OSD nodes will normally run a single OSD, so this would not apply to most nodes.
Only in specific cases (where multiple OSDs run on a single node) this would come up, and these specific cases might even require to have the journals split over multiple devices (multiple ssd-disks ...)
I think that's a wrong assumption. On most systems I think multiple OSDs
will exist, it's debatable if one would run OSDs from different clusters
very often.
I'm currently using: osd data = /var/lib/ceph/$name
To get back to what sage mentioned, why add the "-data" suffix to a
directory name? Isn't it obvious that a directory will contain data?
As I think it is a very specific scenario where a machine would be
participating in multiple Ceph clusters I'd vote for:
/var/lib/ceph/$type/$id
Wido
In my case, this doesn't really matter, it is up to the provision software to make the needed symlinks/mounts.
Rgds,
Bernard
On 05 Apr 2012, at 09:37, Andrey Korolyov wrote:
In ceph case, such layout breakage may be necessary in almost all
installations(except testing), comparing to almost all general-purpose
server software which need division like that only in very specific
setups.
On Thu, Apr 5, 2012 at 11:28 AM, Bernard Grymonpon<bernard@xxxxxxxxxxxx> wrote:
I feel it's up to the sysadmin to mount / symlink the correct storage devices on the correct paths - ceph should not be concerned that some volumes might need to sit together.
Rgds,
Bernard
On 05 Apr 2012, at 09:12, Andrey Korolyov wrote:
Right, but probably we need journal separation at the directory level
by default, because there is a very small amount of cases when speed
of main storage is sufficient for journal or when resulting speed
decrease is not significant, so journal by default may go into
/var/lib/ceph/osd/journals/$i/journal where osd/journals mounted on
the fast disk.
On Thu, Apr 5, 2012 at 10:57 AM, Bernard Grymonpon<bernard@xxxxxxxxxxxx> wrote:
On 05 Apr 2012, at 08:32, Sage Weil wrote:
We want to standardize the locations for ceph data directories, configs,
etc. We'd also like to allow a single host to run OSDs that participate
in multiple ceph clusters. We'd like easy to deal with names (i.e., avoid
UUIDs if we can).
The metavariables are:
cluster = ceph (by default)
type = osd, mon, mds
id = 1, foo,
name = $type.$id = osd.0, mds.a, etc.
The $cluster variable will come from the command line (--cluster foo) or,
in the case of a udev hotplug tool or something, matching the uuid on the
device with the 'fsid =<uuid>' line in the available config files found
in /etc/ceph.
The locations could be:
ceph config file:
/etc/ceph/$cluster.conf (default is thus ceph.conf)
keyring:
/etc/ceph/$cluster.keyring (fallback to /etc/ceph/keyring)
osd_data, mon_data:
/var/lib/ceph/$cluster.$name
/var/lib/ceph/$cluster/$name
/var/lib/ceph/data/$cluster.$name
/var/lib/ceph/$type-data/$cluster-$id
TV and I talked about this today, and one thing we want is for items of a
given type to live together in separate directory so that we don't have to
do any filtering to, say, get all osd data directories. This suggests the
last option (/var/lib/ceph/osd-data/ceph-1,
/var/lib/ceph/mon-data/ceph-foo, etc.), but it's kind of fugly.
Another option would be to make it
/var/lib/ceph/$type-data/$id
(with no $cluster) and make users override the default with something that
includes $cluster (or $fsid, or whatever) in their $cluster.conf if/when
they want multicluster nodes that don't interfere. Then we'd get
/var/lib/ceph/osd-data/1 for non-crazy people, which is pretty easy.
As a osd consists of data and the journal, it should stay together, with all info for that one osd in one place:
I would suggest
/var/lib/ceph/osd/$id/data
and
/var/lib/ceph/osd/$id/journal
($id could be replaced by $uuid or $name, for which I would prefer $uuid)
Rgds,
Bernard
Any other suggestions? Thoughts?
sage
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
