I assume most OSD nodes will normally run a single OSD, so this would not apply to most nodes. Only in specific cases (where multiple OSDs run on a single node) this would come up, and these specific cases might even require to have the journals split over multiple devices (multiple ssd-disks ...) In my case, this doesn't really matter, it is up to the provision software to make the needed symlinks/mounts. Rgds, Bernard On 05 Apr 2012, at 09:37, Andrey Korolyov wrote: > In ceph case, such layout breakage may be necessary in almost all > installations(except testing), comparing to almost all general-purpose > server software which need division like that only in very specific > setups. > > On Thu, Apr 5, 2012 at 11:28 AM, Bernard Grymonpon <bernard@xxxxxxxxxxxx> wrote: >> I feel it's up to the sysadmin to mount / symlink the correct storage devices on the correct paths - ceph should not be concerned that some volumes might need to sit together. >> >> Rgds, >> Bernard >> >> On 05 Apr 2012, at 09:12, Andrey Korolyov wrote: >> >>> Right, but probably we need journal separation at the directory level >>> by default, because there is a very small amount of cases when speed >>> of main storage is sufficient for journal or when resulting speed >>> decrease is not significant, so journal by default may go into >>> /var/lib/ceph/osd/journals/$i/journal where osd/journals mounted on >>> the fast disk. >>> >>> On Thu, Apr 5, 2012 at 10:57 AM, Bernard Grymonpon <bernard@xxxxxxxxxxxx> wrote: >>>> >>>> On 05 Apr 2012, at 08:32, Sage Weil wrote: >>>> >>>>> We want to standardize the locations for ceph data directories, configs, >>>>> etc. We'd also like to allow a single host to run OSDs that participate >>>>> in multiple ceph clusters. We'd like easy to deal with names (i.e., avoid >>>>> UUIDs if we can). >>>>> >>>>> The metavariables are: >>>>> cluster = ceph (by default) >>>>> type = osd, mon, mds >>>>> id = 1, foo, >>>>> name = $type.$id = osd.0, mds.a, etc. >>>>> >>>>> The $cluster variable will come from the command line (--cluster foo) or, >>>>> in the case of a udev hotplug tool or something, matching the uuid on the >>>>> device with the 'fsid = <uuid>' line in the available config files found >>>>> in /etc/ceph. >>>>> >>>>> The locations could be: >>>>> >>>>> ceph config file: >>>>> /etc/ceph/$cluster.conf (default is thus ceph.conf) >>>>> >>>>> keyring: >>>>> /etc/ceph/$cluster.keyring (fallback to /etc/ceph/keyring) >>>>> >>>>> osd_data, mon_data: >>>>> /var/lib/ceph/$cluster.$name >>>>> /var/lib/ceph/$cluster/$name >>>>> /var/lib/ceph/data/$cluster.$name >>>>> /var/lib/ceph/$type-data/$cluster-$id >>>>> >>>>> TV and I talked about this today, and one thing we want is for items of a >>>>> given type to live together in separate directory so that we don't have to >>>>> do any filtering to, say, get all osd data directories. This suggests the >>>>> last option (/var/lib/ceph/osd-data/ceph-1, >>>>> /var/lib/ceph/mon-data/ceph-foo, etc.), but it's kind of fugly. >>>>> >>>>> Another option would be to make it >>>>> >>>>> /var/lib/ceph/$type-data/$id >>>>> >>>>> (with no $cluster) and make users override the default with something that >>>>> includes $cluster (or $fsid, or whatever) in their $cluster.conf if/when >>>>> they want multicluster nodes that don't interfere. Then we'd get >>>>> /var/lib/ceph/osd-data/1 for non-crazy people, which is pretty easy. >>>> >>>> As a osd consists of data and the journal, it should stay together, with all info for that one osd in one place: >>>> >>>> I would suggest >>>> >>>> /var/lib/ceph/osd/$id/data >>>> and >>>> /var/lib/ceph/osd/$id/journal >>>> >>>> ($id could be replaced by $uuid or $name, for which I would prefer $uuid) >>>> >>>> Rgds, >>>> Bernard >>>> >>>>> >>>>> Any other suggestions? Thoughts? >>>>> sage >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
