> On Dec 1, 2016, at 6:26 PM, Christian Balzer <chibi@xxxxxxx> wrote:
>
> On Thu, 1 Dec 2016 18:06:38 -0600 Reed Dier wrote:
>
>> Apologies if this has been asked dozens of times before, but most answers are from pre-Jewel days, and want to double check that the methodology still holds.
>>
> It does.
>
>> Currently have 16 OSD’s across 8 machines with on-disk journals, created using ceph-deploy.
>>
>> These machines have NVMe storage (Intel P3600 series) for the system volume, and am thinking about carving out a partition for SSD journals for the OSD’s. The drives don’t make tons of use of the local storage, so should have plenty of io overhead to support the OSD journaling, as well as the P3600 should have the endurance to handle the added write wear.
>>
> Slight disconnect there, money for a NVMe (which size?) and on disk
> journals? ^_-
NVMe was already in place before the ceph project began. 400GB P3600, with ~275GB available space after swap partition.
>> From what I’ve read, you need a partition per OSD journal, so with the probability of a third (and final) OSD being added to each node, I should create 3 partitions, each ~8GB in size (is this a good value? 8TB OSD’s, is the journal size based on size of data or number of objects, or something else?).
>>
> Journal size is unrelated to the OSD per se, with default parameters and
> HDDs for OSDs a size of 10GB would be more than adequate, the default of
> 5GB would do as well.
I was under the impression that it was agnostic to either metric, but figured I should ask while I had the chance.
>> So:
>> {create partitions}
>> set noout
>> service ceph stop osd.$i
>> ceph-osd -i osd.$i —flush-journal
>> rm -f rm -f /var/lib/ceph/osd/<osd-id>/journal
> Typo and there should be no need for -f. ^_^
>
>> ln -s /var/lib/ceph/osd/<osd-id>/journal /dev/<ssd-partition-for-journal>
> Even though in your case with a single(?) NVMe there is little chance for
> confusion, ALWAYS reference to devices by their UUID or similar, I prefer
> the ID:
> ---
> lrwxrwxrwx 1 root root 44 May 21 2015 journal -> /dev/disk/by-id/wwn-0x55cd2e404b73d570-part4
> —
Correct, would reference by UUID.
Thanks again for the sanity check.
Reed
>
>> ceph-osd -i osd.$i -mkjournal
>> service ceph start osd.$i
>> ceph osd unset noout
>>
>> Does this logic appear to hold up?
>>
> Yup.
>
> Christian
>
>> Appreciate the help.
>>
>> Thanks,
>>
>> Reed
>
> --
> Christian Balzer Network/Systems Engineer
> chibi@xxxxxxx Global OnLine Japan/Rakuten Communications
> http://www.gol.com/
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
