I’ve actually had to migrate every single journal in many clusters from one (horrible) SSD model to a better SSD. It went smoothly. You’ll also need to update your /var/lib/ceph/osd/ceph-*/journal_uuid file.
Honestly, the only challenging part was mapping and automating the back and forth conversion from /dev/sd* to the uuid for the corresponding osd. I would share the script, but it was at my previous employer.
Warren Wang
Walmart ✻
On 12/1/16, 7:26 PM, "ceph-users on behalf of Christian Balzer" <ceph-users-bounces@xxxxxxxxxxxxxx on behalf of chibi@xxxxxxx> wrote:
On Thu, 1 Dec 2016 18:06:38 -0600 Reed Dier wrote:
> Apologies if this has been asked dozens of times before, but most answers are from pre-Jewel days, and want to double check that the methodology still holds.
>
It does.
> Currently have 16 OSD’s across 8 machines with on-disk journals, created using ceph-deploy.
>
> These machines have NVMe storage (Intel P3600 series) for the system volume, and am thinking about carving out a partition for SSD journals for the OSD’s. The drives don’t make tons of use of the local storage, so should have plenty of io overhead to support the OSD journaling, as well as the P3600 should have the endurance to handle the added write wear.
>
Slight disconnect there, money for a NVMe (which size?) and on disk
journals? ^_-
> From what I’ve read, you need a partition per OSD journal, so with the probability of a third (and final) OSD being added to each node, I should create 3 partitions, each ~8GB in size (is this a good value? 8TB OSD’s, is the journal size based on size of data or number of objects, or something else?).
>
Journal size is unrelated to the OSD per se, with default parameters and
HDDs for OSDs a size of 10GB would be more than adequate, the default of
5GB would do as well.
> So:
> {create partitions}
> set noout
> service ceph stop osd.$i
> ceph-osd -i osd.$i —flush-journal
> rm -f rm -f /var/lib/ceph/osd/<osd-id>/journal
Typo and there should be no need for -f. ^_^
> ln -s /var/lib/ceph/osd/<osd-id>/journal /dev/<ssd-partition-for-journal>
Even though in your case with a single(?) NVMe there is little chance for
confusion, ALWAYS reference to devices by their UUID or similar, I prefer
the ID:
---
lrwxrwxrwx 1 root root 44 May 21 2015 journal -> /dev/disk/by-id/wwn-0x55cd2e404b73d570-part4
---
> ceph-osd -i osd.$i -mkjournal
> service ceph start osd.$i
> ceph osd unset noout
>
> Does this logic appear to hold up?
>
Yup.
Christian
> Appreciate the help.
>
> Thanks,
>
> Reed
--
Christian Balzer Network/Systems Engineer
chibi@xxxxxxx Global OnLine Japan/Rakuten Communications
http://www.gol.com/
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
This email and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. If you have received this email in error destroy it immediately. *** Walmart Confidential ***
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
