This came from a previous thread that I started last year, so you may want to look in the archive. https://www.mail-archive.com/ceph-users@xxxxxxx/msg11572.html Although the doc page it refers to looks to have disappeared :( You can use "ceph orch daemon add osd <host>:<path to multipath device>" I've been using 1=mpatha pvcreate /dev/mapper/$1 vgcreate $1-vg /dev/mapper/$1 lvcreate -l 100%FREE -n $1-lv $1-vg ceph orch daemon add osd dampwood48:$1-vg/$1-lv to create osd's on the multipath devices terrible dev opt script but it works. I've not currently got a method to use the yaml device description method (which would be much more ideal), hence there is no obvious way to use separate db_devices, but this does look to work for me as far as it goes. Hope that helps Peter Childs On Tue, 25 Jan 2022, 17:53 Thomas Roth, <t.roth@xxxxxx> wrote: > Would like to know that as well. > > I have the same setup - cephadm, Pacific, CentOS8, and a host with a > number of HDDs which are all connect by 2 paths. > No way to use these without multipath > > > ceph orch daemon add osd serverX:/dev/sdax > > > Cannot update volume group ceph-51f8b9b0-2917-431d-8a6d-8ff90440641b > with duplicate PV devices > > (because sdax == sdce, etc.) > > and with multipath, it fails with > > > ceph orch daemon add osd serverX:/dev/mapper/mpathbq > > > podman: stderr --> IndexError: list index out of range > > > Quite strange that the 'future of storage' does not know how to handle > multipath devices? > > Regrads, > Thomas > > > On 12/23/21 18:40, Michal Strnad wrote: > > Hi all. > > > > We have problem using disks accessible via multipath. We are using > cephadm for deployment, Pacific version for containers, CentOS 8 Stream on > servers > > and following LVM configuration. > > > > devices { > > multipath_component_detection = 1 > > } > > > > > > > > We tried several methods. > > > > 1.) Direct approach. > > > > cephadm shell > > /mapper/mpatha > > > > Errors are attached in 1.output file. > > > > > > > > 2. With the help of OSD specifications where they are mpathX devices > used. > > > > service_type: osd > > service_id: osd-spec-serverX > > placement: > > host_pattern: 'serverX' > > spec: > > data_devices: > > paths: > > - /dev/mapper/mpathaj > > - /dev/mapper/mpathan > > - /dev/mapper/mpatham > > db_devices: > > paths: > > - /dev/sdc > > encrypted: true > > > > Errors are attached in 2.output file. > > > > > > 2. With the help of OSD specifications where they are dm-X devices used. > > > > service_type: osd > > service_id: osd-spec-serverX > > placement: > > host_pattern: 'serverX' > > spec: > > data_devices: > > paths: > > - /dev/dm-1 > > - /dev/dm-2 > > - /dev/dm-3 > > - /dev/dm-X > > db_devices: > > size: ':2TB' > > encrypted: true > > > > Errors are attached in 3.output file. > > > > What is the right method for multipath deployments? I didn't find much > on this topic. > > > > Thank you > > > > Michal > > > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > -- > -------------------------------------------------------------------- > Thomas Roth > HPC Department > > GSI Helmholtzzentrum für Schwerionenforschung GmbH > Planckstr. 1, 64291 Darmstadt, http://www.gsi.de/ > > Gesellschaft mit beschraenkter Haftung > > Sitz der Gesellschaft / Registered Office: Darmstadt > Handelsregister / Commercial Register: > Amtsgericht Darmstadt, HRB 1528 > > Geschaeftsfuehrung / Managing Directors: > Professor Dr. Paolo Giubellino, Ursula Weyrich, Jörg Blaurock > > Vorsitzender des GSI-Aufsichtsrates / > Chairman of the Supervisory Board: > Staatssekretaer / State Secretary Dr. Georg Schütte > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |