Hello, thanks for your answer, but zapping the disk did not make any difference. I still get the same error. Looking at the debug output I found this error message that is probably the root of all trouble: # ceph-volume lvm prepare --bluestore --data /dev/sdg .... stderr: 2019-02-18 08:29:25.544 7fdaa50ed240 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid I found the bugreport below that seems to be exactly that problem I have: http://tracker.ceph.com/issues/15386 However there seems to be no solution up to now. Does anyone have more information how to get around this problem? Thanks Rainer Am 15.02.19 um 18:12 schrieb David Turner: > I have found that running a zap before all prepare/create commands with > ceph-volume helps things run smoother. Zap is specifically there to > clear everything on a disk away to make the disk ready to be used as an > OSD. Your wipefs command is still fine, but then I would lvm zap the > disk before continuing. I would run the commands like [1] this. I also > prefer the single command lvm create as opposed to lvm prepare and lvm > activate. Try that out and see if you still run into the problems > creating the BlueStore filesystem. > > [1] ceph-volume lvm zap /dev/sdg > ceph-volume lvm prepare --bluestore --data /dev/sdg > > On Thu, Feb 14, 2019 at 10:25 AM Rainer Krienke <krienke@xxxxxxxxxxxxxx > <mailto:krienke@xxxxxxxxxxxxxx>> wrote: > > Hi, > > I am quite new to ceph and just try to set up a ceph cluster. Initially > I used ceph-deploy for this but when I tried to create a BlueStore osd > ceph-deploy fails. Next I tried the direct way on one of the OSD-nodes > using ceph-volume to create the osd, but this also fails. Below you can > see what ceph-volume says. > > I ensured that there was no left over lvm VG and LV on the disk sdg > before I started the osd creation for this disk. The very same error > happens also on other disks not just for /dev/sdg. All the disk have 4TB > in size and the linux system is Ubuntu 18.04 and finally ceph is > installed in version 13.2.4-1bionic from this repo: > https://download.ceph.com/debian-mimic. > > There is a VG and two LV's on the system for the ubuntu system itself > that is installed on two separate disks configured as software raid1 and > lvm on top of the raid. But I cannot imagine that this might do any harm > to cephs osd creation. > > Does anyone have an idea what might be wrong? > > Thanks for hints > Rainer > > root@ceph1:~# wipefs -fa /dev/sdg > root@ceph1:~# ceph-volume lvm prepare --bluestore --data /dev/sdg > Running command: /usr/bin/ceph-authtool --gen-print-key > Running command: /usr/bin/ceph --cluster ceph --name > client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring > -i - osd new 14d041d6-0beb-4056-8df2-3920e2febce0 > Running command: /sbin/vgcreate --force --yes > ceph-1433ffd0-0a80-481a-91f5-d7a47b78e17b /dev/sdg > stdout: Physical volume "/dev/sdg" successfully created. > stdout: Volume group "ceph-1433ffd0-0a80-481a-91f5-d7a47b78e17b" > successfully created > Running command: /sbin/lvcreate --yes -l 100%FREE -n > osd-block-14d041d6-0beb-4056-8df2-3920e2febce0 > ceph-1433ffd0-0a80-481a-91f5-d7a47b78e17b > stdout: Logical volume "osd-block-14d041d6-0beb-4056-8df2-3920e2febce0" > created. > Running command: /usr/bin/ceph-authtool --gen-print-key > Running command: /bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0 > --> Absolute path not found for executable: restorecon > --> Ensure $PATH environment variable contains common executable > locations > Running command: /bin/chown -h ceph:ceph > /dev/ceph-1433ffd0-0a80-481a-91f5-d7a47b78e17b/osd-block-14d041d6-0beb-4056-8df2-3920e2febce0 > Running command: /bin/chown -R ceph:ceph /dev/dm-8 > Running command: /bin/ln -s > /dev/ceph-1433ffd0-0a80-481a-91f5-d7a47b78e17b/osd-block-14d041d6-0beb-4056-8df2-3920e2febce0 > /var/lib/ceph/osd/ceph-0/block > Running command: /usr/bin/ceph --cluster ceph --name > client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring > mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap > stderr: got monmap epoch 1 > Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-0/keyring > --create-keyring --name osd.0 --add-key > AQAAY2VcU968HxAAvYWMaJZmriUc4H9bCCp8XQ== > stdout: creating /var/lib/ceph/osd/ceph-0/keyring > added entity osd.0 auth auth(auid = 18446744073709551615 > key=AQAAY2VcU968HxAAvYWMaJZmriUc4H9bCCp8XQ== with 0 caps) > Running command: /bin/chown -R ceph:ceph > /var/lib/ceph/osd/ceph-0/keyring > Running command: /bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/ > Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore > bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap > --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid > 14d041d6-0beb-4056-8df2-3920e2febce0 --setuser ceph --setgroup ceph > stderr: 2019-02-14 13:45:54.788 7f3fcecb3240 -1 > bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid > stderr: /build/ceph-13.2.4/src/os/bluestore/KernelDevice.cc: In > function 'virtual int KernelDevice::read(uint64_t, uint64_t, > ceph::bufferlist*, IOContext*, bool)' thread 7f3fcecb3240 time > 2019-02-14 13:45:54.841130 > stderr: /build/ceph-13.2.4/src/os/bluestore/KernelDevice.cc: 821: > FAILED assert((uint64_t)r == len) > stderr: ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) > mimic (stable) -- Rainer Krienke, Uni Koblenz, Rechenzentrum, A22, Universitaetsstrasse 1 56070 Koblenz, Tel: +49261287 1312 Fax +49261287 100 1312 Web: http://userpages.uni-koblenz.de/~krienke PGP: http://userpages.uni-koblenz.de/~krienke/mypgp.html _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |