Hi, I am trying to apply the resharding to a containerized OSD (16.2.4) as described here: https://docs.ceph.com/en/latest/rados/configuration/bluestore-config-ref/#rocksdb-sharding # ceph osd set noout # ceph orch daemon stop osd.13 # cephadm shell --name osd.13 # ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-13 --sharding="m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P" reshard 2021-07-08T15:12:03.392+0000 7f2e2173b3c0 -1 rocksdb: prepare_for_reshard failure parsing column options: block_cache={type=binned_lru} error resharding: (22) Invalid argument # exit # ceph orch daemon start osd.13 # ceph osd unset noout The OSD cannot start any more and has these errors in its log: Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 0 osd.13:7.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196) Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 1 bluestore(/var/lib/ceph/osd/ceph-13) _mount path /var/lib/ceph/osd/ceph-13 Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 0 bluestore(/var/lib/ceph/osd/ceph-13) _open_db_and_around read-only:0 repair:0 Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 1 bdev(0x564271210800 /var/lib/ceph/osd/ceph-13/block) open path /var/lib/ceph/osd/ceph-13/block Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 1 bdev(0x564271210800 /var/lib/ceph/osd/ceph-13/block) open size 107369988096 (0x18ffc00000, 100 GiB) block_size 4096 (4 KiB) non-rotational discard supported Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 1 bluestore(/var/lib/ceph/osd/ceph-13) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 data 0.06 Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 1 bdev(0x564271210c00 /var/lib/ceph/osd/ceph-13/block) open path /var/lib/ceph/osd/ceph-13/block Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 1 bdev(0x564271210c00 /var/lib/ceph/osd/ceph-13/block) open size 107369988096 (0x18ffc00000, 100 GiB) block_size 4096 (4 KiB) non-rotational discard supported Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-13/block size 100 GiB Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 1 bluefs mount Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.293+0000 7efc32db4080 1 bluefs _init_alloc shared, id 1, capacity 0x18ffc00000, block size 0x10000 Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.325+0000 7efc32db4080 1 bluefs mount shared_bdev_used = 0 Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.325+0000 7efc32db4080 -1 rocksdb: verify_sharding extra columns in rocksdb. rocksdb columns = [default,m-0,m-1,m-2,p-0,p-1,p-2] target columns = [reshardingXcommencingXlocked] Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.325+0000 7efc32db4080 -1 bluestore(/var/lib/ceph/osd/ceph-13) _open_db erroring opening db: Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.325+0000 7efc32db4080 1 bluefs umount Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.325+0000 7efc32db4080 1 bdev(0x564271210c00 /var/lib/ceph/osd/ceph-13/block) close Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.577+0000 7efc32db4080 1 bdev(0x564271210800 /var/lib/ceph/osd/ceph-13/block) close Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.825+0000 7efc32db4080 -1 osd.13 0 OSD:init: unable to mount object store Jul 08 17:13:46 cephtest24 bash[4161252]: debug 2021-07-08T15:13:46.825+0000 7efc32db4080 -1 ** ERROR: osd init failed: (5) Input/output error How do I correct the issue? Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
