We've started hitting this issue again, despite having bitmap allocator configured. The logs just before the crash look similar to before (pasted below).
So perhaps this isn't a hybrid allocator issue after all?
I'm still struggling to collect the full set of diags / run ceph-bluestore-tool commands but I'll keep trying today.
Sep 8 08:29:25 condor_sc2 container_name/ceph-osd-2[1003]: 2021-09-08T08:29:25.041+0000 7fa0c0597f00 1 bluefs _allocate failed to allocate 0xf0124 on bdev 1, free 0x30000; fallback to
bdev 2
Sep 8 08:29:25 condor_sc2 container_name/ceph-osd-2[1003]: 2021-09-08T08:29:25.041+0000 7fa0c0597f00 1 bluefs _allocate unable to allocate 0xf0124 on bdev 2, free 0xffffffffffffffff;
fallback to slow device expander
Sep 8 08:29:25 condor_sc2 container_name/ceph-osd-2[1003]: 2021-09-08T08:29:25.219+0000 7fa0c0597f00 -1 bluestore(/var/lib/ceph/osd/ceph-2) allocate_bluefs_freespace failed to allocate
on 0x3b3a0000 min_size 0x100000 > allocated total 0x4c30000 bluefs_shared_alloc_size 0x10000 allocated 0x0 available 0x 7090ac000
Sep 8 08:29:25 condor_sc2 container_name/ceph-osd-2[1003]: 2021-09-08T08:29:25.219+0000 7fa0c0597f00 -1 bluefs _allocate failed to expand slow device to fit +0xf0124
Sep 8 08:29:25 condor_sc2 container_name/ceph-osd-2[1003]: 2021-09-08T08:29:25.219+0000 7fa0c0597f00 -1 bluefs _flush_range allocated: 0x6510000 offset: 0x6500ae5 length: 0xff63f
Sep 8 08:29:25 condor_sc2 container_name/ceph-osd-2[1003]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.13/rpm/el8/BUILD/ceph-15.2.13/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7fa0c0597f00 time 2021-09-08T08:29:25.219765+0000
Sep 8 08:29:25 condor_sc2 container_name/ceph-osd-2[1003]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/15.2.13/rpm/el8/BUILD/ceph-15.2.13/src/os/bluestore/BlueFS.cc: 2892: ceph_abort_msg("bluefs enospc")
Sep 8 08:29:25 condor_sc2 container_name/ceph-osd-2[1003]: ceph version 15.2.13 (c44bc49e7a57a87d84dfff2a077a2058aa2172e2) octopus (stable)
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
