Just left a comment at https://tracker.ceph.com/issues/44509
Generally bdev-new-db performs no migration, RocksDB might eventually do
that but no guarantee it moves everything.
One should use bluefs-bdev-migrate to do actual migration.
And I think that's the root cause for the above ticket.
Thanks,
Igor
On 4/24/2020 2:37 PM, Stefan Priebe - Profihost AG wrote:
No not a standalone Wal I wanted to ask whether bdev-new-db migrated
dB and Wal from hdd to ssd.
Stefan
Am 24.04.2020 um 13:01 schrieb Igor Fedotov <ifedotov@xxxxxxx>:
Unless you have 3 different types of disks beyond OSD (e.g. HDD, SSD,
NVMe) standalone WAL makes no sense.
On 4/24/2020 1:58 PM, Stefan Priebe - Profihost AG wrote:
Is Wal device missing? Do I need to run *bluefs-bdev-new-db and Wal?*
Greets,
Stefan
Am 24.04.2020 um 11:32 schrieb Stefan Priebe - Profihost AG
<s.priebe@xxxxxxxxxxxx>:
Hi Igor,
there must be a difference. I purged osd.0 and recreated it.
Now it gives:
ceph tell osd.0 bench
{
"bytes_written": 1073741824,
"blocksize": 4194304,
"elapsed_sec": 8.1554735639999993,
"bytes_per_sec": 131659040.46819863,
"iops": 31.389961354303033
}
What's wrong wiht adding a block.db device later?
Stefan
Am 23.04.20 um 20:34 schrieb Stefan Priebe - Profihost AG:
Hi,
if the OSDs are idle the difference is even more worse:
# ceph tell osd.0 bench
{
"bytes_written": 1073741824,
"blocksize": 4194304,
"elapsed_sec": 15.396707875000001,
"bytes_per_sec": 69738403.346825853,
"iops": 16.626931034761871
}
# ceph tell osd.38 bench
{
"bytes_written": 1073741824,
"blocksize": 4194304,
"elapsed_sec": 6.8903985170000004,
"bytes_per_sec": 155831599.77624846,
"iops": 37.153148597776521
}
Stefan
Am 23.04.20 um 14:39 schrieb Stefan Priebe - Profihost AG:
Hi,
Am 23.04.20 um 14:06 schrieb Igor Fedotov:
I don't recall any additional tuning to be applied to new DB
volume. And assume the hardware is pretty the same...
Do you still have any significant amount of data spilled over
for these updated OSDs? If not I don't have any valid
explanation for the phenomena.
just the 64k from here:
https://tracker.ceph.com/issues/44509
You might want to try "ceph osd bench" to compare OSDs under
pretty the same load. Any difference observed
Servers are the same HW. OSD Bench is:
# ceph tell osd.0 bench
{
"bytes_written": 1073741824,
"blocksize": 4194304,
"elapsed_sec": 16.091414781000001,
"bytes_per_sec": 66727620.822242722,
"iops": 15.909104543266945
}
# ceph tell osd.36 bench
{
"bytes_written": 1073741824,
"blocksize": 4194304,
"elapsed_sec": 10.023828538,
"bytes_per_sec": 107118933.6419194,
"iops": 25.539143953780986
}
OSD 0 is a Toshiba MG07SCA12TA SAS 12G
OSD 36 is a Seagate ST12000NM0008-2H SATA 6G
SSDs are all the same like the rest of the HW. But both drives
should give the same performance from their specs. The only other
difference is that OSD 36 was directly created with the block.db
device (Nautilus 14.2.7) and OSD 0 (14.2.8) does not.
Stefan
On 4/23/2020 8:35 AM, Stefan Priebe - Profihost AG wrote:
Hello,
is there anything else needed beside running:
ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-${OSD}
bluefs-bdev-new-db --dev-target /dev/vgroup/lvdb-1
I did so some weeks ago and currently i'm seeing that all osds
originally deployed with --block-db show 10-20% I/O waits while
all those got converted using ceph-bluestore-tool show 80-100%
I/O waits.
Also is there some tuning available to use more of the SSD? The
SSD (block-db) is only saturated at 0-2%.
Greets,
Stefan
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
