Hi,
just remove the wal_devices part from your specs, ceph-volume will
automatically put the wal onto the same SSD. If you want to use both
SSDs for DB I would also remove the "limit" filter so ceph-volume can
use both SSDs for block.db. You don't seem to have more than those two
SSDs per node so you wouldn't need a limit, just the rotational filter.
Zitat von Chris <hagfelsh@xxxxxxxxx>:
The lines I cited as examples of cephadm misinterpreting rotational states
were pulled from the mgr container stderr, acquired via docker logs <ceph
mgr container>.
Your comments on my deployment strategy are very helpful--I figured
(incorrectly?) that having the db & wal on faster drives would benefit the
overall throughput of the cluster.
The use case for this is just warm storage. I built it out of scrap parts
in the lab, so our expectations of its performance should be carefully
managed :)
That being said, I'd be happy to purge the cluster again and apply what
you're describing: both WAL & DB on both SSDs. I haven't figured out how
to assign both roles to them via an osd spec yaml.
On the OSD Spec page ( https://docs.ceph.com/en/latest/cephadm/osd/ ), I
see mention of what I was trying to do: db_device and wal_device, but
nothing that would permit both to be simultaneously assigned to the SSDs.
Here's the spec I was working on to get this far:
service_type: osd
service_id: osd_spec_default
placement:
host_pattern: '*'
data_devices:
rotational: 1
db_devices:
rotational: 0
limit: 1
wal_devices:
rotational: 0
limit: 1
Do you know what to change to apply the plan you described? I'd be happy
to try it!
From: Eugen Block <eblock@xxxxxx>
To: ceph-users@xxxxxxx
Cc:
Bcc:
Date: Mon, 27 Sep 2021 10:06:43 +0000
Subject: Re: Orchestrator is internally ignoring applying a
spec against SSDs, apparently determining they're rotational.
Hi,
I read your first email again and noticed that ceph-volume already
identifies the drives sdr and sds as non-rotational and as available.
That would also explain the empty rejected_reasons field because they
are not rejected at (this stage?). Where do you read that information
that one SSD is identified as rotational?
The next thing I'm wondering about is the ratio, if I counted
correctly you try to fit 22 DBs into one SSD with 186 GB size which
would make one DB only about 8 GB. What is your use case for this
cluster? For RBD this small DB might be enough but for S3 this most
likely won't be enough, depending on the actual workload of course.
And why do you want to split WAL and DB anyway if both SSDs are
identical? You would only benefit from a separate WAL device if that
is faster than the DB device. This doesn't solve the mentioned
problem, of course, but it doesn't really make sense. I would
recommend to use both SSDs for both WAL and DB, this would improve
your ratio (11 OSDs per SSD) and also reduce the impact of a failing
SSD (22 drives vs. 11 drives if one SSD fails).
Regards,
Eugen
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
