Thanks, this should help me with some debugging around the setting
Igor suggested.
Rich
On Tue, 5 Apr 2022 at 21:20, Rudenko Aleksandr <ARudenko@xxxxxxx> wrote:
>
> OSD uses sysfs device parameter "rotational" for detecting device type (HDD/SSD).
>
> You can see it:
>
> ceph osd metadata {osd_id}
>
> On 05.04.2022, 11:49, "Richard Bade" <hitrich@xxxxxxxxx> wrote:
>
> Hi Frank, yes I changed the device class to HDD but there seems to be some
> smarts in the background that apply the different settings that are not
> based on the class but some other internal mechanism.
> However, I did apply the class after creating the osd, rather than during.
> If someone knows how to manually specify this, I'd also be interested to
> know.
>
> I probably should also have said that I am using Nautilus and it may be
> different in newer versions.
>
> Rich
>
>
> On Tue, 5 Apr 2022, 20:39 Frank Schilder, <frans@xxxxxx> wrote:
>
> > Hi Richard,
> >
> > I'm planning to use dm_cache with bluestore OSDs on LVM. I was also
> > wondering how the device will be detected. I guess if I build the OSD
> > before assigning dm_cache space it will use the usual HDD defaults. Did you
> > try forcing the OSD to be in class HDD on build? I believe the OSD create
> > commands have a flag for that.
> >
> > If any of the OSD gurus looks at this, could you possibly point to a
> > reference about what parameters might need attention in such scenarios and
> > what the preferred deployment method would be?
> >
> > Thanks and best regards,
> > =================
> > Frank Schilder
> > AIT Risø Campus
> > Bygning 109, rum S14
> >
> > ________________________________________
> > From: Richard Bade <hitrich@xxxxxxxxx>
> > Sent: 05 April 2022 00:07:34
> > To: Ceph Users
> > Subject: Ceph Bluestore tweaks for Bcache
> >
> > Hi Everyone,
> > I just wanted to share a discovery I made about running bluestore on
> > top of Bcache in case anyone else is doing this or considering it.
> > We've run Bcache under Filestore for a long time with good results but
> > recently rebuilt all the osds on bluestore. This caused some
> > degradation in performance that I couldn't quite put my finger on.
> > Bluestore osds have some smarts where they detect the disk type.
> > Unfortunately in the case of Bcache it detects as SSD, when in fact
> > the HDD parameters are better suited.
> > I changed the following parameters to match the HDD default values and
> > immediately saw my average osd latency during normal workload drop
> > from 6ms to 2ms. Peak performance didn't change really, but a test
> > machine that I have running a constant iops workload was much more
> > stable as was the average latency.
> > Performance has returned to Filestore or better levels.
> > Here are the parameters.
> >
> > ; Make sure that we use values appropriate for HDD not SSD - Bcache
> > gets detected as SSD
> > bluestore_prefer_deferred_size = 32768
> > bluestore_compression_max_blob_size = 524288
> > bluestore_deferred_batch_ops = 64
> > bluestore_max_blob_size = 524288
> > bluestore_min_alloc_size = 65536
> > bluestore_throttle_cost_per_io = 670000
> >
> > ; Try to improve responsiveness when some disks are fully utilised
> > osd_op_queue = wpq
> > osd_op_queue_cut_off = high
> >
> > Hopefully someone else finds this useful.
> > _______________________________________________
> > ceph-users mailing list -- ceph-users@xxxxxxx
> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
> >
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
