Re: Performance issues RGW (S3)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 2024-06-10 21:37, Anthony D'Atri wrote:

You are right here, but we use Ceph mainly for RBD. It performs 'good enough' for our RBD load.
You use RBD for archival?

No, storage for (light-weight) virtual machines.

I'm surprised that it's enough, I've seen HDDs fail miserably in that role.

The (CPU) load on the OSD nodes is quite low. Our MON/MGR/RGW aren't hosted on the OSD nodes and are running on modern hardware.
You didn't list additional nodes so I assumed. You might still do well to have a larger number of RGWs, wherever they run. RGWs often scale better horizontally than vertically.

Good to know. I'll check if adding more RGW nodes is possible.

To be clear, you don't need more nodes. You can add RGWs to the ones you already have. You have 12 OSD nodes - why not put an RGW on each?

Might be an option, just don't like the idea to host multiple components on nodes. But I'll consider it.


There are still serializations in the OSD and PG code. You have 240 OSDs, does your index pool have *at least* 256 PGs?

Index as the data pool has 256 PG's.

To be clear, that means whatever.rgw.buckets.index ?

No, sorry my bad. .index is 32 and .data is 256.


You might also disable Nagle on the RGW nodes.
I need to lookup what that exactly is and does.
It depends on the concurrency setting of Warp.
It look likes the objects/s is the bottleneck, not the throughput.
Max memory usage is about 80-90GB per node. CPU's are quite idling. Is it reasonable to expect more IOps / objects/s for RGW with my setup? At this moment I am not able to find the bottleneck what is causing the low obj/s.
HDDs are a false economy.
Got it :)
Ceph version is 15.2.
Thanks!
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux