My ceph cluster was made of all-flash nvme, like this:
1. 10*nodes , every node has 21 nvme device was used to rgw data pool, and other 1 nvme was used to index pool
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-26 1467.11609 root bd-ssd03-data
-43 146.71161 host bd-ssd03-node01-f6fe3bdc-a1b6-496f-8fff-adb240afaedf
0 nvme 6.98627 osd.0 up 1.00000 1.00000
4 nvme 6.98627 osd.4 up 1.00000 1.00000
11 nvme 6.98627 osd.11 up 1.00000 1.00000
21 nvme 6.98627 osd.21 up 1.00000 1.00000
31 nvme 6.98627 osd.31 up 1.00000 1.00000
46 nvme 6.98627 osd.46 up 1.00000 1.00000
56 nvme 6.98627 osd.56 up 1.00000 1.00000
67 nvme 6.98627 osd.67 up 1.00000 1.00000
77 nvme 6.98627 osd.77 up 1.00000 1.00000
87 nvme 6.98627 osd.87 up 1.00000 1.00000
97 nvme 6.98627 osd.97 up 1.00000 1.00000
107 nvme 6.98627 osd.107 up 1.00000 1.00000
117 nvme 6.98627 osd.117 up 1.00000 1.00000
127 nvme 6.98627 osd.127 up 1.00000 1.00000
137 nvme 6.98627 osd.137 up 1.00000 1.00000
147 nvme 6.98627 osd.147 up 1.00000 1.00000
157 nvme 6.98627 osd.157 up 1.00000 1.00000
168 nvme 6.98627 osd.168 up 1.00000 1.00000
178 nvme 6.98627 osd.178 up 1.00000 1.00000
188 nvme 6.98627 osd.188 up 1.00000 1.00000
198 nvme 6.98627 osd.198 up 1.00000 1.00000
-31 146.71161 host bd-ssd03-node02-ffedbbd9-3794-4b10-ac1d-bf7595395892
1 nvme 6.98627 osd.1 up 1.00000 1.00000
6 nvme 6.98627 osd.6 up 1.00000 1.00000
14 nvme 6.98627 osd.14 up 1.00000 1.00000
24 nvme 6.98627 osd.24 up 1.00000 1.00000
34 nvme 6.98627 osd.34 up 1.00000 1.00000
45 nvme 6.98627 osd.45 up 1.00000 1.00000
55 nvme 6.98627 osd.55 up 1.00000 1.00000
66 nvme 6.98627 osd.66 up 1.00000 1.00000
76 nvme 6.98627 osd.76 up 1.00000 1.00000
86 nvme 6.98627 osd.86 up 1.00000 1.00000
96 nvme 6.98627 osd.96 up 1.00000 1.00000
106 nvme 6.98627 osd.106 up 1.00000 1.00000
116 nvme 6.98627 osd.116 up 1.00000 1.00000
130 nvme 6.98627 osd.130 up 1.00000 1.00000
141 nvme 6.98627 osd.141 up 1.00000 1.00000
151 nvme 6.98627 osd.151 up 1.00000 1.00000
161 nvme 6.98627 osd.161 up 1.00000 1.00000
171 nvme 6.98627 osd.171 up 1.00000 1.00000
181 nvme 6.98627 osd.181 up 1.00000 1.00000
191 nvme 6.98627 osd.191 up 1.00000 1.00000
201 nvme 6.98627 osd.201 up 1.00000 1.00000
-29 146.71161 host bd-ssd03-node03-637c96a1-6b93-42cc-9b35-44b729bd7a5c
2 nvme 6.98627 osd.2 up 1.00000 1.00000
8 nvme 6.98627 osd.8 up 1.00000 1.00000
17 nvme 6.98627 osd.17 up 1.00000 1.00000
27 nvme 6.98627 osd.27 up 1.00000 1.00000
37 nvme 6.98627 osd.37 up 1.00000 1.00000
48 nvme 6.98627 osd.48 up 1.00000 1.00000
58 nvme 6.98627 osd.58 up 1.00000 1.00000
68 nvme 6.98627 osd.68 up 1.00000 1.00000
79 nvme 6.98627 osd.79 up 1.00000 1.00000
89 nvme 6.98627 osd.89 up 1.00000 1.00000
99 nvme 6.98627 osd.99 up 1.00000 1.00000
109 nvme 6.98627 osd.109 up 1.00000 1.00000
119 nvme 6.98627 osd.119 up 1.00000 1.00000
129 nvme 6.98627 osd.129 up 1.00000 1.00000
140 nvme 6.98627 osd.140 up 1.00000 1.00000
150 nvme 6.98627 osd.150 up 1.00000 1.00000
166 nvme 6.98627 osd.166 up 1.00000 1.00000
176 nvme 6.98627 osd.176 up 1.00000 1.00000
186 nvme 6.98627 osd.186 up 1.00000 1.00000
196 nvme 6.98627 osd.196 up 1.00000 1.00000
206 nvme 6.98627 osd.206 up 1.00000 1.00000
-41 146.71161 host bd-ssd03-node04-87186f77-3df0-4b71-8103-fdc2a104eb17
3 nvme 6.98627 osd.3 up 1.00000 1.00000
10 nvme 6.98627 osd.10 up 1.00000 1.00000
20 nvme 6.98627 osd.20 up 1.00000 1.00000
30 nvme 6.98627 osd.30 up 1.00000 1.00000
40 nvme 6.98627 osd.40 up 1.00000 1.00000
50 nvme 6.98627 osd.50 up 1.00000 1.00000
60 nvme 6.98627 osd.60 up 1.00000 1.00000
70 nvme 6.98627 osd.70 up 1.00000 1.00000
81 nvme 6.98627 osd.81 up 1.00000 1.00000
90 nvme 6.98627 osd.90 up 1.00000 1.00000
100 nvme 6.98627 osd.100 up 1.00000 1.00000
110 nvme 6.98627 osd.110 up 1.00000 1.00000
120 nvme 6.98627 osd.120 up 1.00000 1.00000
132 nvme 6.98627 osd.132 up 1.00000 1.00000
143 nvme 6.98627 osd.143 up 1.00000 1.00000
153 nvme 6.98627 osd.153 up 1.00000 1.00000
162 nvme 6.98627 osd.162 up 1.00000 1.00000
172 nvme 6.98627 osd.172 up 1.00000 1.00000
182 nvme 6.98627 osd.182 up 1.00000 1.00000
192 nvme 6.98627 osd.192 up 1.00000 1.00000
202 nvme 6.98627 osd.202 up 1.00000 1.00000
-25 146.71161 host bd-ssd03-node05-a6f31379-cb27-4bc3-bbda-726023d9ff96
5 nvme 6.98627 osd.5 up 1.00000 1.00000
13 nvme 6.98627 osd.13 up 1.00000 1.00000
23 nvme 6.98627 osd.23 up 1.00000 1.00000
33 nvme 6.98627 osd.33 up 1.00000 1.00000
43 nvme 6.98627 osd.43 up 1.00000 1.00000
53 nvme 6.98627 osd.53 up 1.00000 1.00000
64 nvme 6.98627 osd.64 up 1.00000 1.00000
78 nvme 6.98627 osd.78 up 1.00000 1.00000
88 nvme 6.98627 osd.88 up 1.00000 1.00000
98 nvme 6.98627 osd.98 up 1.00000 1.00000
108 nvme 6.98627 osd.108 up 1.00000 1.00000
118 nvme 6.98627 osd.118 up 1.00000 1.00000
128 nvme 6.98627 osd.128 up 1.00000 1.00000
139 nvme 6.98627 osd.139 up 1.00000 1.00000
149 nvme 6.98627 osd.149 up 1.00000 1.00000
159 nvme 6.98627 osd.159 up 1.00000 1.00000
169 nvme 6.98627 osd.169 up 1.00000 1.00000
179 nvme 6.98627 osd.179 up 1.00000 1.00000
189 nvme 6.98627 osd.189 up 1.00000 1.00000
199 nvme 6.98627 osd.199 up 1.00000 1.00000
209 nvme 6.98627 osd.209 up 1.00000 1.00000
-33 146.71161 host bd-ssd03-node06-e4288b8f-7d64-40c1-a3f1-c60757e1fbb1
7 nvme 6.98627 osd.7 up 1.00000 1.00000
16 nvme 6.98627 osd.16 up 1.00000 1.00000
26 nvme 6.98627 osd.26 up 1.00000 1.00000
36 nvme 6.98627 osd.36 up 1.00000 1.00000
51 nvme 6.98627 osd.51 up 1.00000 1.00000
62 nvme 6.98627 osd.62 up 1.00000 1.00000
71 nvme 6.98627 osd.71 up 1.00000 1.00000
82 nvme 6.98627 osd.82 up 1.00000 1.00000
93 nvme 6.98627 osd.93 up 1.00000 1.00000
103 nvme 6.98627 osd.103 up 1.00000 1.00000
113 nvme 6.98627 osd.113 up 1.00000 1.00000
123 nvme 6.98627 osd.123 up 1.00000 1.00000
134 nvme 6.98627 osd.134 up 1.00000 1.00000
145 nvme 6.98627 osd.145 up 1.00000 1.00000
155 nvme 6.98627 osd.155 up 1.00000 1.00000
165 nvme 6.98627 osd.165 up 1.00000 1.00000
175 nvme 6.98627 osd.175 up 1.00000 1.00000
185 nvme 6.98627 osd.185 up 1.00000 1.00000
195 nvme 6.98627 osd.195 up 1.00000 1.00000
208 nvme 6.98627 osd.208 up 1.00000 1.00000
215 nvme 6.98627 osd.215 up 1.00000 1.00000
-37 146.71161 host bd-ssd03-node07-fabbd3f5-1466-493b-85f4-dac9254720c4
9 nvme 6.98627 osd.9 up 1.00000 1.00000
18 nvme 6.98627 osd.18 up 1.00000 1.00000
28 nvme 6.98627 osd.28 up 1.00000 1.00000
44 nvme 6.98627 osd.44 up 1.00000 1.00000
54 nvme 6.98627 osd.54 up 1.00000 1.00000
65 nvme 6.98627 osd.65 up 1.00000 1.00000
75 nvme 6.98627 osd.75 up 1.00000 1.00000
85 nvme 6.98627 osd.85 up 1.00000 1.00000
95 nvme 6.98627 osd.95 up 1.00000 1.00000
105 nvme 6.98627 osd.105 up 1.00000 1.00000
115 nvme 6.98627 osd.115 up 1.00000 1.00000
125 nvme 6.98627 osd.125 up 1.00000 1.00000
136 nvme 6.98627 osd.136 up 1.00000 1.00000
146 nvme 6.98627 osd.146 up 1.00000 1.00000
156 nvme 6.98627 osd.156 up 1.00000 1.00000
167 nvme 6.98627 osd.167 up 1.00000 1.00000
177 nvme 6.98627 osd.177 up 1.00000 1.00000
187 nvme 6.98627 osd.187 up 1.00000 1.00000
197 nvme 6.98627 osd.197 up 1.00000 1.00000
207 nvme 6.98627 osd.207 up 1.00000 1.00000
214 nvme 6.98627 osd.214 up 1.00000 1.00000
-35 146.71161 host bd-ssd03-node08-65ab0995-22d0-4b4c-b661-4cc53ceb5b61
12 nvme 6.98627 osd.12 up 1.00000 1.00000
22 nvme 6.98627 osd.22 up 1.00000 1.00000
32 nvme 6.98627 osd.32 up 1.00000 1.00000
42 nvme 6.98627 osd.42 up 1.00000 1.00000
52 nvme 6.98627 osd.52 up 1.00000 1.00000
63 nvme 6.98627 osd.63 up 1.00000 1.00000
73 nvme 6.98627 osd.73 up 1.00000 1.00000
84 nvme 6.98627 osd.84 up 1.00000 1.00000
94 nvme 6.98627 osd.94 up 1.00000 1.00000
104 nvme 6.98627 osd.104 up 1.00000 1.00000
114 nvme 6.98627 osd.114 up 1.00000 1.00000
124 nvme 6.98627 osd.124 up 1.00000 1.00000
138 nvme 6.98627 osd.138 up 1.00000 1.00000
148 nvme 6.98627 osd.148 up 1.00000 1.00000
158 nvme 6.98627 osd.158 up 1.00000 1.00000
170 nvme 6.98627 osd.170 up 1.00000 1.00000
180 nvme 6.98627 osd.180 up 1.00000 1.00000
190 nvme 6.98627 osd.190 up 1.00000 1.00000
200 nvme 6.98627 osd.200 up 1.00000 1.00000
210 nvme 6.98627 osd.210 up 1.00000 1.00000
216 nvme 6.98627 osd.216 up 1.00000 1.00000
-39 146.71161 host bd-ssd03-node09-4833dda0-f95f-4116-97fa-635ee8685ba2
15 nvme 6.98627 osd.15 up 1.00000 1.00000
25 nvme 6.98627 osd.25 up 1.00000 1.00000
35 nvme 6.98627 osd.35 up 1.00000 1.00000
47 nvme 6.98627 osd.47 up 1.00000 1.00000
61 nvme 6.98627 osd.61 up 1.00000 1.00000
72 nvme 6.98627 osd.72 up 1.00000 1.00000
83 nvme 6.98627 osd.83 up 1.00000 1.00000
92 nvme 6.98627 osd.92 up 1.00000 1.00000
102 nvme 6.98627 osd.102 up 1.00000 1.00000
112 nvme 6.98627 osd.112 up 1.00000 1.00000
122 nvme 6.98627 osd.122 up 1.00000 1.00000
133 nvme 6.98627 osd.133 up 1.00000 1.00000
144 nvme 6.98627 osd.144 up 1.00000 1.00000
154 nvme 6.98627 osd.154 up 1.00000 1.00000
164 nvme 6.98627 osd.164 up 1.00000 1.00000
174 nvme 6.98627 osd.174 up 1.00000 1.00000
184 nvme 6.98627 osd.184 up 1.00000 1.00000
194 nvme 6.98627 osd.194 up 1.00000 1.00000
204 nvme 6.98627 osd.204 up 1.00000 1.00000
213 nvme 6.98627 osd.213 up 1.00000 1.00000
218 nvme 6.98627 osd.218 up 1.00000 1.00000
-45 146.71161 host bd-ssd03-node10-9cfe177c-2e0f-43d3-a1a9-6524dac0009a
19 nvme 6.98627 osd.19 up 1.00000 1.00000
29 nvme 6.98627 osd.29 up 1.00000 1.00000
39 nvme 6.98627 osd.39 up 1.00000 1.00000
49 nvme 6.98627 osd.49 up 1.00000 1.00000
59 nvme 6.98627 osd.59 up 1.00000 1.00000
69 nvme 6.98627 osd.69 up 1.00000 1.00000
80 nvme 6.98627 osd.80 up 1.00000 1.00000
91 nvme 6.98627 osd.91 up 1.00000 1.00000
101 nvme 6.98627 osd.101 up 1.00000 1.00000
111 nvme 6.98627 osd.111 up 1.00000 1.00000
121 nvme 6.98627 osd.121 up 1.00000 1.00000
131 nvme 6.98627 osd.131 up 1.00000 1.00000
142 nvme 6.98627 osd.142 up 1.00000 1.00000
152 nvme 6.98627 osd.152 up 1.00000 1.00000
163 nvme 6.98627 osd.163 up 1.00000 1.00000
173 nvme 6.98627 osd.173 up 1.00000 1.00000
183 nvme 6.98627 osd.183 up 1.00000 1.00000
193 nvme 6.98627 osd.193 up 1.00000 1.00000
203 nvme 6.98627 osd.203 up 1.00000 1.00000
212 nvme 6.98627 osd.212 up 1.00000 1.00000
217 nvme 6.98627 osd.217 up 1.00000 1.00000
-4 14.55475 root bd-ssd03-index
-11 1.45547 host bd-ssd03-node01-0581ef65-8df6-4cfc-8992-08ffb7959914
41 nvme 1.45547 osd.41 up 1.00000 1.00000
-13 1.45547 host bd-ssd03-node02-0025ee25-77d9-4731-bc94-1c541e1fa2e7
126 nvme 1.45547 osd.126 up 1.00000 1.00000
-3 1.45547 host bd-ssd03-node03-a2629c55-9ddc-4c16-ba85-187af70eef23
160 nvme 1.45547 osd.160 up 1.00000 1.00000
-9 1.45547 host bd-ssd03-node04-9e02d968-28b5-43a8-b93a-418cb61af7ee
211 nvme 1.45547 osd.211 up 1.00000 1.00000
-15 1.45547 host bd-ssd03-node05-536f9f5e-cf15-4bb5-abaa-afd313473ab1
74 nvme 1.45547 osd.74 up 1.00000 1.00000
-23 1.45547 host bd-ssd03-node06-43949775-80b1-4297-b23f-d36ffb784d76
205 nvme 1.45547 osd.205 up 1.00000 1.00000
-21 1.45547 host bd-ssd03-node07-11ca9165-47b2-4b1d-84d7-c6b4f62f51eb
38 nvme 1.45547 osd.38 up 1.00000 1.00000
-7 1.45547 host bd-ssd03-node08-d6739f0c-a356-415c-ae4d-323cb4197efe
135 nvme 1.45547 osd.135 up 1.00000 1.00000
-17 1.45547 host bd-ssd03-node09-ece27fca-7711-45f1-b899-6d6f61aad2cf
57 nvme 1.45547 osd.57 up 1.00000 1.00000
-19 1.45547 host bd-ssd03-node10-c53d891c-b393-4663-88bf-f2ec4e9e0a9a
219 nvme 1.45547 osd.219 up 1.00000 1.00000
2. memory 512G
[root@bd-ssd03-node01 ~]# free -g
total used free shared buff/cache available
Mem: 503 105 134 0 262 393
Swap: 0 0 0
3. cpu 128
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 256
On-line CPU(s) list: 0-255
Thread(s) per core: 2
Core(s) per socket: 64
Socket(s): 2
NUMA node(s): 2
Vendor ID: AuthenticAMD
BIOS Vendor ID: Advanced Micro Devices, Inc.
CPU family: 25
Model: 1
Model name: AMD EPYC 7763 64-Core Processor
BIOS Model name: AMD EPYC 7763 64-Core Processor
Stepping: 1
CPU MHz: 3243.566
CPU max MHz: 3529.0520
CPU min MHz: 1500.0000
BogoMIPS: 4890.70
Virtualization: AMD-V
L1d cache: 32K
L1i cache: 32K
L2 cache: 512K
L3 cache: 32768K
4. Network 50Gb/s
Settings for bond4:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 50000Mb/s
Duplex: Full
Auto-negotiation: off
Port: Other
PHYAD: 0
Transceiver: internal
Link detected: yes
I'm using cosbench to test the rgw, the xml config of the cosbench is 10 driver to 10 rgw and every driver with 16 works:
<?xml version="1.0" encoding="UTF-8" ?>
<workload name="test" description="sample benchmark for s3">
<workflow>
<workstage name="create bucket" closuredelay="0" config="">
<auth type="none" config=""/>
<work name="rgw1-create" type="init" workers="1" interval="5"
division="container" totalOps="2560000" rampup="0" rampdown="0"
afr="0" totalBytes="0" config="cprefix=bdssd03-bucket;containers=r(1,10)">
<auth type="none" config=""/>
<storage type="s3" config="accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.143:80;path_style_access=true"/>
<operation type="init" ratio="100" division="container"
config="containers=r(1,1);objects=r(0,0);sizes=c(0)B" id="none"/>
</work>
</workstage>
<workstage name="write">
<work name="w1" workers="16" totalOps="2560000" driver="driver1">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(1);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.144:80;server_side_encryption=false; "/>
</work>
<work name="w2" workers="16" totalOps="2560000" driver="driver2">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(2);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.145:80;server_side_encryption=false;"; />
</work>
<work name="w3" workers="16" totalOps="2560000" driver="driver3">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(3);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.146:80;server_side_encryption=false;"; />
</work>
<work name="w4" workers="16" totalOps="2560000" driver="driver4">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(4);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.147:80;server_side_encryption=false; "/>
</work>
<work name="w5" workers="16" totalOps="2560000" driver="driver5">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(5);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.148:80;server_side_encryption=false;"; />
</work>
<work name="w6" workers="16" totalOps="2560000" driver="driver6">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(6);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.149:80;server_side_encryption=false;"; />
</work>
<work name="w7" workers="16" totalOps="2560000" driver="driver7">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(7);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.150:80;server_side_encryption=false; "/>
</work>
<work name="w8" workers="16" totalOps="2560000" driver="driver8">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(8);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.151:80;server_side_encryption=false;"; />
</work>
<work name="w9" workers="16" totalOps="2560000" driver="driver9">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(9);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.152:80;server_side_encryption=false;"; />
</work>
<work name="w10" workers="16" totalOps="2560000" driver="driver10">
<operation type="write" ratio="100" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=c(10);objects=r(1,2000000);sizes=c(4)MB;" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.143:80;server_side_encryption=false; "/>
</work>
</workstage>
<workstage name="cleanup">
<work type="cleanup" workers="16" config="cprefix=bdssd03-bucket;oprefix=test4m-;containers=r(1,10);objects=r(1,2000000)" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.143:80;server_side_encryption=false;"; />
</workstage>
<workstage name="dispose">
<work type="dispose" workers="10" config="cprefix=bdssd03-bucket;containers=r(1,10)" />
<storage type="s3" config="path_style_access=true;accesskey=GHWT013TX7ISVKR430Y0;secretkey=u7aCkNfznEztWpVJcARHceVC0ZPjR8XiVQDPmzAR;endpoint=http://10.x.x.143:80;server_side_encryption=false;"; />
</workstage>
</workflow>
</workload>
the performance result is:
cluster:
id: 075b70fa-6a72-4ce2-8956-d249c7cf10ce
health: HEALTH_OK
services:
mon: 3 daemons, quorum a,b,c (age 17h)
mgr: a(active, since 17h), standbys: b
osd: 220 osds: 220 up (since 17h), 220 in (since 17h)
rgw: 10 daemons active (10 hosts, 1 zones)
data:
pools: 9 pools, 4273 pgs
objects: 17.76M objects, 65 TiB
usage: 82 TiB used, 1.4 PiB / 1.4 PiB avail
pgs: 4273 active+clean
io:
client: 11 MiB/s rd, 9.2 GiB/s wr, 15.44k op/s rd, 28.70k op/s wr
only can reach to 9.2 GiB with 4MB write, I don't know that whether the performance meet expectations?
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
