Could we infer from this if the usage model is large object sizes rather than small I/Os the benefit of offloading WAL/DB is questionable given that the failure of the SSD (assuming shared amongst HDDs) could take down a number of OSDs and in this case a best practice would be to collocate? -----Original Message----- From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Serkan Çoban Sent: Friday, April 27, 2018 10:05 AM To: Steven Vacaroaia <stef97@xxxxxxxxx> Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx> Subject: Re: ceph 12.2.5 - atop DB/WAL SSD usage 0% rados bench is using 4MB block size for io. Try with with io size 4KB, you will see ssd will be used for write operations. On Fri, Apr 27, 2018 at 4:54 PM, Steven Vacaroaia <stef97@xxxxxxxxx> wrote: > Hi > > During rados bench tests, I noticed that HDD usage goes to 100% but > SSD stays at ( or very close to 0) > > Since I created OSD with BLOCK/WAL on SSD, shouldnt I see some "activity' > on SSD ? > > How can I be sure CEPH is actually using SSD for WAL /DB ? > > > Note > I only have 2 HDD and one SSD per server for now > > > Comands used > > rados bench -p rbd 50 write -t 32 --no-cleanup && rados bench -p rbd > -t 32 > 50 rand > > > /usr/sbin/ceph-volume --cluster ceph lvm create --bluestore --data > /dev/sdc --block.wal > /dev/disk/by-partuuid/32ffde6f-7249-40b9-9bc5-2b70f0c3f7ad > --block.db /dev/disk/by-partuuid/2d9ab913-7553-46fc-8f96-5ffee028098a > > ( partitions are on SSD ...see below) > > sgdisk -p /dev/sda > Disk /dev/sda: 780140544 sectors, 372.0 GiB Logical sector size: 512 > bytes Disk identifier (GUID): 5FE0EA74-7E65-45B8-A356-62240333491E > Partition table holds up to 128 entries First usable sector is 34, > last usable sector is 780140510 Partitions will be aligned on > 2048-sector boundaries Total free space is 520093629 sectors (248.0 > GiB) > > Number Start (sector) End (sector) Size Code Name > 1 251660288 253757439 1024.0 MiB FFFF ceph WAL > 2 2048 62916607 30.0 GiB FFFF ceph DB > 3 253757440 255854591 1024.0 MiB FFFF ceph WAL > 4 62916608 125831167 30.0 GiB FFFF ceph DB > 5 255854592 257951743 1024.0 MiB FFFF ceph WAL > 6 125831168 188745727 30.0 GiB FFFF ceph DB > 7 257951744 260048895 1024.0 MiB FFFF ceph WAL > 8 188745728 251660287 30.0 GiB FFFF ceph DB > [root@osd04 ~]# ls -al /dev/disk/by-partuuid/ total 0 drwxr-xr-x 2 > root root 200 Apr 26 15:39 . > drwxr-xr-x 8 root root 160 Apr 27 08:45 .. > lrwxrwxrwx 1 root root 10 Apr 27 09:38 > 0baf986d-f786-4c1a-8962-834743b33e3a > -> ../../sda8 > lrwxrwxrwx 1 root root 10 Apr 27 09:38 > 2d9ab913-7553-46fc-8f96-5ffee028098a > -> ../../sda2 > lrwxrwxrwx 1 root root 10 Apr 27 09:38 > 32ffde6f-7249-40b9-9bc5-2b70f0c3f7ad > -> ../../sda3 > lrwxrwxrwx 1 root root 10 Apr 27 09:38 > 3f4e2d47-d553-4809-9d4e-06ba37b4c384 > -> ../../sda6 > lrwxrwxrwx 1 root root 10 Apr 27 09:38 > 3fc98512-a92e-4e3b-9de7-556b8e206786 > -> ../../sda1 > lrwxrwxrwx 1 root root 10 Apr 27 09:38 > 64b8ae66-cf37-4676-bf9f-9c4894788a7f > -> ../../sda7 > lrwxrwxrwx 1 root root 10 Apr 27 09:38 > 96254af9-7fe4-4ce0-886e-2e25356eff81 > -> ../../sda5 > lrwxrwxrwx 1 root root 10 Apr 27 09:38 > ae616b82-35ab-4f7f-9e6f-3c65326d76a8 > -> ../../sda4 > > > > > > > dm-0 | busy 90% | | read 2516 | write 0 | > | KiB/r 512 | KiB/w 0 | | MBr/s 125.8 | MBw/s 0.0 > | | avq 10.65 | avio 3.57 ms | | > LVM | dm-1 | busy 80% | | read 2406 | write > 0 | | KiB/r 512 | KiB/w 0 | | MBr/s > 120.3 | MBw/s 0.0 | | avq 12.59 | avio 3.30 ms | > | > DSK | sdc | busy 90% | | read 5044 | write > 0 | | KiB/r 256 | KiB/w 0 | | MBr/s > 126.1 | MBw/s 0.0 | | avq 19.53 | avio 1.78 ms | > | > DSK | sdd | busy 80% | | read 4805 | write > 0 | | KiB/r 256 | KiB/w 0 | | MBr/s > 120.1 | MBw/s 0.0 | | avq 23.97 | avio 1.65 ms | > | > DSK | sda | busy 0% | | read 0 | write > 7 | | KiB/r 0 | KiB/w 10 | | MBr/s > 0.0 | MBw/s 0.0 | | avq 0.00 | avio 0.00 ms | > | > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ceph.com_lis > tinfo.cgi_ceph-2Dusers-2Dceph.com&d=DwICAg&c=4DxX-JX0i28X6V65hK0ft5M-1 > rZQeWgdMry9v8-eNr4&r=eqMv5yFFe6-lAM9jJfUusNFzzcFAGwmoAez_acfPOtw&m=Gkb > AzUQpHU6F0PQW4cXglhdQN00DLmI75Ge2zPFqeeQ&s=R5UDTadunkDZPcYZfMoWS_0Vead > oXB5jfcy-FKfJYPM&e= > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ceph.com_listinfo.cgi_ceph-2Dusers-2Dceph.com&d=DwICAg&c=4DxX-JX0i28X6V65hK0ft5M-1rZQeWgdMry9v8-eNr4&r=eqMv5yFFe6-lAM9jJfUusNFzzcFAGwmoAez_acfPOtw&m=GkbAzUQpHU6F0PQW4cXglhdQN00DLmI75Ge2zPFqeeQ&s=R5UDTadunkDZPcYZfMoWS_0VeadoXB5jfcy-FKfJYPM&e= _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |