Hello everybody,
I thought I would share the benchmarks from these four ssd's I tested
(see attachment)
I do still have some question:
#1 * Data Set Management TRIM supported (limit 1 block)
vs
* Data Set Management TRIM supported (limit 8 blocks)
and how this effects Ceph and also how can I test if TRIM is actually
working and not corruption data.
#2 are there other things I should test to compare ssd's for Ceph Journals
#3 are the power loss security mechanisms on SSD relevant in Ceph when
configured in a way that a full node can fully die and that a power loss
of all nodes at the same time should not be possible (or has an extreme
low probability)
#4 how to benchmarks the OSD (disk+ssd-journal) combination so I can
compare them.
I got some other benchmarks question, but I will make an separate mail
for them.
Kind regards,
Jelle de Jong
#-----------------------------------------------------------------------
# Plextor M6 Pro 128G
root@ceph01:~# uname -a
Linux ceph01 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux
root@ceph01:~# smartctl -i /dev/sdc
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-3.16.0-4-amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Device Model: PLEXTOR PX-128M6Pro
Serial Number: P02441106228
LU WWN Device Id: 5 002303 1002de43e
Add. Product Id: NC702090
Firmware Version: 1.02
User Capacity: 128,035,676,160 bytes [128 GB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ATA8-ACS, ATA/ATAPI-7 T13/1532D revision 4a
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Thu Jun 18 15:46:33 2015 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
root@ceph01:~# hdparm -I /dev/sdc | grep TRIM
* Data Set Management TRIM supported (limit 8 blocks)
root@ceph01:~# hdparm -W 0 /dev/sdc 0
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=2 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=4 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=8 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=16 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=32 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=64 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
1# write: io=163136KB, bw=2718.1KB/s, iops=679, runt= 60001msec
2# write: io=323768KB, bw=5396.5KB/s, iops=1349, runt= 60001msec
4# write: io=643624KB, bw=10727KB/s, iops=2681, runt= 60001msec
8# write: io=1238.3MB, bw=21132KB/s, iops=5283, runt= 60002msec
16# write: io=2218.9MB, bw=37868KB/s, iops=9466, runt= 60001msec
32# write: io=3342.7MB, bw=57045KB/s, iops=14261, runt= 60003msec
64# write: io=3149.6MB, bw=53745KB/s, iops=13436, runt= 60007msec
# second run after testing the other ssd's
1# write: io=162100KB, bw=2701.7KB/s, iops=675, runt= 60001msec
2# write: io=321076KB, bw=5351.2KB/s, iops=1337, runt= 60001msec
4# write: io=641076KB, bw=10684KB/s, iops=2671, runt= 60001msec
8# write: io=1230.5MB, bw=20999KB/s, iops=5249, runt= 60002msec
16# write: io=2199.9MB, bw=37543KB/s, iops=9385, runt= 60002msec
32# write: io=3367.4MB, bw=57467KB/s, iops=14366, runt= 60002msec
64# write: io=3270.5MB, bw=55809KB/s, iops=13952, runt= 60006msec
root@ceph01:~# dd if=/dev/zero of=/dev/sdc bs=4k count=10000 oflag=direct,dsync
10000+0 records in
10000+0 records out
40960000 bytes (41 MB) copied, 14.6745 s, 2.8 MB/s
root@ceph01:~# dmidecode -t system
# dmidecode 2.12
SMBIOS 2.6 present.
Handle 0x0002, DMI type 1, 27 bytes
System Information
Manufacturer: Hewlett-Packard
Product Name: HP Z600 Workstation
Version:
Serial Number: CZC0121R1J
UUID: CD0720D9-378D-11DF-BBDA-05C40AB118A9
Wake-up Type: Power Switch
SKU Number: FW863AV
Family: 103C_53335X
Handle 0x004B, DMI type 32, 11 bytes
System Boot Information
Status: No errors detected
#-----------------------------------------------------------------------
# Kingston HyperX Fury 120G
root@ceph01:~# uname -a
Linux ceph01 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux
root@ceph01:~# smartctl -i /dev/sdc
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-3.16.0-4-amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Device Model: KINGSTON SHFS37A120G
Serial Number: 50026B72540A8EC0
LU WWN Device Id: 5 0026b7 2540a8ec0
Firmware Version: 603ABBF0
User Capacity: 120,034,123,776 bytes [120 GB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ATA8-ACS, ACS-2 T13/2015-D revision 3
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Thu Jun 18 16:17:57 2015 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
root@ceph01:~# hdparm -I /dev/sdc | grep TRIM
* Data Set Management TRIM supported (limit 1 block)
root@ceph01:~# hdparm -W 0 /dev/sdc 0
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=2 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=4 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=8 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=16 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=32 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=64 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
1# write: io=5885.4MB, bw=100444KB/s, iops=25111, runt= 60000msec
2# write: io=8443.3MB, bw=144095KB/s, iops=36023, runt= 60001msec
4# write: io=10237MB, bw=174705KB/s, iops=43676, runt= 60001msec
8# write: io=11628MB, bw=198454KB/s, iops=49613, runt= 60001msec
16# write: io=11750MB, bw=200530KB/s, iops=50132, runt= 60001msec
32# write: io=11755MB, bw=200613KB/s, iops=50153, runt= 60003msec
64# write: io=11858MB, bw=202364KB/s, iops=50591, runt= 60002msec
root@ceph01:~# dd if=/dev/zero of=/dev/sdc bs=4k count=10000 oflag=direct,dsync
10000+0 records in
10000+0 records out
40960000 bytes (41 MB) copied, 0.423074 s, 96.8 MB/s
root@ceph01:~# dd if=/dev/zero of=/dev/sdc bs=4M count=1000 oflag=direct
1000+0 records in
1000+0 records out
4194304000 bytes (4.2 GB) copied, 18.1823 s, 231 MB/s
root@ceph01:~# dmidecode -t system
# dmidecode 2.12
SMBIOS 2.6 present.
Handle 0x0002, DMI type 1, 27 bytes
System Information
Manufacturer: Hewlett-Packard
Product Name: HP Z600 Workstation
Version:
Serial Number: CZC0121R1J
UUID: CD0720D9-378D-11DF-BBDA-05C40AB118A9
Wake-up Type: Power Switch
SKU Number: FW863AV
Family: 103C_53335X
Handle 0x004B, DMI type 32, 11 bytes
System Boot Information
Status: No errors detected
#-----------------------------------------------------------------------
# Kingston SSDNow V300 120G
root@ceph01:~# uname -a
Linux ceph01 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux
root@ceph01:~# smartctl -i /dev/sdc
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-3.16.0-4-amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: SandForce Driven SSDs
Device Model: KINGSTON SV300S37A120G
Serial Number: 50026B7854014BEC
LU WWN Device Id: 5 0026b7 854014bec
Firmware Version: 603ABBF0
User Capacity: 120,034,123,776 bytes [120 GB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA8-ACS, ACS-2 T13/2015-D revision 3
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Thu Jun 18 17:04:17 2015 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
root@ceph01:~# hdparm -I /dev/sdc | grep TRIM
* Data Set Management TRIM supported (limit 1 block)
root@ceph01:~# hdparm -W 0 /dev/sdc 0
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=2 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=4 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=8 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=16 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=32 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=64 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
1# write: io=5761.7MB, bw=98320KB/s, iops=24580, runt= 60001msec
2# write: io=8096.9MB, bw=138184KB/s, iops=34546, runt= 60001msec
4# write: io=10623MB, bw=181298KB/s, iops=45324, runt= 60001msec
8# write: io=11608MB, bw=198102KB/s, iops=49525, runt= 60001msec
16# write: io=11735MB, bw=200277KB/s, iops=50069, runt= 60001msec
32# write: io=11772MB, bw=200897KB/s, iops=50224, runt= 60001msec
64# write: io=11865MB, bw=202487KB/s, iops=50621, runt= 60003msec
root@ceph01:~# dd if=/dev/zero of=/dev/sdc bs=4k count=10000 oflag=direct,dsync
10000+0 records in
10000+0 records out
40960000 bytes (41 MB) copied, 0.424145 s, 96.6 MB/s
root@ceph01:~# dd if=/dev/zero of=/dev/sdc bs=4M count=1000 oflag=direct
1000+0 records in
1000+0 records out
4194304000 bytes (4.2 GB) copied, 17.9363 s, 234 MB/s
root@ceph01:~# dmidecode -t system
# dmidecode 2.12
SMBIOS 2.6 present.
Handle 0x0002, DMI type 1, 27 bytes
System Information
Manufacturer: Hewlett-Packard
Product Name: HP Z600 Workstation
Version:
Serial Number: CZC0121R1J
UUID: CD0720D9-378D-11DF-BBDA-05C40AB118A9
Wake-up Type: Power Switch
SKU Number: FW863AV
Family: 103C_53335X
Handle 0x004B, DMI type 32, 11 bytes
System Boot Information
Status: No errors detected
#-----------------------------------------------------------------------
# ADATA SP900 64G (returned these ssd's)
root@ceph01:~# uname -a
Linux ceph01 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux
root@ceph01:~# smartctl -i /dev/sdd
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-3.16.0-4-amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: SandForce Driven SSDs
Device Model: ADATA SP900
Serial Number: 2F1620041997
LU WWN Device Id: 5 707c18 1001507d8
Firmware Version: 5.0.Be
User Capacity: 64,023,257,088 bytes [64.0 GB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA8-ACS, ACS-2 T13/2015-D revision 3
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Wed Jun 17 14:23:25 2015 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
root@ceph01:~# hdparm -W 0 /dev/sdd 0
fio --filename=/dev/sdd --direct=1 --sync=1 --rw=write --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdd --direct=1 --sync=1 --rw=write --bs=4k --numjobs=2 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdd --direct=1 --sync=1 --rw=write --bs=4k --numjobs=4 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdd --direct=1 --sync=1 --rw=write --bs=4k --numjobs=8 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdd --direct=1 --sync=1 --rw=write --bs=4k --numjobs=16 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
fio --filename=/dev/sdd --direct=1 --sync=1 --rw=write --bs=4k --numjobs=32 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
1# write: io=23480KB, bw=400671B/s, iops=97, runt= 60008msec
2# write: io=46940KB, bw=800975B/s, iops=195, runt= 60010msec
4# write: io=46952KB, bw=801167B/s, iops=195, runt= 60011msec
8# write: io=46996KB, bw=801637B/s, iops=195, runt= 60032msec
16# write: io=47296KB, bw=806150B/s, iops=196, runt= 60077msec
32# write: io=48464KB, bw=824905B/s, iops=201, runt= 60161msec
root@ceph01:~# dmidecode -t system
# dmidecode 2.12
SMBIOS 2.6 present.
Handle 0x0002, DMI type 1, 27 bytes
System Information
Manufacturer: Hewlett-Packard
Product Name: HP Z600 Workstation
Version:
Serial Number: CZC0121R1J
UUID: CD0720D9-378D-11DF-BBDA-05C40AB118A9
Wake-up Type: Power Switch
SKU Number: FW863AV
Family: 103C_53335X
Handle 0x004B, DMI type 32, 11 bytes
System Boot Information
Status: No errors detected
#-----------------------------------------------------------------------
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
