On 07/02/13 23:01, Brad Campbell wrote: > On 07/02/13 18:19, Adam Goryachev wrote: > >> problem. Is there some way to instruct the disk (overnight) to TRIM the >> extra blank space, and do whatever it needs to tidy things up? Perhaps >> this would help, at least first thing in the morning if it isn't enough >> to get through the day. Potentially I could add a 6th SSD, reduce the >> partition size across all of them, just so there is more blank space to >> get through a full day worth of writes? > > I have 6 SSD's in a RAID10, and with 3.7.x (I forget which x - 2 or 3 > from memory) md will pass the TRIM down to the underlying devices (at > least for RAID10 and from memory 1). Yes, I have read that the very new kernel has those patches, but I'm on 2.6.x at the moment, and in addition, see below why they wouldn't help anyway... > I have a cronjob that runs at midnight : > Based on the run times, and the bytes trimmed count I suspect it works. > All filesystems are ext4. Two of them are passed through encryption, but > that passes TRIM down also. I do not have the discard option on any > mounts (that way lies severe performance issues). I don't have any FS on this RAID, it is like this: 5 x SSD RAID5 (doesn't support TRIM, though I've seen some patches but I think they are not included in any kernel yet). DRBD (doubt this supports TRIM LVM (don't think it supports TRIM, maybe in newer kernel) iSCSI (don't think it support TRIM Windows 2003 and Windows 2000 (don't think it supports TRIM) So, really, all I want to do is use TRIM on the portion of the drive which is not partitioned at all, and I suspect the SSD knows that section is available, but how do I tell the drive "please go and do a cleanup now, because the users are all sleeping"? BTW, I just created a small LV (15G) and ran a couple of write tests (well, not proper one, but at least you get some idea how bad things are). dd if=/dev/zero of=/dev/vg0/testlv oflag=direct bs=16k count=50k ^C50695+0 records in 50695+0 records out 830586880 bytes (831 MB) copied, 99.4635 s, 8.4 MB/s I killed it after waiting a while.... this is while most of the systems are idle, except one which is currently being backed up (lots of reads, small number of writes). This is indicative of IO starvation though, I would have expected a significantly higher write performance? While I was running the dd, I ran a iostat -x 5 in another session: See text file attached for output, as seems to want to line wrap because it is too wide.... dm-13 is the client (windows 2003) which is currently being backed up, dm-14 is the testlv I'm writing to from the localhost. Suggestions on better testing methods or is this expected? Thanks, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 0.00 0.00 0.00 0.20 0.00 1.60 8.00 0.00 0.00 0.00 0.00 sdb 0.00 0.00 500.80 881.00 4648.20 7029.00 8.45 0.12 0.09 0.03 3.76 sdc 0.00 0.00 509.60 895.20 4676.20 7142.60 8.41 0.11 0.08 0.03 4.08 sdd 0.00 0.00 520.00 900.60 4753.80 7185.80 8.40 0.11 0.07 0.03 4.00 sde 0.00 0.00 528.80 922.80 4831.20 7363.40 8.40 0.16 0.11 0.04 5.20 sdf 0.60 0.00 515.00 906.60 4744.00 7233.80 8.43 0.14 0.10 0.04 5.04 md1 0.00 0.00 200.60 601.40 4659.80 17917.60 28.15 0.00 0.00 0.00 0.00 drbd2 0.00 0.00 200.60 599.80 4659.80 17916.00 28.21 1.15 1.45 1.25 99.84 dm-1 0.00 0.00 29.00 4.60 464.00 67.40 15.82 0.01 0.26 0.21 0.72 dm-5 0.00 0.00 0.20 2.80 1.60 13.40 5.00 0.01 2.40 1.60 0.48 dm-13 0.00 0.00 170.20 67.40 4184.60 1129.60 22.37 0.14 0.58 0.38 8.96 dm-21 0.00 0.00 0.00 0.60 0.00 8.00 13.33 0.00 0.00 0.00 0.00 dm-29 0.00 0.00 0.20 0.80 1.60 9.60 11.20 0.00 0.80 0.80 0.08 dm-37 0.00 0.00 0.20 0.00 1.60 0.00 8.00 0.00 0.00 0.00 0.00 dm-45 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-53 0.00 0.00 0.00 1.20 0.00 9.60 8.00 0.00 1.33 0.67 0.08 dm-61 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-65 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-71 0.00 0.00 0.20 0.80 1.60 11.20 12.80 0.00 1.60 1.60 0.16 dm-9 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-15 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-2 0.00 0.00 0.40 1.20 3.20 9.60 8.00 0.00 0.00 0.00 0.00 dm-11 0.00 0.00 0.40 2.80 3.20 13.40 5.19 0.01 2.00 1.25 0.40 dm-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-3 0.00 0.00 0.00 0.60 0.00 4.80 8.00 0.00 2.67 1.33 0.08 dm-4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-8 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-10 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-12 0.00 0.00 0.00 0.40 0.00 3.20 8.00 0.00 2.00 2.00 0.08 dm-14 0.00 0.00 0.00 520.60 0.00 16659.20 32.00 1.00 1.92 1.92 100.00
