RE: caviar black + mpt2sas horrific performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 

On Thursday, March 31, 2011 9:35 AM, Louis-David Mitterrand wrote:
> On Thu, Mar 31, 2011 at 09:07:05AM -0600, Moore, Eric wrote:
> > On Wednesday, March 30, 2011 2:17 PM, Louis-David Mitterrand wrote:
> > > On Wed, Mar 30, 2011 at 02:02:12PM -0600, Moore, Eric wrote:
> > > > Hi Louis, can you send me your /var/log/messages and dmesg output?
> > > > Please capture after you've run your test.
> > >
> > > Hi,
> > >
> > > My test produces no specific output through dmesg or /var/log/messages
> > > as there is no apparent error from the controller.
> > >
> > > However I can send you the syslog startup output. Let me know if I can
> > > run any further tests or provide more information.
> > >
> >
> > need you to set the mpt2sas logging level to 0x310 just before you run
> you test, then capture the logs after.
> > Please make sure your syslog level is set to capture all KERN_XXX levels
> to include KERN_DEBUG.
> >
> > Example:
> > # echo 0x310 > /sys/module/mpt2sas/parameters/logging_level
> 
> Hi,
> 
> Here is the output:
> 
> Mar 31 17:07:50 zenon kernel: setting logging_level(0x00000310)
> Mar 31 17:24:29 zenon kernel: sd 0:0:0:0: [sda] CDB: ATA command pass
> through(16): 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
> Mar 31 17:24:29 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221103000000), phy(3)
> Mar 31 17:24:29 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(0)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	handle(0x0009),
> ioc_status(success)(0x0000), smid(1344)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:29 zenon kernel: sd 0:0:0:0: [sda] CDB: ATA command pass
> through(16): 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
> Mar 31 17:24:29 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221103000000), phy(3)
> Mar 31 17:24:29 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(0)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	handle(0x0009),
> ioc_status(success)(0x0000), smid(884)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:29 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:30 zenon kernel: sd 0:0:1:0: [sdb] CDB: ATA command pass
> through(16): 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
> Mar 31 17:24:30 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221102000000), phy(2)
> Mar 31 17:24:30 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(1)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	handle(0x000a),
> ioc_status(success)(0x0000), smid(100)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:30 zenon kernel: sd 0:0:1:0: [sdb] CDB: ATA command pass
> through(16): 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
> Mar 31 17:24:30 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221102000000), phy(2)
> Mar 31 17:24:30 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(1)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	handle(0x000a),
> ioc_status(success)(0x0000), smid(510)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:30 zenon kernel: sd 0:0:2:0: [sdc] CDB: ATA command pass
> through(16): 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
> Mar 31 17:24:30 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221101000000), phy(1)
> Mar 31 17:24:30 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(2)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	handle(0x000b),
> ioc_status(success)(0x0000), smid(1455)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:30 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:31 zenon kernel: sd 0:0:2:0: [sdc] CDB: ATA command pass
> through(16): 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
> Mar 31 17:24:31 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221101000000), phy(1)
> Mar 31 17:24:31 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(2)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	handle(0x000b),
> ioc_status(success)(0x0000), smid(966)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:31 zenon kernel: sd 0:0:3:0: [sdd] CDB: ATA command pass
> through(16): 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
> Mar 31 17:24:31 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221100000000), phy(0)
> Mar 31 17:24:31 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(3)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	handle(0x000c),
> ioc_status(success)(0x0000), smid(519)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:31 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:32 zenon kernel: sd 0:0:3:0: [sdd] CDB: ATA command pass
> through(16): 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
> Mar 31 17:24:32 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221100000000), phy(0)
> Mar 31 17:24:32 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(3)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	handle(0x000c),
> ioc_status(success)(0x0000), smid(1125)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:32 zenon kernel: sd 0:0:4:0: [sde] CDB: ATA command pass
> through(16): 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
> Mar 31 17:24:32 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221104000000), phy(4)
> Mar 31 17:24:32 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(7)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	handle(0x000d),
> ioc_status(success)(0x0000), smid(570)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:32 zenon kernel: sd 0:0:4:0: [sde] CDB: ATA command pass
> through(16): 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
> Mar 31 17:24:32 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221104000000), phy(4)
> Mar 31 17:24:32 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(7)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	handle(0x000d),
> ioc_status(success)(0x0000), smid(28)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:32 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:33 zenon kernel: sd 0:0:5:0: [sdf] CDB: ATA command pass
> through(16): 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
> Mar 31 17:24:33 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221107000000), phy(7)
> Mar 31 17:24:33 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(4)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	handle(0x000e),
> ioc_status(success)(0x0000), smid(471)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:33 zenon kernel: sd 0:0:5:0: [sdf] CDB: ATA command pass
> through(16): 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
> Mar 31 17:24:33 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221107000000), phy(7)
> Mar 31 17:24:33 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(4)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	handle(0x000e),
> ioc_status(success)(0x0000), smid(789)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:33 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:34 zenon kernel: sd 0:0:6:0: [sdg] CDB: ATA command pass
> through(16): 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
> Mar 31 17:24:34 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221106000000), phy(6)
> Mar 31 17:24:34 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(5)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	handle(0x000f),
> ioc_status(success)(0x0000), smid(594)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:34 zenon kernel: sd 0:0:6:0: [sdg] CDB: ATA command pass
> through(16): 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
> Mar 31 17:24:34 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221106000000), phy(6)
> Mar 31 17:24:34 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(5)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	handle(0x000f),
> ioc_status(success)(0x0000), smid(761)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:34 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:35 zenon kernel: sd 0:0:7:0: [sdh] CDB: ATA command pass
> through(16): 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
> Mar 31 17:24:35 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221105000000), phy(5)
> Mar 31 17:24:35 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(6)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	handle(0x0010),
> ioc_status(success)(0x0000), smid(1127)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)
> Mar 31 17:24:35 zenon kernel: sd 0:0:7:0: [sdh] CDB: ATA command pass
> through(16): 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
> Mar 31 17:24:35 zenon kernel: mpt2sas0:
> 	sas_address(0x4433221105000000), phy(5)
> Mar 31 17:24:35 zenon kernel: mpt2sas0:
> 	enclosure_logical_id(0x5842b2b05020c600), slot(6)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	handle(0x0010),
> ioc_status(success)(0x0000), smid(1583)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	request_len(0), underflow(0),
> resid(0)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	tag(0), transfer_count(0), sc-
> >result(0x00000002)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	scsi_status(check
> condition)(0x02), scsi_state(autosense valid )(0x01)
> Mar 31 17:24:35 zenon kernel: mpt2sas0: 	[sense_key,asc,ascq]:
> [0x01,0x00,0x1d], count(22)


Well you've got a load of SATA passthru request with RECOVERED_ERROR sense key is probably effecting your performance.

Any chance you could try shutdown the daemons that are sending those request?   I doubt its coming via the filesystem and the test your running.


Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [SCSI Target Devel]     [Linux SCSI Target Infrastructure]     [Kernel Newbies]     [IDE]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux ATA RAID]     [Linux IIO]     [Samba]     [Device Mapper]
  Powered by Linux