In 2009, While running "cache read" performance test of drives behind
SII PMP we encountered a "all 5 drives" timeout on more than 30% of the
machines under test. This patch reduces the rate by a factor of about 70.
Low enough that we didn't care to further investigate the issue.
Performance impact with any sort of "normal" use was ~2%+ CPU and less
than 1% throughput degradation. Worst case impact (cached read) was
6% IOPS reduction. This is with NCQ off (q=1) but I believe FIS based
switching enabled in the SATA driver.
The patch disables "Early ACK" in the 3726/4726 port multiplier.
"Early ACK" is issued when device sends a FIS to the host (via PMP)
and the PMP sends an ACK immediately back to the device - well before
the host gets the response. Under worst case IOPs load (cached read
test) and more than 2 PMPs connected to a 4-port SATA controller,
I suspect the time to service all of the PMPs is exceeding the PMPs
ability to keep track of outstanding FIS it owes the Host. Reducing
the number of PMPs to 2 (or 1) reduces the frequency by several orders
of magnitude. Kudos to Gwendal for initial debugging of this issue.
[Any errors in the description are mine, not his.]
Patch is currently in production on Google servers.
Signed-off-by: Grant Grundler <grundler@xxxxxxxxxx>
Signed-off-by: Gwendal Grignou <gwendal@xxxxxxxxxx>
---
And my apologies in advance for using gmail. Patch appended below
is also attached since I expect gmail will mangle the inlined content.
diff --git a/drivers/ata/libata-pmp.c b/drivers/ata/libata-pmp.c
index 00305f4..69bb203 100644
--- a/drivers/ata/libata-pmp.c
+++ b/drivers/ata/libata-pmp.c
@@ -235,6 +235,8 @@ static int sata_pmp_configure(struct ata_device
*dev, int print_info)
{
struct ata_port *ap = dev->link->ap;
u32 *gscr = dev->gscr;
+ u16 vendor = sata_pmp_gscr_vendor(gscr);
+ u16 devid = sata_pmp_gscr_devid(gscr);
unsigned int err_mask = 0;
const char *reason;
int nr_ports, rc;
@@ -260,12 +262,28 @@ static int sata_pmp_configure(struct ata_device
*dev, int print_info)
goto fail;
}
+ /* For Sil3726, disable sending Early R_OK */
+ if (vendor == 0x1095 && devid == 0x3726) {
+ u32 reg;
+ err_mask = sata_pmp_read(&ap->link,
SATA_PMP_GSCR_SII_POL, ®);
+ if (err_mask) {
+ rc = -EIO;
+ reason = "failed to read Sil3726 Private Register";
+ goto fail;
+ }
+ reg &= ~0x1;
+ err_mask = sata_pmp_write(&ap->link,
SATA_PMP_GSCR_SII_POL, reg);
+ if (err_mask) {
+ rc = -EIO;
+ reason = "failed to write Sil3726 Private Register";
+ goto fail;
+ }
+ }
+
if (print_info) {
ata_dev_printk(dev, KERN_INFO, "Port Multiplier %s, "
"0x%04x:0x%04x r%d, %d ports, feat 0x%x/0x%x\n",
- sata_pmp_spec_rev_str(gscr),
- sata_pmp_gscr_vendor(gscr),
- sata_pmp_gscr_devid(gscr),
+ sata_pmp_spec_rev_str(gscr), vendor, devid,
sata_pmp_gscr_rev(gscr),
nr_ports, gscr[SATA_PMP_GSCR_FEAT_EN],
gscr[SATA_PMP_GSCR_FEAT]);
diff --git a/include/linux/ata.h b/include/linux/ata.h
index 700c5b9..18fc208 100644
--- a/include/linux/ata.h
+++ b/include/linux/ata.h
@@ -386,6 +386,7 @@ enum {
SATA_PMP_GSCR_ERROR_EN = 33,
SATA_PMP_GSCR_FEAT = 64,
SATA_PMP_GSCR_FEAT_EN = 96,
+ SATA_PMP_GSCR_SII_POL = 129, /* VENDOR : Silicon Image 3726/4726 */
SATA_PMP_PSCR_STATUS = 0,
SATA_PMP_PSCR_ERROR = 1,
Attachment:
2.6.18-mv_sata-measure_port_handler-03
Description: Binary data
