On 10/06/2020 18:29, John Garry wrote: > From: Hannes Reinecke <hare@xxxxxxx> > > The smart array HBAs can steer interrupt completion, so this patch > switches the implementation to use multiqueue and enables > 'host_tagset' as the HBA has a shared host-wide tagset. > >>Hi Don, >>I am preparing the next iteration of this series, and >>we're getting close to dropping the RFC tags. The >>series has grown a bit, and I am not sure what to do >>with hpsa support. >>The latest versions of this series have not been tested for hpsa, AFAIK. >>Or someone else let me know it's tested (Hannes?) >>Thanks John, I cloned: https://github.com/hisilicon/kernel-dev switched to branch: origin/private-topic-blk-mq-shared-tags-rfc-v8 And built the kernel. The hpsa driver oopsed on load. It was attempting to do driver initiated commands, so there would need to be some reserved tags set aside to communicate with the controller. Was I supposed to add this patch on top of Hannes's hpsa patches? The patches in the indicated series seem to be included in the branch. ---- [ 13.340977] HP HPSA Driver (v 3.4.20-170) [ 13.374719] usbcore: registered new interface driver usb-storage [ 13.379626] hpsa 0000:0d:00.0: can't disable ASPM; OS doesn't have ASPM control [ 13.473790] scsi host0: scsi_eh_0: sleeping [ 13.475191] scsi host0: uas [ 13.487978] hpsa 0000:0d:00.0: Logical aborts not supported [ 13.498111] tg3 0000:02:00.0 eth0: Tigon3 [partno(N/A) rev 5719001] (PCI Express) MAC address d4:c9:ef:ce:0a:c4 [ 13.498113] tg3 0000:02:00.0 eth0: attached PHY is 5719C (10/100/1000Base-T Ethernet) (WireSpeed[1], EEE[1]) [ 13.498116] tg3 0000:02:00.0 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] TSOcap[1] [ 13.498117] tg3 0000:02:00.0 eth0: dma_rwctrl[00000001] dma_mask[64-bit] [ 13.499661] scsi host1: scsi_eh_1: sleeping [ 13.522611] tg3 0000:02:00.1 eth1: Tigon3 [partno(N/A) rev 5719001] (PCI Express) MAC address d4:c9:ef:ce:0a:c5 [ 13.541519] usbcore: registered new interface driver uas [ 13.549508] scsi 0:0:0:0: Direct-Access ASMT 2105 0 PQ: 0 ANSI: 6 [ 13.576119] tg3 0000:02:00.1 eth1: attached PHY is 5719C (10/100/1000Base-T Ethernet) (WireSpeed[1], EEE[1]) [ 14.046521] tg3 0000:02:00.1 eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] TSOcap[1] [ 14.046524] tg3 0000:02:00.1 eth1: dma_rwctrl[00000001] dma_mask[64-bit] [ 14.047114] BUG: kernel NULL pointer dereference, address: 0000000000000010 [ 14.193228] #PF: supervisor read access in kernel mode [ 14.193228] #PF: error_code(0x0000) - not-present page [ 14.193229] PGD 0 P4D 0 [ 14.193232] Oops: 0000 [#1] SMP PTI [ 14.193235] CPU: 0 PID: 495 Comm: kworker/0:8 Not tainted 5.8.0-rc1-host-tagset+ #3 [ 14.193236] Hardware name: HP ProLiant ML350 Gen9/ProLiant ML350 Gen9, BIOS P92 10/17/2018 [ 14.193243] Workqueue: events work_for_cpu_fn [ 14.470674] RIP: 0010:blk_mq_unique_tag+0x5/0x20 [ 14.470676] Code: cd 0f 1f 40 00 0f 1f 44 00 00 8b 87 cc 00 00 00 83 f8 02 75 03 83 06 01 b8 01 00 00 00 c3 0f 1f 80 00 00 00 00 0f 1f 44 00 00 <48> 8b 47 10 0f b7 57 20 8b 80 94 01 00 00 c1 e0 10 09 d0 c3 0f 1f [ 14.470677] RSP: 0000:ffff989f42893d08 EFLAGS: 00010246 [ 14.470680] RAX: ffffffffc0493f80 RBX: ffff8ab761c00000 RCX: 0000000000000000 [ 14.717021] RDX: ffff8ab9b7600000 RSI: ffff8ab761c00000 RDI: 0000000000000000 [ 14.717021] RBP: ffff8ab9a5b98000 R08: ffffffffffffffff R09: 0000000000000000 [ 14.717022] R10: ffff8ab8b5280070 R11: 0000000000000000 R12: 000000000000000a [ 14.717022] R13: 0000000000000002 R14: ffff8ab761c00000 R15: ffffffffc0493b60 [ 14.717023] FS: 0000000000000000(0000) GS:ffff8ab9b7600000(0000) knlGS:0000000000000000 [ 14.717024] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 14.717025] CR2: 0000000000000010 CR3: 000000024f33e006 CR4: 00000000001606f0 [ 14.717025] Call Trace: [ 14.717034] __enqueue_cmd_and_start_io.isra.60+0x20/0x170 [hpsa] [ 14.717039] hpsa_scsi_do_simple_cmd.isra.62+0x6b/0xd0 [hpsa] [ 14.717042] hpsa_scsi_do_simple_cmd_with_retry+0x63/0x160 [hpsa] [ 14.717045] hpsa_scsi_do_inquiry+0x62/0xc0 [hpsa] [ 14.717048] hpsa_init_one+0x1167/0x1400 [hpsa] [ 14.717052] local_pci_probe+0x42/0x80 [ 14.717054] work_for_cpu_fn+0x16/0x20 [ 14.717057] process_one_work+0x1a7/0x370 [ 14.717059] ? process_one_work+0x370/0x370 [ 14.717061] worker_thread+0x1c9/0x370 [ 14.717062] ? process_one_work+0x370/0x370 [ 14.717064] kthread+0x114/0x130 [ 14.717065] ? kthread_park+0x80/0x80 [ 14.717068] ret_from_fork+0x22/0x30 [ 14.717070] Modules linked in: crc32c_intel libahci(+) uas tg3(+) libata usb_storage i2c_algo_bit hpsa(+) scsi_transport_sas wmi dm_mirror dm_region_hash dm_log dm_mod [ 14.717077] CR2: 0000000000000010 [ 14.717099] ---[ end trace 3845f459e9223caa ]--- [ 14.724750] ERST: [Firmware Warn]: Firmware does not respond in time. [ 14.724753] RIP: 0010:blk_mq_unique_tag+0x5/0x20 [ 14.724754] Code: cd 0f 1f 40 00 0f 1f 44 00 00 8b 87 cc 00 00 00 83 f8 02 75 03 83 06 01 b8 01 00 00 00 c3 0f 1f 80 00 00 00 00 0f 1f 44 00 00 <48> 8b 47 10 0f b7 57 20 8b 80 94 01 00 00 c1 e0 10 09 d0 c3 0f 1f [ 14.724755] RSP: 0000:ffff989f42893d08 EFLAGS: 00010246 [ 14.724756] RAX: ffffffffc0493f80 RBX: ffff8ab761c00000 RCX: 0000000000000000 [ 14.724757] RDX: ffff8ab9b7600000 RSI: ffff8ab761c00000 RDI: 0000000000000000 [ 14.724757] RBP: ffff8ab9a5b98000 R08: ffffffffffffffff R09: 0000000000000000 [ 14.724758] R10: ffff8ab8b5280070 R11: 0000000000000000 R12: 000000000000000a [ 14.724758] R13: 0000000000000002 R14: ffff8ab761c00000 R15: ffffffffc0493b60 [ 14.724759] FS: 0000000000000000(0000) GS:ffff8ab9b7600000(0000) knlGS:0000000000000000 [ 14.724760] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 14.724760] CR2: 0000000000000010 CR3: 000000024f33e006 CR4: 00000000001606f0 [ 14.724761] Kernel panic - not syncing: Fatal exception [ 14.724833] Kernel Offset: 0x38400000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) [ 16.487017] ---[ end Kernel panic - not syncing: Fatal exception ]--- > Signed-off-by: Hannes Reinecke <hare@xxxxxxx> > --- > drivers/scsi/hpsa.c | 44 +++++++------------------------------------- > drivers/scsi/hpsa.h | 1 - > 2 files changed, 7 insertions(+), 38 deletions(-) > > diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c index > 1e9302e99d05..f807f9bdae85 100644 > --- a/drivers/scsi/hpsa.c > +++ b/drivers/scsi/hpsa.c > @@ -980,6 +980,7 @@ static struct scsi_host_template hpsa_driver_template = { > .shost_attrs = hpsa_shost_attrs, > .max_sectors = 2048, > .no_write_same = 1, > + .host_tagset = 1, > }; > > static inline u32 next_command(struct ctlr_info *h, u8 q) @@ > -1144,12 +1145,14 @@ static void dial_up_lockup_detection_on_fw_flash_complete(struct ctlr_info *h, > static void __enqueue_cmd_and_start_io(struct ctlr_info *h, > struct CommandList *c, int reply_queue) > { > + u32 blk_tag = blk_mq_unique_tag(c->scsi_cmd->request); > + > dial_down_lockup_detection_during_fw_flash(h, c); > atomic_inc(&h->commands_outstanding); > if (c->device) > atomic_inc(&c->device->commands_outstanding); > > - reply_queue = h->reply_map[raw_smp_processor_id()]; > + reply_queue = blk_mq_unique_tag_to_hwq(blk_tag); > switch (c->cmd_type) { > case CMD_IOACCEL1: > set_ioaccel1_performant_mode(h, c, reply_queue); @@ > -5653,8 +5656,6 @@ static int hpsa_scsi_queue_command(struct Scsi_Host *sh, struct scsi_cmnd *cmd) > /* Get the ptr to our adapter structure out of cmd->host. */ > h = sdev_to_hba(cmd->device); > > - BUG_ON(cmd->request->tag < 0); > - > dev = cmd->device->hostdata; > if (!dev) { > cmd->result = DID_NO_CONNECT << 16; @@ -5830,7 +5831,7 > @@ static int hpsa_scsi_host_alloc(struct ctlr_info *h) > sh->hostdata[0] = (unsigned long) h; > sh->irq = pci_irq_vector(h->pdev, 0); > sh->unique_id = sh->irq; > - > + sh->nr_hw_queues = h->msix_vectors > 0 ? h->msix_vectors : 1; > h->scsi_host = sh; > return 0; > } > @@ -5856,7 +5857,8 @@ static int hpsa_scsi_add_host(struct ctlr_info *h) > */ > static int hpsa_get_cmd_index(struct scsi_cmnd *scmd) > { > - int idx = scmd->request->tag; > + u32 blk_tag = blk_mq_unique_tag(scmd->request); > + int idx = blk_mq_unique_tag_to_tag(blk_tag); > > if (idx < 0) > return idx; > @@ -7456,26 +7458,6 @@ static void hpsa_disable_interrupt_mode(struct ctlr_info *h) > h->msix_vectors = 0; > } > > -static void hpsa_setup_reply_map(struct ctlr_info *h) -{ > - const struct cpumask *mask; > - unsigned int queue, cpu; > - > - for (queue = 0; queue < h->msix_vectors; queue++) { > - mask = pci_irq_get_affinity(h->pdev, queue); > - if (!mask) > - goto fallback; > - > - for_each_cpu(cpu, mask) > - h->reply_map[cpu] = queue; > - } > - return; > - > -fallback: > - for_each_possible_cpu(cpu) > - h->reply_map[cpu] = 0; > -} > - > /* If MSI/MSI-X is supported by the kernel we will try to enable it on > * controllers that are capable. If not, we use legacy INTx mode. > */ > @@ -7872,9 +7854,6 @@ static int hpsa_pci_init(struct ctlr_info *h) > if (err) > goto clean1; > > - /* setup mapping between CPU and reply queue */ > - hpsa_setup_reply_map(h); > - > err = hpsa_pci_find_memory_BAR(h->pdev, &h->paddr); > if (err) > goto clean2; /* intmode+region, pci */ > @@ -8613,7 +8592,6 @@ static struct workqueue_struct > *hpsa_create_controller_wq(struct ctlr_info *h, > > static void hpda_free_ctlr_info(struct ctlr_info *h) > { > - kfree(h->reply_map); > kfree(h); > } > > @@ -8622,14 +8600,6 @@ static struct ctlr_info *hpda_alloc_ctlr_info(void) > struct ctlr_info *h; > > h = kzalloc(sizeof(*h), GFP_KERNEL); > - if (!h) > - return NULL; > - > - h->reply_map = kcalloc(nr_cpu_ids, sizeof(*h->reply_map), GFP_KERNEL); > - if (!h->reply_map) { > - kfree(h); > - return NULL; > - } > return h; > } > > diff --git a/drivers/scsi/hpsa.h b/drivers/scsi/hpsa.h index > f8c88fc7b80a..ea4a609e3eb7 100644 > --- a/drivers/scsi/hpsa.h > +++ b/drivers/scsi/hpsa.h > @@ -161,7 +161,6 @@ struct bmic_controller_parameters { > #pragma pack() > > struct ctlr_info { > - unsigned int *reply_map; > int ctlr; > char devname[8]; > char *product_name; >
