On 03/06/2019 12:00, Ming Lei wrote:
On Fri, May 31, 2019 at 12:38:10PM +0100, John Garry wrote:
-fallback:
- for_each_possible_cpu(cpu)
- hisi_hba->reply_map[cpu] = cpu % hisi_hba->queue_count;
- /* Don't clean all CQ masks */
-}
-
static int interrupt_init_v3_hw(struct hisi_hba *hisi_hba)
{
struct device *dev = hisi_hba->dev;
@@ -2383,11 +2359,6 @@ static int interrupt_init_v3_hw(struct hisi_hba *hisi_hba)
min_msi = MIN_AFFINE_VECTORS_V3_HW;
- hisi_hba->reply_map = devm_kcalloc(dev, nr_cpu_ids,
- sizeof(unsigned int),
- GFP_KERNEL);
- if (!hisi_hba->reply_map)
- return -ENOMEM;
vectors = pci_alloc_irq_vectors_affinity(hisi_hba->pci_dev,
min_msi, max_msi,
PCI_IRQ_MSI |
@@ -2395,7 +2366,6 @@ static int interrupt_init_v3_hw(struct hisi_hba *hisi_hba)
&desc);
if (vectors < 0)
return -ENOENT;
- setup_reply_map_v3_hw(hisi_hba, vectors - BASE_VECTORS_V3_HW);
} else {
min_msi = max_msi;
vectors = pci_alloc_irq_vectors(hisi_hba->pci_dev, min_msi,
@@ -2896,6 +2866,18 @@ static void debugfs_snapshot_restore_v3_hw(struct hisi_hba *hisi_hba)
clear_bit(HISI_SAS_REJECT_CMD_BIT, &hisi_hba->flags);
}
+static int hisi_sas_map_queues(struct Scsi_Host *shost)
+{
+ struct hisi_hba *hisi_hba = shost_priv(shost);
+ struct blk_mq_queue_map *qmap = &shost->tag_set.map[HCTX_TYPE_DEFAULT];
+
+ if (auto_affine_msi_experimental)
+ return blk_mq_pci_map_queues(qmap, hisi_hba->pci_dev,
+ BASE_VECTORS_V3_HW);
+ else
+ return blk_mq_map_queues(qmap);
I don't think that the mapping which blk_mq_map_queues() creates are not
want we want. I'm guessing that we still would like a mapping similar to
what blk_mq_pci_map_queues() produces, which is an even spread, putting
adjacent CPUs on the same queue.
For my system with 96 cpus and 16 queues, blk_mq_map_queues() would map
queue 0 to cpu 0, 16, 32, 48 ..., queue 1 to cpu 1, 17, 33 and so on.
Hi Ming,
blk_mq_map_queues() is the default or fallback mapping in case that managed
irq isn't used. If the mapping isn't good enough, we still can improve it
in future, then any driver applying it can got improved.
That's the right attitude. However, as I see, we can only know the
mapping when we know the interrupt affinity or some other mapping
restriction or rule etc, which we don't know in this case.
For now, personally I would rather if we only expose multiple queues for
when auto_affine_msi_experimental is set. I fear that we may make a
performance regression for !auto_affine_msi_experimental with this
patch. We would need to test.
Hopefully we can drop !auto_affine_msi_experimental support when CPU
hotplug issue is resolved.
+}
+
static struct scsi_host_template sht_v3_hw = {
.name = DRV_NAME,
.module = THIS_MODULE,
As mentioned, we should be using a common function here.
@@ -2906,6 +2888,8 @@ static struct scsi_host_template sht_v3_hw = {
.scan_start = hisi_sas_scan_start,
.change_queue_depth = sas_change_queue_depth,
.bios_param = sas_bios_param,
+ .map_queues = hisi_sas_map_queues,
+ .host_tagset = 1,
.this_id = -1,
.sg_tablesize = HISI_SAS_SGE_PAGE_CNT,
.sg_prot_tablesize = HISI_SAS_SGE_PAGE_CNT,
@@ -3092,6 +3076,8 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (hisi_sas_debugfs_enable)
hisi_sas_debugfs_init(hisi_hba);
+ shost->nr_hw_queues = hisi_hba->cq_nvecs;
There's an ordering issue here, which can be fixed without too much trouble.
Value hisi_hba->cq_nvecs is not set until after this point, in
hisi_sas_v3_probe()->hw->hw_init->hisi_sas_v3_init()->interrupt_init_v3_hw()
Please see revised patch, below.
Good catch, will integrate it in V2.
Thanks!
Thanks,
Ming
.
