On 04/14/2016 09:42 AM, Gavin Shan wrote:
On Wed, Apr 13, 2016 at 07:14:59PM +1000, Alexey Kardashevskiy wrote:
On 04/13/2016 05:42 PM, Gavin Shan wrote:
On Wed, Apr 13, 2016 at 05:28:15PM +1000, Alexey Kardashevskiy wrote:
On 02/17/2016 02:43 PM, Gavin Shan wrote:
This series of patches rebases on powerpc/next branch, plus below additional
patches:
<This series of patches>
<Followup 3 patches from Gavin on SRIOV EEH, which aren't posted>
https://patchwork.ozlabs.org/patch/581315/ (PATCH[1/9] Richard's SRIOV EEH)
https://patchwork.ozlabs.org/patch/582639/ (PATCH[1/1] Gavin's EEH fix)
https://patchwork.ozlabs.org/patch/582093/ (PATCH[1/1] Gavin's EEH fix)
https://patchwork.ozlabs.org/patch/580626/ (PATCH[1/4] Gavin's PCI fix)
https://patchwork.ozlabs.org/patch/580153/ (PATCH[1/1] Andrew's EEH minor fix)
https://patchwork.ozlabs.org/patch/566827/ (PATCH[1/1] Russell's P5IOC2 removal)
https://patchwork.ozlabs.org/patch/534154/ (PATCH[1/7] Richard's SRIOV rework)
commit 388f7b1 ("Linux 4.5-rc3")
The series of patches intend to support PCI slot for PowerPC PowerNV platform,
which is running on top of skiboot firmware. The patchset requires corresponding
changes from skiboot firmware, which is sent to skiboot@xxxxxxxxxxxxxxxx
for review. The PCI slots are exposed by skiboot with device node properties,
and kernel utilizes those properties to populated PCI slots accordingly.
The original PCI infrastructure on PowerNV platform can't support hotplug
because the PE is assigned during PHB fixup time, which is called for once
during system boot time. For this, the PCI infrastructure on PowerNV platform
has been reworked for a lot. After that, the PE and its corresponding resources
(IODT, M32DT, M64 segments, DMA32 and bypass window) are assigned upon updating
PCI bridge's resources, which might decide PE# assigned to the PE (e.g. M64
resources, on P8 strictly speaking). Each PE will maintain a reference count,
which is (number of child PCI devices + 1). That indicates when last child PCI
device leaves the PE, the PE and its included resources will be relased and put
back into free pool again. With this design, the PE will be released when EEH PE
is released. PATCH[1 - 23] are related to this part.
From skiboot perspective, PCI slot is providing (hot/fundamental/complete)
resets to EEH. The kernel gets to know if skiboot supports various reset on one
particular PCI slot through device-tree node. If it does, EEH will utilize the
functionality provided by skiboot. Besides, the device-tree nodes have to change
in order to support PCI hotplug. For example, when one PCI adapter inserted to
one slot, its device-tree node should be added to the system dynamically. Conversely,
the device-tree node should be removed from the system when the PCI adapter is going
to be offline. Since pci_dn and eeh_dev have same life cyle as PCI device nodes,
they should be added/removed accordingly during PCI hotplug. PATCH[24 - 39] are
doing the related work.
The OF driver is changed to support unflattening FDT blob for sub-stree, which
is covered by PATCH[40 - 44].
The last one, PATCH[45], is the standalone PCI hotplug driver for PowerPC PowerNV
platform.
=======
Testing
=======
1. Unplug adapters behind non-empty slot, then plug them.
1.1 Check status
# cat /sys/bus/pci/slots/C10/address
0003:09:00
# cat /sys/bus/pci/slots/C10/adapter
1
# cat /sys/bus/pci/slots/C10/power
1
# lspci
0003:09:00.0 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.1 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.2 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0003:09:00.3 Ethernet controller: \
Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
# lspci -t
# lspci -t
-+-[0003:00]---00.0-[01-13]----00.0-[02-13]--+-01.0-[03]----00.0
| +-08.0-[04-08]--
| +-09.0-[09]--+-00.0
| | +-00.1
| | +-00.2
| | \-00.3
| +-10.0-[0a-0e]--
| \-11.0-[0f-13]--
1.2 Unplug adapter 0003:09.00.x
# echo 0 > /sys/bus/pci/slots/C10/power
# lspci -t
-+-[0003:00]---00.0-[01-13]----00.0-[02-13]--+-01.0-[03]----00.0
| +-08.0-[04-08]--
| +-09.0-[09]--
| +-10.0-[0a-0e]--
| \-11.0-[0f-13]--
1.3 Plug adapter 0003:09.00.x
# echo 1 > /sys/bus/pci/slots/C10/power
Do I understand correctly that the adapter was not physically moved in/out of
the slot between 1.2 and 1.3?
Correct.
This is not right then... Someone should try it, on both P7 and P8.
Do you mean physically pull the adapter out and insert the same
adapter back? What's the point for the test case?
Because this is what the patchset is for - to replace a physical device on
a physical machine. Powering on/off the slots via sysfs is just an
approximation (which is fine when you are debugging), something can go
wrong and require some work but you do not know it for sure.
--
Alexey
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
