Hi,
See my reply below.
On Thu, 2021-12-23 at 06:03 +0000, Nava kishore Manne wrote:
> Hi ,
> Please find my response inline.
>
> > > -----Original Message-----
> > > From: Simon Doppler <simon.doppler@xxxxxxxxxxxxx>
> > > Sent: Wednesday, December 22, 2021 2:14 PM
> > > To: mdf@xxxxxxxxxx; trix@xxxxxxxxxx; Michal Simek
> > > <michals@xxxxxxxxxx>; linux-fpga@xxxxxxxxxxxxxxx
> > > Cc: Adrian Fiergolski <adrian.fiergolski@xxxxxxxxxxxxx>
> > > Subject: Partial reconfiguration on ZynqMP
> > >
> > > Hi everyone,
> > >
> > > We are currently trying to use partial reconfiguration for our
> > > ZynqMP-
> > > based system (part is xczu6eg) and device-tree overlay based
> > > reconfiguration crashes.
> > > Our Linux system is using 5.10.0-xilinx-v2021.1.
> > >
> > > The reconfigurable partition is decoupled from the rest of the
> > > design
> > > with an AXI-shutdown manager IP and a DFX decoupler (the
> > > shutdown_requested signal of the AXI shutdown manager is used to
> > > drive
> > > the request_shutdown of the DFX decoupler).
> > >
> > > The relevant parts of the device tree are:
> > >
> > > ---- BEGIN SNIP ----
> > > fpga_full: fpga-full {
> > > status = "okay";
> > > reconf_region: reconf_region0 {
> > > compatible = "fpga-region";
> > > fpga-bridges = <&reconf_bridge0>;
> > > };
> > > };
> > >
> > > reconf_bridge0: reconf_bridge@a0080000 {
> > > status = "okay";
> > > compatible = "xlnx,dfx-axi-shutdown-manager-
> > 1.00";
> > > reg = <0x0 0xa0080000 0x0 0x10>;
> > > clock-names = "aclk";
> > > clocks = <&zynqmp_clk 73>;
> > > bridge-enable = <1>;
> > > };
> > > ---- END SNIP ----
> > >
> > > To reconfigure, I tested the following cases:
> > >
> > > 1. manual reconfiguration: WORKS
> > > - decouple by writing to the register directly
> > > - reconfigure directly with the fpga-manager
> > > - recouple by writing directly to the register
> > >
> > > 2. full reconfiguration with device-tree overlay: WORKS
> > > - we feed the full bitstream to the FPGA manager
> > > - using the following device-tree overlay:
> > > ---- BEGIN SNIP ----
> > > /dts-v1/;
> > > /plugin/;
> > >
> > > &reconf_region {
> > > firmware-name = "full.bit";
> > > };
> > > ---- BEGIN SNIP ----
> > >
> > > 3. partial reconfiguration with device-tree overlay:
> > > - we feed the full bitstream to the FPGA manager
> > > - using the following device-tree overlay:
> > > ---- BEGIN SNIP ----
> > > /dts-v1/;
> > > /plugin/;
> > >
> > > &reconf_region {
> > > partial-fpga-config;
> > > fpga-bridges = <&reconf_bridge0>;
> > > firmware-name = "partial.bit";
> > > };
> > > ---- BEGIN SNIP ----
> > >
> > > In this case, the whole system crashes (even the platform cable
> > > does
> > > not respond anymore).
> > > When setting an ILA trigger on the write valid signal of the AXI
> > > control path of the AXI Shutdown manager, the ILA appears to
> > > trigger
> > > but crashes before the data is shown in the hardware manager
> > > window of
> > Vivado.
> > > When the partial bitstream written by the overlay is already
> > > loaded
> > > (by hand, as is case 1), the reconfiguration using an overlay
> > > works
> > > and the AXI Shutdown manager is written to as expected.
> > >
> > > Do you have an idea on what is going wrong here ?
> > > As I manage to do the reconfiguration by hand, I suspect a
> > > configuration/device-tree/software error.
> > >
>
> In the case 2 use case could you please provide more details (if
> possible please share the log)
> needs to cross verify whether the axi shutdown manager is
> written/configured as expected.
>
In case 2, the bridge is not triggered as I replace the full bitstream
including the bridge. The relevant parts of the log are below:
U-Boot SPL 2021.01 (Oct 12 2021 - 09:28:42 +0000)
[....]
U-Boot 2021.01 (Oct 12 2021 - 09:28:42 +0000)
[....]
Scanning mmc 1:1...
Found U-Boot script /boot.scr
509 bytes read in 6 ms (82 KiB/s)
## Executing script at 20000000
26510890 bytes read in 1759 ms (14.4 MiB/s)
design filename = "Falcon;UserID=0XFFFFFFFF;Version=2021.1"
part number = "xczu6eg-ffvc900-1-i"
date = "2021/12/20"
time = "16:29:50"
bytes in bitstream = 26510780
zynqmp_align_dma_buffer: Align buffer at 000000000010006e to
00000000000fff80(swap 0)
49559 bytes read in 16 ms (3 MiB/s)
21019136 bytes read in 1393 ms (14.4 MiB/s)
## Flattened Device Tree blob at 00100000
Booting using the fdt blob at 0x100000
Loading Device Tree to 000000007de03000, end 000000007de12196 ... OK
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[ 0.000000] Linux version 5.10.0-xilinx-v2021.1 (oe-user@oe-host)
(aarch64-poky-linux-gcc (GCC) 10.2.0, GNU ld (GNU Binutils)
2.36.1.20210209) #1 SMP Tue Oct 12 09:30:57 UTC 2021
[....]
[ 0.000000] Kernel command line: video=DP-1:1920x1080 xilinx-
video.is_mplane=0 root=/dev/mmcblk1p2 rw rootwait earlycon
clk_ignore_unused
[....]
[ 3.217384] zynqmp-ipi-mbox mailbox@ff990400: Registered ZynqMP IPI
mbox with TX/RX channels.
[ 3.217581] FPGA manager framework
[....]
[ 3.439816] fpga_manager fpga0: Xilinx ZynqMP FPGA Manager
registered
[ 3.440223] xlnx_pr_decoupler a0080000.user_proc_bridge: fpga bridge
[Xilinx DFX AXI shutdown mgr] registered
[....]
falcon login: root
root@falcon:~# mkdir /sys/kernel/config/device-tree/overlays/user
partialBitstream-average-fwonly.dtbo partialBitstream-average-
nobridge.dtbo partialBitstream-average-
nofw.dtbo partialBitstream-average-nopartial.dtbo
root@falcon:~# cat full.dtbo > /sys/kernel/config/device-
tree/overlays/user/dtbo
[ 92.045900] fpga_manager fpga0: writing partialBitstream-average.bit
to Xilinx ZynqMP FPGA Manager
[ 104.099858] OF: overlay: WARNING: memory leak will occur if overlay
removed, property: /fpga-full/reconf_region0/fpga-bridges
[ 104.111882] OF: overlay: WARNING: memory leak will occur if overlay
removed, property: /fpga-full/reconf_region0/firmware-name
root@falcon:~#
--- END LOG ---
When loading the partial bitstream, I get the following log:
U-Boot SPL 2021.01 (Oct 12 2021 - 09:28:42 +0000)
[... irrelevant U-Boot SPL log ...]
U-Boot 2021.01 (Oct 12 2021 - 09:28:42 +0000)
[... irrelevant U-Boot log ...]
Scanning mmc 1:1...
Found U-Boot script /boot.scr
509 bytes read in 6 ms (82 KiB/s)
## Executing script at 20000000
26510890 bytes read in 1786 ms (14.2 MiB/s)
design filename = "Falcon;UserID=0XFFFFFFFF;Version=2021.1"
part number = "xczu6eg-ffvc900-1-i"
date = "2021/12/20"
time = "16:29:50"
bytes in bitstream = 26510780
zynqmp_align_dma_buffer: Align buffer at 000000000010006e to
00000000000fff80(swap 0)
49559 bytes read in 16 ms (3 MiB/s)
21019136 bytes read in 1413 ms (14.2 MiB/s)
## Flattened Device Tree blob at 00100000
Booting using the fdt blob at 0x100000
Loading Device Tree to 000000007de03000, end 000000007de12196 ... OK
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[ 0.000000] Linux version 5.10.0-xilinx-v2021.1 (oe-user@oe-host)
(aarch64-poky-linux-gcc (GCC) 10.2.0, GNU ld (GNU Binutils)
2.36.1.20210209) #1 SMP Tue Oct 12 09:30:57 UTC 2021
[....]
[ 0.000000] Kernel command line: video=DP-1:1920x1080 xilinx-
video.is_mplane=0 root=/dev/mmcblk1p2 rw rootwait earlycon
clk_ignore_unused
[....]
[ 2.372821] zynqmp-ipi-mbox mailbox@ff990400: Registered ZynqMP IPI
mbox with TX/RX channels.
[ 2.373018] FPGA manager framework
[....]
[ 2.595794] fpga_manager fpga0: Xilinx ZynqMP FPGA Manager
registered
[ 2.596206] xlnx_pr_decoupler a0080000.user_proc_bridge: fpga bridge
[Xilinx DFX AXI shutdown mgr] registered
[....]
falcon login: root
root@falcon:~# mkdir /sys/kernel/config/device-tree/overlays/user
root@falcon:~# file /lib/firmware/partial.bit
/lib/firmware/partial.bit: Xilinx BIT data - from
Falcon;PARTIAL=TRUE;UserID=0XFFFFFFFF;COMPRESS=FALSE;Version=2021.1 -
for xczu6eg-ffvc900-1-i - built 2021/12/20(16:35:18) - data length
0xf4510
root@falcon:~# cat partial.dtbo > /sys/kernel/config/device-
tree/overlays/user/dtbo
[ 259.672997] fpga_manager fpga0: writing partialBitstream-average.bit
to Xilinx ZynqMP FPGA Manager
[ system crash at this point ]
--- END LOG ---
The result of the Vivado Hardware manager is here:
https://nextcloud.bfh.science/index.php/s/HWSEfzawEm2kWpa
As you can see, the System ILA triggerd on a AWVALID signal rising edge
with the 0xA0080000 address (the address of the bridge). There does not
appear to be a read to that address which would check the the
decoupling actually happened.
Is is possible that there is a delay missing when the overlay is
applied? This delay is present since I have to type the command when I
do it manually (and then it works).
> Regards,
> Navakishore.
Best regards
Simon
