Hi all,
As it's a new tracing device for PCIe traffic, any comments about
the driver or the user interface is appreciated.
Thanks.
On 2020/6/13 17:32, Yicong Yang wrote:
> HiSilicon PCIe tune and trace device(PTT) is a PCIe Root Complex
> integrated Endpoint(RCiEP) device, providing the capability
> to dynamically monitor and tune the PCIe traffic parameters(tune),
> and trace the TLP headers to the memory(trace).
>
> Add the driver for the device to enable its functions. The driver
> will create debugfs directory for each PTT device, and users can
> operate the device through the files under its directory.
>
> RFC:
> - The hardware interface is not yet finalized.
> - The interface to the users is through debugfs, and the usage will
> be further illustrated in the document.
> - The driver is intended to be put under drivers/hwtracing, where
> we think best match the device's function.
>
> Signed-off-by: Yicong Yang <yangyicong@xxxxxxxxxxxxx>
> ---
> Documentation/trace/hisi-ptt.rst | 272 ++++++++
> drivers/hwtracing/Kconfig | 2 +
> drivers/hwtracing/hisilicon/Kconfig | 8 +
> drivers/hwtracing/hisilicon/Makefile | 2 +
> drivers/hwtracing/hisilicon/hisi_ptt.c | 1172 ++++++++++++++++++++++++++++++++
> 5 files changed, 1456 insertions(+)
> create mode 100644 Documentation/trace/hisi-ptt.rst
> create mode 100644 drivers/hwtracing/hisilicon/Kconfig
> create mode 100644 drivers/hwtracing/hisilicon/Makefile
> create mode 100644 drivers/hwtracing/hisilicon/hisi_ptt.c
>
> diff --git a/Documentation/trace/hisi-ptt.rst b/Documentation/trace/hisi-ptt.rst
> new file mode 100644
> index 0000000..c99fbf9
> --- /dev/null
> +++ b/Documentation/trace/hisi-ptt.rst
> @@ -0,0 +1,272 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +======================================
> +HiSilicon PCIe Tune and Trace device
> +======================================
> +
> +Introduction
> +============
> +
> +HiSilicon PCIe tune and trace device(PTT) is a PCIe Root Complex
> +integrated Endpoint(RCiEP) device, providing the capability
> +to dynamically monitor and tune the PCIe link's events(tune),
> +and trace the TLP headers to the memory(trace). The two functions
> +are inpendent, but is recommended to use them together to analyze
> +and enhance the PCIe link's performance.
> +
> +On Hip09, the PCIe root complex is composed of several PCIe cores.
> +And each core is composed of several root ports, RCiEPs, and one
> +PTT device, like below. The PTT device is capable of tuning and
> +tracing the link on and downstream the PCIe core.
> +::
> + +--------------Core 0-------+
> + | | [ PTT ] |
> + | | [Root Port]---[Endpoint]
> + | | [Root Port]---[Endpoint]
> + | | [Root Port]---[Endpoint]
> + Root Complex |------Core 1-------+
> + | | [ PTT ] |
> + | | [Root Port]---[ Switch ]---[Endpoint]
> + | | [Root Port]---[Endpoint] `-[Endpoint]
> + | | [Root Port]---[Endpoint]
> + +---------------------------+
> +
> +The PTT device driver cannot be loaded if debugfs is not mounted.
> +Each PTT device will be presented under /sys/kernel/debugfs/hisi_ptt
> +as its root directory, with name of its BDF number.
> +::
> +
> + /sys/kernel/debug/hisi_ptt/<domain>:<bus>:<device>.<function>
> +
> +Tune
> +====
> +
> +PTT tune is designed for monitoring and adjusting PCIe link parameters(events).
> +Currently we support events 4 classes. The scope of the events
> +covers the PCIe core with which the PTT device belongs to.
> +
> +Each event is presented as a file under $(PTT root dir)/$(BDF)/tune, and
> +mostly this will be a simple open/read/write/close cycle to tune
> +the event.
> +::
> + $ cd /sys/kernel/debug/hisi_ptt/$(BDF)/tune
> + $ ls
> + buf_rx_cpld buf_rx_pd dllp_link_ack_freq link_credit_rx_cplh
> + link_credit_rx_ph qos_tx_dp qos_tx_dp
> + $ cat qos_tx_dp
> + 100
> + $ echo 50 > qos_tx_dp
> + $ cat qos_tx_dp
> + 50
> +
> +Current value(numerical value) of the event can be get by simply
> +read the file, and write the desired value to the file to tune.
> +Tune multiple events at the same time is not permitted, which means
> +you cannot read or write more than one tune file at one time.
> +
> +1. Link credit control
> +----------------------
> +
> +Following files are provided for tune the link credit events of the PCIe core.
> +PCIe link uses credit to control the flow, refer to the PCIe Spec for further
> +information.
> +
> +- link_credit_rx_nph: rx non-posted request headers' credit
> +- link_credit_rx_npd: rx non-posted request data payload's credit
> +- link_credit_rx_ph: rx posted request headers' credit
> +- link_credit_rx_pd: rx posted request data payload's credit
> +- link_credit_rx_cplh: rx completion headers' credit
> +- link_credit_rx_cpld: rx completion data payload's credit
> +
> +Note that the event value is not accurate but a probable one to indicate
> +the level of each event, for example, perhaps 100 for high level,
> +50 for median and 0 for low.
> +
> +2. Link DLLP control
> +--------------------
> +
> +Following files are provided for tune the link events of DLLP of the PCIe core.
> +
> +- dllp_link_ack_freq: frequency of DLLP ACKs
> +- dllp_link_updatefc_freq: frequency of DLLP flow control updates
> +- dllp_link_ssc: spread spectrum control of DLLP link
> +
> +Note that the event value just indicates a probable level, but not
> +accurate.
> +
> +3. Buffer control
> +-----------------
> +
> +Following files are provided for tune the rx/tx buffer depth of the PCIe core.
> +
> +- buf_tx_header: buffer depth for tx packets headers
> +- buf_tx_data: buffer depth for tx packets data payloads
> +- buf_rx_ph: buffer depth for rx posted request packets headers
> +- buf_rx_pd: buffer depth for rx posted request packets data payloads
> +- buf_rx_nph: buffer depth for rx non-posted request packets headers
> +- buf_rx_npd: buffer depth for rx non-posted request packets data payloads
> +- buf_rx_cplh: buffer depth for rx completion packets headers
> +- buf_rx_cpld: buffer depth for rx completion packets data payloads
> +
> +Note that the event value just indicates a probable level, but not
> +accurate.
> +
> +4. Data path QoS control
> +------------------------
> +
> +Following files are provided for tune the QoS of the data path of the PCIe core.
> +
> +- qos_tx_dp: QoS for tx data path
> +- qos_rx_dp: QoS for rx data path
> +
> +Note that the event value just indicates a probable level, but not
> +accurate.
> +
> +Trace
> +=====
> +
> +PTT trace is designed for dumping the TLP headers to the memory, which
> +can be used to analyze the transactions and usage condition of the PCIe
> +Link. You can choose to trace the headers either by its requester ID,
> +or the headers from the link downstream certain root ports, which are
> +on the same core of PTT device. It's also support to trace the headers
> +of certain type and of certain direction.
> +
> +In order to start trace, you need to configure the parameters first.
> +The parameters files is provided under $(PTT root dir)/$(BDF)/trace.
> +::
> + $ cd /sys/kernel/debug/hisi_ptt/$(BDF)/trace
> + $ ls
> + free_buffer filter buflet_nums buflet_size
> + direction type data trace_on
> +
> +1. filter
> +---------
> +
> +You can configure the filter of TLP headers through the file. The filter
> +is provided as BDF numbers of either root port or subordinates, which
> +belong to the same PCIe core. You can get the filters available and
> +currently configure by read the file, and write the desired BDF to the
> +file to set the filters. The default filter is the first root port on
> +the core, and write invalid BDF(not in the available list) will return
> +a failure.
> +::
> + $ echo 0000:80:04.0 > filter
> + $ cat filter
> + 0000:80:00.0 [0000:80:04.0] 0000:81:00.0 0000:81:00.1 0000:82:00.0
> +
> +2. type
> +-------
> +
> +You can trace the TLP headers of certain types by configure the file.
> +Read the file will get available types and current setting, and write
> +the desired type to the file to configure. The default type is
> +`posted_request` and write types not in the available list will return
> +a failure.
> +::
> + $ echo completion > type
> + $ cat type
> + posted_request non-posted_request [completion] all
> +
> +3. direction
> +------------
> +
> +You can trace the TLP headers from certain direction, which is relative
> +to the root port or the PCIe core. Read the file to get available
> +directions and current configurition, and write the desired direction
> +to configure. The default value is `rx` and any invalid direction will
> +return a failure. Note `rxtx_no_dma_p2p` means the headers of both
> +directions, but not include P2P DMA access.
> +::
> + $ echo rxtx > direction
> + $ cat direction
> + rx tx [rxtx] rxtx_no_dma_p2p
> +
> +4. buflet_size
> +--------------
> +
> +The traced TLP headers will be written to the memory allocated
> +by the driver. The hardware accept 4 DMA address with same size,
> +and write the buflet sequetially like below. If DMA addr 3 is
> +finished and the trace is still on, it will return to addr 0.
> +Driver will allocated each DMA buffer (we call it buflet) and
> +swap a preallocated one if it has been finished.
> +::
> + +->[DMA addr 0]->[DMA addr 1]->[DMA addr 2]->[DMA addr 3]-+
> + +---------------------------------------------------------+
> +
> +You should both configure the buflet_size and buflet_nums to
> +configure the `trace buffer` to receive the TLP headers. The
> +total trace buffer size is buflet_size * buflet_nums. Note
> +that the trace buffer will not be allocated immediately after you
> +configure the parameters, but will be allocated right before
> +the trace starts.
> +
> +This file configures the buflet size. Read the file will get
> +available buflet size and size set currently, write the desired
> +size to the file to configure. The default size is 2 MiB and any
> +invalid size written will return a failure.
> +::
> + $ cat buflet_size
> + [2 MiB] 4 MiB 6 MiB 8 MiB 10 MiB
> + $ echo 8 > buflet_size
> + $ cat buflet_size
> + 2 MiB 4 MiB 6 MiB [8 MiB] 10 MiB
> +
> +5. buflet_nums
> +--------------
> +
> +You can write the desired buflet counts to the file to configure,
> +and read the file to get current buflet counts. The default
> +value is 64. And any positive value is valid. Note that big value
> +may lead to DMA memory allocation failure, and you will not be
> +able to start tracing. If it happens, you should consider adjusting
> +buflet_nums or buflet_size.
> +::
> + $ cat buflet_nums
> + 64
> + $ echo 128 > buflet_nums
> + $ cat buflet_nums
> + 128
> +
> +6. data
> +-------
> +
> +The file to access the traced data. You can read the file to get the
> +binary blob of traced TLP headers. The format of the headers is
> +4 Dword length and is just as defined by the PCIe Spec r4.0,
> +Sec 2.2.4.1, or 8 Dword length with additional 4 Dword extra
> +information.
> +
> +echo "" > data will free all the trace buffers allocated as well as
> +the traced datas.
> +
> +7. trace_on
> +-----------
> +
> +Start or end the trace by simple writing to the file, and monitor the
> +trace status by reading the file.
> +::
> + $ echo 1 > trace_on # start trace
> + $ cat trace_on # get the trace status
> + 1
> + $ echo 0 > trace_on # manually end trace
> +
> +The read value of the trace_on will be auto cleared if the buffer
> +allocated is full. 1 indicates the trace is running and 0 for
> +stopped. Write any non-zero value to the file can start trace.
> +
> +8. free_buffer
> +--------------
> +
> +File to indicate the trace buffer status and to manually free the
> +trace buffer. The read value of 1 indicates the trace buffer has
> +been allocated and exists in the memory, while 0 indicates there
> +is no buffer allocated. Write 1 to the file to free the trace
> +buffer as well as the traced datas.
> +::
> + $ cat free_buffer
> + 1 # indicate the buffer exists
> + $ echo 1 > free_buffer # free the trace buffer
> + $ cat free_buffer
> + 0
> diff --git a/drivers/hwtracing/Kconfig b/drivers/hwtracing/Kconfig
> index 1308583..e3796b1 100644
> --- a/drivers/hwtracing/Kconfig
> +++ b/drivers/hwtracing/Kconfig
> @@ -5,4 +5,6 @@ source "drivers/hwtracing/stm/Kconfig"
>
> source "drivers/hwtracing/intel_th/Kconfig"
>
> +source "drivers/hwtracing/hisilicon/Kconfig"
> +
> endmenu
> diff --git a/drivers/hwtracing/hisilicon/Kconfig b/drivers/hwtracing/hisilicon/Kconfig
> new file mode 100644
> index 0000000..95e91b9
> --- /dev/null
> +++ b/drivers/hwtracing/hisilicon/Kconfig
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config HISI_PTT
> + tristate "HiSilicon PCIe Tune and Trace Device"
> + depends on PCI && HAS_DMA && HAS_IOMEM
> + help
> + HiSilicon PCIe Tune and Trace Device exist as a PCIe iEP
> + device, provides support for PCIe traffic tuning and
> + tracing TLP headers to the memory.
> diff --git a/drivers/hwtracing/hisilicon/Makefile b/drivers/hwtracing/hisilicon/Makefile
> new file mode 100644
> index 0000000..908c09a
> --- /dev/null
> +++ b/drivers/hwtracing/hisilicon/Makefile
> @@ -0,0 +1,2 @@
> +# SPDX-License-Identifier: GPL-2.0
> +obj-$(CONFIG_HISI_PTT) += hisi_ptt.o
> diff --git a/drivers/hwtracing/hisilicon/hisi_ptt.c b/drivers/hwtracing/hisilicon/hisi_ptt.c
> new file mode 100644
> index 0000000..c58a3cc
> --- /dev/null
> +++ b/drivers/hwtracing/hisilicon/hisi_ptt.c
> @@ -0,0 +1,1172 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Driver for HiSilicon PCIe tune and trace device
> + *
> + * Copyright (c) 2020 HiSilicon Limited.
> + */
> +
> +#include <linux/bitfield.h>
> +#include <linux/bitops.h>
> +#include <linux/debugfs.h>
> +#include <linux/delay.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/interrupt.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/seq_file.h>
> +#include <linux/pci.h>
> +
> +#define HISI_PTT_CTRL_STR_LEN 40
> +#define HISI_PTT_DEFAULT_TRACE_BUF_CNT 64
> +
> +#define HISI_PTT_RESET_WAIT_MS 1000UL
> +
> +#define HISI_PTT_IRQ_NUMS 1
> +#define HISI_PTT_DMA_IRQ 0
> +#define HISI_PTT_DMA_NUMS 4
> +
> +#define HISI_PTT_TUNING_CTRL 0x0380
> +#define HISI_PTT_TUNING_CTRL_CODE GENMASK(3, 0)
> +#define HISI_PTT_TUNING_CTRL_EN GENMASK(23, 16)
> +#define HISI_PTT_TUNING_DATA 0x0384
> +#define HISI_PTT_TUNING_DATA_VAL GENMASK(15, 0)
> +#define HISI_PTT_TRACE_ADDR_SIZE 0x0400
> +#define HISI_PTT_TRACE_ADDR_BASE_LO_0 0x0410
> +#define HISI_PTT_TRACE_ADDR_BASE_HI_0 0x0414
> +#define HISI_PTT_TRACE_CTRL 0x0450
> +#define HISI_PTT_TRACE_CTRL_EN BIT(0)
> +#define HISI_PTT_TRACE_CTRL_RST BIT(1)
> +#define HISI_PTT_TRACE_CTRL_RXTX_SEL GENMASK(3, 2)
> +#define HISI_PTT_TRACE_CTRL_TYPE_SEL GENMASK(7, 4)
> +#define HISI_PTT_TRACE_CTRL_DATA_FORMAT BIT(14)
> +#define HISI_PTT_TRACE_CTRL_FILTER_MODE BIT(15)
> +#define HISI_PTT_TRACE_CTRL_TARGET_SEL GENMASK(31, 16)
> +#define HISI_PTT_TRACE_INT_STAT 0x0490
> +#define HISI_PTT_TRACE_INT_STAT_MASK GENMASK(3, 0)
> +#define HISI_PTT_TRACE_WR_STS 0x04a0
> +#define HISI_PTT_TRACE_WR_STS_WRITE GENMASK(27, 0)
> +#define HISI_PTT_TRACE_WR_STS_BUFFER GENMASK(29, 28)
> +#define HISI_PTT_TRACE_STS 0x04b0
> +#define HISI_PTT_TRACE_IDLE BIT(0)
> +#define HISI_PTT_MAILBOX_0 0x07e0
> +
> +static struct dentry *hisi_ptt_debugfs_root;
> +
> +struct event_desc {
> + const char *name;
> + u32 event_code;
> +};
> +
> +static struct event_desc tune_events[] = {
> + { "link_credit_rx_nph", 0x1 | (BIT(1) << 16) },
> + { "link_credit_rx_npd", 0x1 | (BIT(2) << 16) },
> + { "link_credit_rx_ph", 0x1 | (BIT(3) << 16) },
> + { "link_credit_rx_pd", 0x1 | (BIT(4) << 16) },
> + { "link_credit_rx_cplh", 0x1 | (BIT(5) << 16) },
> + { "link_credit_rx_cpld", 0x1 | (BIT(6) << 16) },
> + { "dllp_link_ack_freq", 0x2 | (BIT(1) << 16) },
> + { "dllp_link_updatefc_freq", 0x2 | (BIT(2) << 16) },
> + { "dllp_link_ssc", 0x2 | (BIT(2) << 16) },
> + { "buf_tx_header", 0x3 | (BIT(1) << 16) },
> + { "buf_tx_data", 0x3 | (BIT(2) << 16) },
> + { "buf_rx_ph", 0x3 | (BIT(3) << 16) },
> + { "buf_rx_pd", 0x3 | (BIT(4) << 16) },
> + { "buf_rx_nph", 0x3 | (BIT(5) << 16) },
> + { "buf_rx_npd", 0x3 | (BIT(6) << 16) },
> + { "buf_rx_cplh", 0x3 | (BIT(7) << 16) },
> + { "buf_rx_cpld", 0x3 | (BIT(8) << 16) },
> + { "qos_tx_dp", 0x4 | (BIT(1) << 16) },
> + { "qos_rx_dp", 0x4 | (BIT(1) << 16) },
> +};
> +
> +static struct event_desc trace_rxtx[] = {
> + { "rx", 0 },
> + { "tx", 1 },
> + { "rxtx", 2 },
> + { "rxtx_no_dma_p2p", 3 },
> +};
> +
> +static struct event_desc trace_events[] = {
> + { "posted_request", 0 },
> + { "non-posted_request", 2 },
> + { "completion", 4 },
> + { "all", 8 },
> +};
> +
> +static const int available_buflet_size[] = {
> + 0x00200000, /* 2 MiB */
> + 0x00400000, /* 4 MiB */
> + 0x00600000, /* 6 MiB */
> + 0x00800000, /* 8 MiB */
> + 0x00a00000, /* 10 MiB */
> +};
> +
> +struct debugfs_file_desc {
> + struct hisi_ptt *hisi_ptt;
> + const char *name;
> + const struct file_operations *fops;
> + int index;
> +};
> +
> +struct dma_buflet {
> + struct list_head list;
> + dma_addr_t dma;
> + void *addr;
> + int index;
> + u64 size;
> +};
> +
> +struct ptt_trace_ctrl {
> + struct list_head trace_buf;
> + struct dma_buflet *cur;
> + atomic_t status; /* 0:idle, 1:tracing */
> + u64 buflet_nums;
> + u32 buflet_size;
> + u32 tr_event;
> + u32 rxtx;
> +};
> +
> +struct per_func_info {
> + struct list_head list;
> + struct pci_dev *pdev;
> +};
> +
> +struct hisi_ptt {
> + struct ptt_trace_ctrl trace_ctrl;
> + struct per_func_info *target_func;
> + struct list_head avail_devfns;
> + struct dentry *debugfs_dir;
> + void __iomem *iobase;
> + struct pci_dev *pdev;
> + struct mutex mutex; /* protects the hisi_ptt structure */
> + const char *name;
> + u32 tune_event;
> + u32 domain;
> + u32 bus;
> +};
> +
> +static u32 hisi_ptt_tune_data_read(struct hisi_ptt *hisi_ptt)
> +{
> + u32 val;
> +
> + writel(hisi_ptt->tune_event, hisi_ptt->iobase + HISI_PTT_TUNING_CTRL);
> +
> + val = readl(hisi_ptt->iobase + HISI_PTT_TUNING_DATA);
> + val &= HISI_PTT_TUNING_DATA_VAL;
> +
> + return val;
> +}
> +
> +static void hisi_ptt_tune_data_write(struct hisi_ptt *hisi_ptt, u32 data)
> +{
> + writel(hisi_ptt->tune_event, hisi_ptt->iobase + HISI_PTT_TUNING_CTRL);
> + data &= HISI_PTT_TUNING_DATA_VAL;
> + writel(data, hisi_ptt->iobase + HISI_PTT_TUNING_DATA);
> +}
> +
> +static ssize_t hisi_ptt_tune_common_read(struct file *filp, char __user *buf,
> + size_t count, loff_t *pos)
> +{
> + struct debugfs_file_desc *desc = filp->private_data;
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
> + char tbuf[HISI_PTT_CTRL_STR_LEN];
> + int len;
> + u32 val;
> +
> + if (!mutex_trylock(&hisi_ptt->mutex))
> + return -EBUSY;
> + hisi_ptt->tune_event = tune_events[desc->index].event_code;
> + val = hisi_ptt_tune_data_read(hisi_ptt);
> + mutex_unlock(&hisi_ptt->mutex);
> +
> + len = snprintf(tbuf, HISI_PTT_CTRL_STR_LEN,
> + "%d\n", val);
> +
> + return simple_read_from_buffer(buf, count, pos, tbuf, len);
> +}
> +
> +static ssize_t hisi_ptt_tune_common_write(struct file *filp,
> + const char __user *buf, size_t count, loff_t *pos)
> +{
> + struct debugfs_file_desc *desc = filp->private_data;
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
> + char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
> + int len, val;
> +
> + len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN - 1,
> + pos, buf, count);
> + if (len < 0)
> + return -EINVAL;
> + cp = strchr(tbuf, '\n');
> + if (cp)
> + *cp = '\0';
> + if (kstrtouint(tbuf, 0, &val))
> + return -EINVAL;
> +
> + if (!mutex_trylock(&hisi_ptt->mutex))
> + return -EBUSY;
> + hisi_ptt->tune_event = tune_events[desc->index].event_code;
> + hisi_ptt_tune_data_write(hisi_ptt, val);
> + mutex_unlock(&hisi_ptt->mutex);
> +
> + return count;
> +}
> +
> +static const struct file_operations tune_common_fops = {
> + .owner = THIS_MODULE,
> + .open = simple_open,
> + .read = hisi_ptt_tune_common_read,
> + .write = hisi_ptt_tune_common_write,
> + .llseek = no_llseek,
> +};
> +
> +static void hisi_ptt_free_trace_buf(struct hisi_ptt *hisi_ptt)
> +{
> + struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
> + struct device *dev = &hisi_ptt->pdev->dev;
> + struct dma_buflet *buflet, *tbuflet;
> +
> + if (list_empty(&ctrl->trace_buf))
> + return;
> +
> + list_for_each_entry_safe(buflet, tbuflet, &ctrl->trace_buf, list) {
> + dma_free_coherent(dev, buflet->size, buflet->addr, buflet->dma);
> + list_del(&buflet->list);
> + kfree(buflet);
> + }
> +}
> +
> +static int hisi_ptt_alloc_trace_buf(struct hisi_ptt *hisi_ptt)
> +{
> + struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
> + struct device *dev = &hisi_ptt->pdev->dev;
> + struct dma_buflet *buflet;
> + int i, ret;
> +
> + /* Make sure the trace buffer is empty before allocating */
> + if (!list_empty(&ctrl->trace_buf))
> + hisi_ptt_free_trace_buf(hisi_ptt);
> +
> + for (i = 0; i < ctrl->buflet_nums; ++i) {
> + buflet = kzalloc(sizeof(*buflet), GFP_KERNEL);
> + if (!buflet) {
> + ret = -ENOMEM;
> + goto err;
> + }
> + buflet->addr = dma_alloc_coherent(dev, ctrl->buflet_size,
> + &buflet->dma, GFP_KERNEL);
> + if (!buflet->addr) {
> + kfree(buflet);
> + ret = -ENOMEM;
> + goto err;
> + }
> + buflet->index = i;
> + buflet->size = ctrl->buflet_size;
> + list_add_tail(&buflet->list, &ctrl->trace_buf);
> + }
> +
> + return 0;
> +err:
> + hisi_ptt_free_trace_buf(hisi_ptt);
> + return ret;
> +}
> +
> +static void hisi_ptt_trace_end(struct hisi_ptt *hisi_ptt)
> +{
> + writel(0, hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
> + atomic_dec(&hisi_ptt->trace_ctrl.status);
> + hisi_ptt->trace_ctrl.cur = NULL;
> +}
> +
> +static int hisi_ptt_trace_start(struct hisi_ptt *hisi_ptt)
> +{
> + struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
> + struct pci_dev *pdev, *rp;
> + u32 val;
> + int i;
> +
> + if (!hisi_ptt->target_func) {
> + pci_err(hisi_ptt->pdev, "No available root port/function\n");
> + return -ENODEV;
> + }
> + pdev = hisi_ptt->target_func->pdev;
> + rp = pcie_find_root_port(pdev);
> +
> + val = readl(hisi_ptt->iobase + HISI_PTT_TRACE_STS);
> + if (!(val & HISI_PTT_TRACE_IDLE)) /* Trace has already started */
> + return -EBUSY;
> +
> + /* reset the DMA before start tracing */
> + val = readl(hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
> + val |= HISI_PTT_TRACE_CTRL_RST;
> + writel(val, hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
> +
> + msleep(HISI_PTT_RESET_WAIT_MS);
> +
> + val = readl(hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
> + val &= ~HISI_PTT_TRACE_CTRL_RST;
> + writel(val, hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
> +
> + /* clear the interrupt status */
> + writel(0, hisi_ptt->iobase + HISI_PTT_TRACE_INT_STAT);
> +
> + for (i = 0; i < HISI_PTT_DMA_NUMS; ++i) {
> + if (!ctrl->cur)
> + ctrl->cur = list_first_entry(&ctrl->trace_buf, struct dma_buflet, list);
> + else
> + ctrl->cur = list_next_entry(ctrl->cur, list);
> +
> + writel(ctrl->cur->dma,
> + hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_BASE_LO_0 + i * 8);
> + writel(ctrl->cur->dma >> 32,
> + hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_BASE_HI_0 + i * 8);
> + }
> + writel(ctrl->buflet_size, hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_SIZE);
> +
> + /* set the trace control register */
> + val = 0;
> + val |= FIELD_PREP(HISI_PTT_TRACE_CTRL_TYPE_SEL, ctrl->tr_event);
> + val |= FIELD_PREP(HISI_PTT_TRACE_CTRL_RXTX_SEL, ctrl->rxtx);
> + /*
> + * The TLP headers can be filtered either by the root port,
> + * or by the requester ID.
> + */
> + if (pci_pcie_type(pdev) != PCI_EXP_TYPE_ROOT_PORT) {
> + val |= FIELD_PREP(HISI_PTT_TRACE_CTRL_TARGET_SEL, pdev->bus->number << 8 | pdev->devfn);
> + val |= HISI_PTT_TRACE_CTRL_FILTER_MODE;
> + } else {
> + val |= FIELD_PREP(HISI_PTT_TRACE_CTRL_TARGET_SEL, BIT(PCI_SLOT(rp->devfn)));
> + }
> +
> + val |= HISI_PTT_TRACE_CTRL_EN;
> + writel(val, hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
> +
> + atomic_inc(&ctrl->status);
> +
> + return 0;
> +}
> +
> +#define TRACE_ATTR(__name) \
> +static int __name ## _open(struct inode *inode, struct file *filp) \
> +{ \
> + struct debugfs_file_desc *desc = inode->i_private; \
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt; \
> + if (!mutex_trylock(&hisi_ptt->mutex)) \
> + return -EBUSY; \
> + return single_open(filp, __name ## _show, hisi_ptt); \
> +} \
> +static int __name ## _release(struct inode *inode, struct file *filp) \
> +{ \
> + struct debugfs_file_desc *desc = inode->i_private; \
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt; \
> + mutex_unlock(&hisi_ptt->mutex); \
> + return seq_release(inode, filp); \
> +} \
> +static const struct file_operations __name ## _fops = { \
> + .owner = THIS_MODULE, \
> + .open = __name ## _open, \
> + .read = seq_read, \
> + .write = __name ## _write, \
> + .llseek = seq_lseek, \
> + .release = __name ## _release, \
> +}
> +
> +static int set_filter_show(struct seq_file *m, void *v)
> +{
> + struct hisi_ptt *hisi_ptt = m->private;
> + struct per_func_info *func;
> +
> + list_for_each_entry(func, &hisi_ptt->avail_devfns, list) {
> + struct pci_dev *pdev = func->pdev;
> +
> + if (func == hisi_ptt->target_func)
> + seq_printf(m, "[%04x:%02x:%02x.%d]\n",
> + pci_domain_nr(pdev->bus),
> + pdev->bus->number, PCI_SLOT(pdev->devfn),
> + PCI_FUNC(pdev->devfn));
> + else
> + seq_printf(m, "%04x:%02x:%02x.%d\n",
> + pci_domain_nr(pdev->bus),
> + pdev->bus->number, PCI_SLOT(pdev->devfn),
> + PCI_FUNC(pdev->devfn));
> + }
> +
> + return 0;
> +}
> +
> +static ssize_t set_filter_write(struct file *filp, const char __user *buf,
> + size_t count, loff_t *pos)
> +{
> + struct seq_file *m = filp->private_data;
> + struct hisi_ptt *hisi_ptt = m->private;
> + char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
> + u32 domain, bus, dev, fn, devfn;
> + struct per_func_info *tfunc;
> + int len, num;
> + bool found = false;
> +
> + if (list_empty(&hisi_ptt->avail_devfns))
> + return -EINVAL;
> + len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN - 1, pos, buf, count);
> + if (len < 0)
> + return -EINVAL;
> + cp = strchr(tbuf, '\n');
> + if (cp)
> + *cp = '\0';
> +
> + /*
> + * the input should be like 0000:80:01.1, etc. Parse it
> + * and check whether it's in the available func list.
> + */
> + num = sscanf(tbuf, "%04x:%02x:%02x.%d", &domain, &bus, &dev, &fn);
> + if (num != 4)
> + return -EINVAL;
> +
> + devfn = PCI_DEVFN(dev, fn);
> + list_for_each_entry(tfunc, &hisi_ptt->avail_devfns, list) {
> + struct pci_dev *pdev = tfunc->pdev;
> +
> + if (domain == pci_domain_nr(pdev->bus) &&
> + bus == pdev->bus->number && devfn == pdev->devfn) {
> + hisi_ptt->target_func = tfunc;
> + found = true;
> + break;
> + }
> + }
> +
> + if (!found)
> + return -EINVAL;
> +
> + return count;
> +}
> +TRACE_ATTR(set_filter);
> +
> +static int set_direction_show(struct seq_file *m, void *v)
> +{
> + struct hisi_ptt *hisi_ptt = m->private;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(trace_rxtx); ++i) {
> + if (hisi_ptt->trace_ctrl.rxtx == trace_rxtx[i].event_code)
> + seq_printf(m, "[%s] ", trace_rxtx[i].name);
> + else
> + seq_printf(m, "%s ", trace_rxtx[i].name);
> + }
> + seq_putc(m, '\n');
> +
> + return 0;
> +}
> +
> +static ssize_t set_direction_write(struct file *filp, const char __user *buf,
> + size_t count, loff_t *pos)
> +{
> + struct seq_file *m = filp->private_data;
> + struct hisi_ptt *hisi_ptt = m->private;
> + char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
> + bool set = false;
> + int len, i;
> +
> + len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN, pos, buf, count);
> + if (len < 0)
> + return -EINVAL;
> + cp = strchr(tbuf, '\n');
> + if (cp)
> + *cp = '\0';
> +
> + for (i = 0; i < ARRAY_SIZE(trace_rxtx); ++i) {
> + if (!strcmp(tbuf, trace_rxtx[i].name)) {
> + hisi_ptt->trace_ctrl.rxtx = trace_rxtx[i].event_code;
> + set = true;
> + break;
> + }
> + }
> +
> + if (!set)
> + return -EINVAL;
> +
> + return 0;
> +}
> +TRACE_ATTR(set_direction);
> +
> +static int set_trace_type_show(struct seq_file *m, void *v)
> +{
> + struct hisi_ptt *hisi_ptt = m->private;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(trace_events); ++i) {
> + if (hisi_ptt->trace_ctrl.tr_event == trace_events[i].event_code)
> + seq_printf(m, "[%s] ", trace_events[i].name);
> + else
> + seq_printf(m, "%s ", trace_events[i].name);
> + }
> + seq_putc(m, '\n');
> +
> + return 0;
> +}
> +
> +static ssize_t set_trace_type_write(struct file *filp, const char __user *buf,
> + size_t count, loff_t *pos)
> +{
> + struct seq_file *m = filp->private_data;
> + struct hisi_ptt *hisi_ptt = m->private;
> + char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
> + bool set = false;
> + int len, i;
> +
> + len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN, pos, buf, count);
> + if (len < 0)
> + return -EINVAL;
> + cp = strchr(tbuf, '\n');
> + if (cp)
> + *cp = '\0';
> +
> + for (i = 0; i < ARRAY_SIZE(trace_events); ++i) {
> + if (!strcmp(tbuf, trace_events[i].name)) {
> + hisi_ptt->trace_ctrl.tr_event = trace_events[i].event_code;
> + set = true;
> + break;
> + }
> + }
> +
> + if (!set)
> + return -EINVAL;
> +
> + return count;
> +}
> +TRACE_ATTR(set_trace_type);
> +
> +static int trace_on_get(void *data, u64 *val)
> +{
> + struct debugfs_file_desc *desc = data;
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
> +
> + if (!mutex_trylock(&hisi_ptt->mutex))
> + return -EBUSY;
> +
> + *val = atomic_read(&hisi_ptt->trace_ctrl.status);
> +
> + mutex_unlock(&hisi_ptt->mutex);
> +
> + return 0;
> +}
> +
> +static int trace_on_set(void *data, u64 val)
> +{
> + struct debugfs_file_desc *desc = data;
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
> + int ret = 0;
> +
> + if (!mutex_trylock(&hisi_ptt->mutex))
> + return -EBUSY;
> +
> + if (val) {
> + if (atomic_read(&hisi_ptt->trace_ctrl.status))
> + goto out;
> + if (hisi_ptt_alloc_trace_buf(hisi_ptt)) {
> + ret = -ENOMEM;
> + goto out;
> + }
> + if (hisi_ptt_trace_start(hisi_ptt)) {
> + ret = -EBUSY;
> + goto out;
> + }
> + } else {
> + if (!atomic_read(&hisi_ptt->trace_ctrl.status))
> + goto out;
> + hisi_ptt_trace_end(hisi_ptt);
> + }
> +
> +out:
> + hisi_ptt_free_trace_buf(hisi_ptt);
> + mutex_unlock(&hisi_ptt->mutex);
> + return ret;
> +}
> +DEFINE_DEBUGFS_ATTRIBUTE(trace_on_fops, trace_on_get,
> + trace_on_set, "%lld\n");
> +
> +static int set_trace_buf_nums_get(void *data, u64 *val)
> +{
> + struct debugfs_file_desc *desc = data;
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
> +
> + if (!mutex_trylock(&hisi_ptt->mutex))
> + return -EBUSY;
> +
> + *val = hisi_ptt->trace_ctrl.buflet_nums;
> +
> + mutex_unlock(&hisi_ptt->mutex);
> + return 0;
> +}
> +
> +static int set_trace_buf_nums_set(void *data, u64 val)
> +{
> + struct debugfs_file_desc *desc = data;
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
> +
> + if (!mutex_trylock(&hisi_ptt->mutex))
> + return -EBUSY;
> +
> + hisi_ptt->trace_ctrl.buflet_nums = val;
> +
> + mutex_unlock(&hisi_ptt->mutex);
> + return 0;
> +}
> +DEFINE_DEBUGFS_ATTRIBUTE(set_trace_buf_nums_fops, set_trace_buf_nums_get,
> + set_trace_buf_nums_set, "%lld\n");
> +
> +static int set_trace_buflet_size_show(struct seq_file *m, void *v)
> +{
> + struct hisi_ptt *hisi_ptt = m->private;
> + struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(available_buflet_size); ++i) {
> + if (ctrl->buflet_size == available_buflet_size[i]) {
> + seq_printf(m, "[%dMiB] ",
> + available_buflet_size[i] >> 20);
> + continue;
> + }
> + seq_printf(m, "%dMiB ", available_buflet_size[i] >> 20);
> + }
> + seq_putc(m, '\n');
> +
> + return 0;
> +}
> +
> +static ssize_t set_trace_buflet_size_write(struct file *filp,
> + const char __user *buf, size_t count, loff_t *pos)
> +{
> + struct seq_file *m = filp->private_data;
> + struct hisi_ptt *hisi_ptt = m->private;
> + struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
> + char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
> + int i, len, size, set = 0;
> +
> + len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN, pos, buf, count);
> + cp = strchr(tbuf, '\n');
> + if (cp)
> + *cp = '\0';
> + if (kstrtouint(tbuf, 0, &size))
> + return -EINVAL;
> + size <<= 20;
> +
> + for (i = 0; i < ARRAY_SIZE(available_buflet_size); ++i) {
> + if (available_buflet_size[i] == size) {
> + ctrl->buflet_size = size;
> + set = 1;
> + break;
> + }
> + }
> +
> + if (!set)
> + return -EINVAL;
> +
> + return count;
> +}
> +TRACE_ATTR(set_trace_buflet_size);
> +
> +static int free_trace_buf_get(void *data, u64 *val)
> +{
> + struct debugfs_file_desc *desc = data;
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
> +
> + if (!mutex_trylock(&hisi_ptt->mutex))
> + return -EBUSY;
> +
> + *val = list_empty(&hisi_ptt->trace_ctrl.trace_buf);
> +
> + mutex_unlock(&hisi_ptt->mutex);
> + return 0;
> +}
> +
> +static int free_trace_buf_set(void *data, u64 val)
> +{
> + struct debugfs_file_desc *desc = data;
> + struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
> + int ret = 0;
> +
> + if (!mutex_trylock(&hisi_ptt->mutex))
> + return -EBUSY;
> +
> + if (!list_empty(&hisi_ptt->trace_ctrl.trace_buf)) {
> + if (atomic_read(&hisi_ptt->trace_ctrl.status))
> + ret = -EBUSY;
> + else
> + hisi_ptt_free_trace_buf(hisi_ptt);
> + }
> +
> + mutex_unlock(&hisi_ptt->mutex);
> + return ret;
> +}
> +DEFINE_DEBUGFS_ATTRIBUTE(free_trace_buf_fops, free_trace_buf_get,
> + free_trace_buf_set, "%lld\n");
> +
> +static void *trace_data_next(struct seq_file *m, void *v, loff_t *pos)
> +{
> + struct hisi_ptt *hisi_ptt = m->private;
> + struct dma_buflet *buflet = v;
> +
> + (*pos)++;
> +
> + if (!list_is_last(&buflet->list, &hisi_ptt->trace_ctrl.trace_buf))
> + return list_next_entry(buflet, list);
> +
> + return NULL;
> +}
> +
> +static void *trace_data_start(struct seq_file *m, loff_t *pos)
> +{
> + struct hisi_ptt *hisi_ptt = m->private;
> + struct dma_buflet *buflet;
> + loff_t off = 0;
> +
> + buflet = list_first_entry(&hisi_ptt->trace_ctrl.trace_buf, struct dma_buflet, list);
> + while (off < *pos)
> + buflet = trace_data_next(m, buflet, &off);
> +
> + return buflet;
> +}
> +
> +static void trace_data_stop(struct seq_file *m, void *p)
> +{
> + /* Nothing to do, only a stub */
> +}
> +
> +static int trace_data_show(struct seq_file *m, void *v)
> +{
> + struct dma_buflet *buflet = v;
> +
> + if (buflet)
> + seq_write(m, buflet->addr, buflet->size);
> +
> + return 0;
> +}
> +
> +static const struct seq_operations trace_data_seq_ops = {
> + .start = trace_data_start,
> + .next = trace_data_next,
> + .stop = trace_data_stop,
> + .show = trace_data_show,
> +};
> +
> +static int trace_data_open(struct inode *inode, struct file *filep)
> +{
> + struct hisi_ptt *hisi_ptt = inode->i_private;
> + struct seq_file *m;
> + int ret;
> +
> + /*
> + * Check the trace status, we cannot read the
> + * data if the trace is still on. Then hold the
> + * lock when reading the traced data.
> + */
> + if (atomic_read(&hisi_ptt->trace_ctrl.status) ||
> + !mutex_trylock(&hisi_ptt->mutex))
> + return -EBUSY;
> +
> + if (list_empty(&hisi_ptt->trace_ctrl.trace_buf)) {
> + ret = -ENOTTY;
> + goto err;
> + }
> +
> + ret = seq_open(filep, &trace_data_seq_ops);
> + if (ret)
> + goto err;
> +
> + m = filep->private_data;
> + m->private = hisi_ptt;
> +
> + return 0;
> +err:
> + mutex_unlock(&hisi_ptt->mutex);
> + return ret;
> +}
> +
> +static ssize_t trace_data_write(struct file *filp, const char __user *buf,
> + size_t count, loff_t *off)
> +{
> + struct seq_file *m = filp->private_data;
> + struct hisi_ptt *hisi_ptt = m->private;
> + char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
> + int len;
> +
> + len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN, off, buf, count);
> + cp = strchr(tbuf, '\n');
> + if (cp)
> + *cp = '\0';
> +
> + /* Free the trace buffer when echo "" > trace_data */
> + if (!strlen(tbuf) && !list_empty(&hisi_ptt->trace_ctrl.trace_buf)) {
> + if (atomic_read(&hisi_ptt->trace_ctrl.status))
> + return -EBUSY;
> + hisi_ptt_free_trace_buf(hisi_ptt);
> + }
> +
> + return count;
> +}
> +
> +static int trace_data_release(struct inode *inode, struct file *filp)
> +{
> + struct hisi_ptt *hisi_ptt = inode->i_private;
> +
> + mutex_unlock(&hisi_ptt->mutex);
> +
> + return seq_release(inode, filp);
> +}
> +
> +static const struct file_operations trace_data_fops = {
> + .owner = THIS_MODULE,
> + .open = trace_data_open,
> + .read = seq_read,
> + .write = trace_data_write,
> + .llseek = no_llseek,
> + .release = trace_data_release,
> +};
> +
> +static struct debugfs_file_desc trace_entries[] = {
> + { NULL, "filter", &set_filter_fops, 0 },
> + { NULL, "direction", &set_direction_fops, 0 },
> + { NULL, "type", &set_trace_type_fops, 0 },
> + { NULL, "trace_on", &trace_on_fops, 0 },
> + { NULL, "buf_nums", &set_trace_buf_nums_fops, 0 },
> + { NULL, "buflet_size", &set_trace_buflet_size_fops, 0 },
> + { NULL, "free_buffer", &free_trace_buf_fops, 0 },
> + { NULL, "data", &trace_data_fops, 0 },
> +};
> +
> +irqreturn_t hisi_ptt_isr(int irq, void *context)
> +{
> + struct hisi_ptt *hisi_ptt = context;
> + struct dma_buflet *next, *cur;
> + u32 val, buf_idx;
> +
> + val = readl(hisi_ptt->iobase + HISI_PTT_TRACE_INT_STAT);
> + buf_idx = __ffs(val & HISI_PTT_TRACE_INT_STAT_MASK);
> + /*
> + * Check whether the trace buffer is full. Stop tracing
> + * when the last DMA buffer is finished. Otherwise, assign
> + * the address of next buflet to the DMA register.
> + */
> + cur = hisi_ptt->trace_ctrl.cur;
> + if (list_is_last(&cur->list, &hisi_ptt->trace_ctrl.trace_buf)) {
> + if ((val & HISI_PTT_TRACE_INT_STAT_MASK) == HISI_PTT_TRACE_INT_STAT_MASK)
> + hisi_ptt_trace_end(hisi_ptt);
> + } else {
> + next = list_next_entry(cur, list);
> + writel(next->dma, hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_BASE_LO_0 + buf_idx * 8);
> + writel(next->dma >> 32, hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_BASE_HI_0 + buf_idx * 8);
> + hisi_ptt->trace_ctrl.cur = next;
> + val &= ~BIT(buf_idx);
> + writel(val, hisi_ptt->iobase + HISI_PTT_TRACE_INT_STAT);
> + }
> +
> + return IRQ_HANDLED;
> +}
> +
> +irqreturn_t hisi_ptt_irq(int irq, void *context)
> +{
> + struct hisi_ptt *hisi_ptt = context;
> + u32 status;
> +
> + status = readl(hisi_ptt->iobase + HISI_PTT_TRACE_INT_STAT);
> + if (!(status & HISI_PTT_TRACE_INT_STAT_MASK))
> + return IRQ_NONE;
> +
> + return IRQ_WAKE_THREAD;
> +}
> +
> +static int hisi_ptt_irq_register(struct hisi_ptt *hisi_ptt)
> +{
> + struct pci_dev *pdev = hisi_ptt->pdev;
> + int ret;
> +
> + ret = pci_alloc_irq_vectors(pdev, HISI_PTT_IRQ_NUMS, HISI_PTT_IRQ_NUMS,
> + PCI_IRQ_MSI);
> + if (ret < 0) {
> + pci_err(pdev, "allocate irq vector failed %d", ret);
> + return ret;
> + }
> +
> + ret = request_threaded_irq(pci_irq_vector(pdev, HISI_PTT_DMA_IRQ),
> + hisi_ptt_irq, hisi_ptt_isr, IRQF_SHARED,
> + hisi_ptt->name, hisi_ptt);
> +
> + if (ret) {
> + pci_err(pdev, "request irq %d failed",
> + pci_irq_vector(pdev, HISI_PTT_DMA_IRQ));
> + pci_free_irq_vectors(pdev);
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static void hisi_ptt_irq_unregister(struct hisi_ptt *hisi_ptt)
> +{
> + struct pci_dev *pdev = hisi_ptt->pdev;
> +
> + free_irq(pci_irq_vector(pdev, HISI_PTT_DMA_IRQ), hisi_ptt);
> + pci_free_irq_vectors(pdev);
> +}
> +
> +static void hisi_ptt_init_ctrls(struct hisi_ptt *hisi_ptt)
> +{
> + struct pci_dev *pdev = hisi_ptt->pdev, *tpdev;
> + struct pci_bus *child_bus;
> + unsigned long port_mask, bit;
> +
> + hisi_ptt->domain = pci_domain_nr(pdev->bus);
> + hisi_ptt->bus = pdev->bus->number;
> + /*
> + * The mailbox register provides the information about the
> + * root ports which the RCiEP can control and monitor.
> + */
> + port_mask = readl(hisi_ptt->iobase + HISI_PTT_MAILBOX_0);
> +
> + /*
> + * The ports traced by the RCiEP are masked by the register.
> + * Some ports may not exists even if they are masked. We'll check
> + * whether one port is enabled by finding its pci_dev structure
> + * in the device list, and add it and its subordinates
> + * in the available devfns list.
> + */
> + for_each_set_bit(bit, &port_mask, sizeof(port_mask) * 8) {
> + struct per_func_info *func;
> +
> + tpdev = pci_get_domain_bus_and_slot(hisi_ptt->domain,
> + hisi_ptt->bus,
> + PCI_DEVFN(bit, 0));
> + /*
> + * If the root port is not existed in the system,
> + * just skip it and check next one.
> + */
> + if (!tpdev)
> + continue;
> +
> + func = devm_kmalloc(&pdev->dev, sizeof(*func), GFP_KERNEL);
> + if (!func)
> + continue;
> + func->pdev = tpdev;
> + pci_dev_put(tpdev);
> +
> + list_add_tail(&func->list, &hisi_ptt->avail_devfns);
> +
> + /*
> + * The PTT can designate function for trace.
> + * Add the root port's subordinates in the list as we
> + * can specify certain function.
> + */
> + child_bus = tpdev->subordinate;
> + list_for_each_entry(tpdev, &child_bus->devices, bus_list) {
> + func = devm_kmalloc(&pdev->dev, sizeof(*func),
> + GFP_KERNEL);
> + if (!func)
> + continue;
> + func->pdev = tpdev;
> + list_add_tail(&func->list, &hisi_ptt->avail_devfns);
> + }
> + }
> +
> + /* Initialize the target function */
> + if (!list_empty(&hisi_ptt->avail_devfns))
> + hisi_ptt->target_func = list_first_entry(&hisi_ptt->avail_devfns,
> + struct per_func_info, list);
> +
> + /* Initialize trace controls */
> + INIT_LIST_HEAD(&hisi_ptt->trace_ctrl.trace_buf);
> + hisi_ptt->trace_ctrl.buflet_nums = HISI_PTT_DEFAULT_TRACE_BUF_CNT;
> + hisi_ptt->trace_ctrl.buflet_size = available_buflet_size[0];
> + hisi_ptt->trace_ctrl.tr_event = trace_events[0].event_code;
> + hisi_ptt->trace_ctrl.rxtx = trace_rxtx[0].event_code;
> +}
> +
> +static int hisi_ptt_create_debugfs_entries(struct hisi_ptt *hisi_ptt)
> +{
> + struct dentry *tdir;
> + int i;
> +
> + hisi_ptt->debugfs_dir = debugfs_create_dir(hisi_ptt->name,
> + hisi_ptt_debugfs_root);
> + if (IS_ERR(hisi_ptt->debugfs_dir))
> + return -ENOENT;
> +
> + tdir = debugfs_create_dir("tune", hisi_ptt->debugfs_dir);
> + if (IS_ERR(tdir))
> + goto err;
> + for (i = 0; i < ARRAY_SIZE(tune_events); ++i) {
> + struct debugfs_file_desc *tune_file;
> +
> + tune_file = devm_kzalloc(&hisi_ptt->pdev->dev,
> + sizeof(*tune_file), GFP_KERNEL);
> + if (!tune_file)
> + goto err;
> + tune_file->hisi_ptt = hisi_ptt;
> + /* We use tune event string as control file name. */
> + tune_file->name = tune_events[i].name;
> + tune_file->fops = &tune_common_fops;
> + tune_file->index = i;
> + if (IS_ERR(debugfs_create_file(tune_events[i].name, 0600,
> + tdir, tune_file,
> + &tune_common_fops)))
> + goto err;
> + }
> +
> + tdir = debugfs_create_dir("trace", hisi_ptt->debugfs_dir);
> + if (IS_ERR(tdir))
> + goto err;
> + for (i = 0; i < ARRAY_SIZE(trace_entries); ++i) {
> + trace_entries[i].hisi_ptt = hisi_ptt;
> + trace_entries[i].index = i;
> + if (IS_ERR(debugfs_create_file(trace_entries[i].name, 0600,
> + tdir, &trace_entries[i],
> + trace_entries[i].fops)))
> + goto err;
> + }
> +
> + return 0;
> +err:
> + pci_err(hisi_ptt->pdev, "create debugfs entries failed\n");
> + debugfs_remove_recursive(hisi_ptt->debugfs_dir);
> + return -ENOENT;
> +}
> +
> +static int hisi_ptt_probe(struct pci_dev *pdev,
> + const struct pci_device_id *id)
> +{
> + struct hisi_ptt *hisi_ptt;
> + int ret;
> +
> + hisi_ptt = devm_kzalloc(&pdev->dev, sizeof(*hisi_ptt), GFP_KERNEL);
> + if (!hisi_ptt)
> + return -ENOMEM;
> +
> + mutex_init(&hisi_ptt->mutex);
> + INIT_LIST_HEAD(&hisi_ptt->avail_devfns);
> + hisi_ptt->pdev = pdev;
> + /*
> + * Lifetime of pci_dev is longer than hisi_ptt,
> + * so directly reference to the pci name string.
> + */
> + hisi_ptt->name = pci_name(pdev);
> + pci_set_drvdata(pdev, hisi_ptt);
> +
> + ret = pcim_enable_device(pdev);
> + if (ret) {
> + pci_err(pdev, "fail to enable device\n");
> + return ret;
> + }
> +
> + ret = pcim_iomap_regions(pdev, BIT(0), hisi_ptt->name);
> + if (ret) {
> + pci_err(pdev, "fail to remap io memory\n");
> + return ret;
> + }
> +
> + ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
> + if (ret) {
> + pci_err(pdev, "fail to set 64 bit dma mask %d", ret);
> + return ret;
> + }
> + pci_set_master(pdev);
> +
> + ret = hisi_ptt_irq_register(hisi_ptt);
> + if (ret)
> + return ret;
> +
> + hisi_ptt_init_ctrls(hisi_ptt);
> +
> + ret = hisi_ptt_create_debugfs_entries(hisi_ptt);
> + if (ret) {
> + hisi_ptt_irq_unregister(hisi_ptt);
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +void hisi_ptt_remove(struct pci_dev *pdev)
> +{
> + struct hisi_ptt *hisi_ptt = pci_get_drvdata(pdev);
> +
> + hisi_ptt_free_trace_buf(hisi_ptt);
> + debugfs_remove_recursive(hisi_ptt->debugfs_dir);
> + hisi_ptt_irq_unregister(hisi_ptt);
> +}
> +
> +static pci_ers_result_t hisi_ptt_err_detected(struct pci_dev *pdev,
> + pci_channel_state_t state)
> +{
> + /* The PTT device doesn't support error recovery */
> + return PCI_ERS_RESULT_RECOVERED;
> +}
> +
> +static const struct pci_error_handlers hisi_ptt_err_handler = {
> + .error_detected = hisi_ptt_err_detected,
> +};
> +
> +#define PCI_DEVICE_ID_HISI_PTT 0xa250
> +static const struct pci_device_id hisi_ptt_id_tbl[] = {
> + { PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, PCI_DEVICE_ID_HISI_PTT) },
> + { 0, }
> +};
> +
> +static struct pci_driver hisi_ptt_driver = {
> + .name = "hisi_ptt",
> + .id_table = hisi_ptt_id_tbl,
> + .probe = hisi_ptt_probe,
> + .remove = hisi_ptt_remove,
> + .err_handler = &hisi_ptt_err_handler,
> +};
> +
> +static int hisi_ptt_register_debugfs(void)
> +{
> + if (!debugfs_initialized()) {
> + pr_err("error: debugfs uninitialized\n");
> + return -ENOENT;
> + }
> +
> + hisi_ptt_debugfs_root = debugfs_create_dir("hisi_ptt", NULL);
> + if (IS_ERR(hisi_ptt_debugfs_root)) {
> + pr_err("error: fail to create debugfs directory\n");
> + return -ENOENT;
> + }
> +
> + return 0;
> +}
> +
> +static void hisi_ptt_unregister_debugfs(void)
> +{
> + debugfs_remove_recursive(hisi_ptt_debugfs_root);
> +}
> +
> +static int __init hisi_ptt_module_init(void)
> +{
> + int ret;
> +
> + /* The driver cannot work without debugfs entry */
> + ret = hisi_ptt_register_debugfs();
> + if (ret)
> + return ret;
> +
> + ret = pci_register_driver(&hisi_ptt_driver);
> + if (ret) {
> + pr_err("error: fail to register hisi ptt driver\n");
> + hisi_ptt_unregister_debugfs();
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static void __exit hisi_ptt_module_exit(void)
> +{
> + pci_unregister_driver(&hisi_ptt_driver);
> + hisi_ptt_unregister_debugfs();
> +}
> +
> +module_init(hisi_ptt_module_init);
> +module_exit(hisi_ptt_module_exit);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_AUTHOR("Yicong Yang <yangyicong@xxxxxxxxxxxxx>");
> +MODULE_DESCRIPTION("Driver for HiSilicon PCIe tune and trace device");
