HiSilicon PCIe tune and trace device(PTT) is a PCIe Root Complex
integrated Endpoint(RCiEP) device, providing the capability
to dynamically monitor and tune the PCIe traffic parameters(tune),
and trace the TLP headers to the memory(trace).
Add the driver for the device to enable its functions. The driver
will create debugfs directory for each PTT device, and users can
operate the device through the files under its directory.
RFC:
- The hardware interface is not yet finalized.
- The interface to the users is through debugfs, and the usage will
be further illustrated in the document.
- The driver is intended to be put under drivers/hwtracing, where
we think best match the device's function.
Signed-off-by: Yicong Yang <yangyicong@xxxxxxxxxxxxx>
---
Documentation/trace/hisi-ptt.rst | 272 ++++++++
drivers/hwtracing/Kconfig | 2 +
drivers/hwtracing/hisilicon/Kconfig | 8 +
drivers/hwtracing/hisilicon/Makefile | 2 +
drivers/hwtracing/hisilicon/hisi_ptt.c | 1172 ++++++++++++++++++++++++++++++++
5 files changed, 1456 insertions(+)
create mode 100644 Documentation/trace/hisi-ptt.rst
create mode 100644 drivers/hwtracing/hisilicon/Kconfig
create mode 100644 drivers/hwtracing/hisilicon/Makefile
create mode 100644 drivers/hwtracing/hisilicon/hisi_ptt.c
diff --git a/Documentation/trace/hisi-ptt.rst b/Documentation/trace/hisi-ptt.rst
new file mode 100644
index 0000000..c99fbf9
--- /dev/null
+++ b/Documentation/trace/hisi-ptt.rst
@@ -0,0 +1,272 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======================================
+HiSilicon PCIe Tune and Trace device
+======================================
+
+Introduction
+============
+
+HiSilicon PCIe tune and trace device(PTT) is a PCIe Root Complex
+integrated Endpoint(RCiEP) device, providing the capability
+to dynamically monitor and tune the PCIe link's events(tune),
+and trace the TLP headers to the memory(trace). The two functions
+are inpendent, but is recommended to use them together to analyze
+and enhance the PCIe link's performance.
+
+On Hip09, the PCIe root complex is composed of several PCIe cores.
+And each core is composed of several root ports, RCiEPs, and one
+PTT device, like below. The PTT device is capable of tuning and
+tracing the link on and downstream the PCIe core.
+::
+ +--------------Core 0-------+
+ | | [ PTT ] |
+ | | [Root Port]---[Endpoint]
+ | | [Root Port]---[Endpoint]
+ | | [Root Port]---[Endpoint]
+ Root Complex |------Core 1-------+
+ | | [ PTT ] |
+ | | [Root Port]---[ Switch ]---[Endpoint]
+ | | [Root Port]---[Endpoint] `-[Endpoint]
+ | | [Root Port]---[Endpoint]
+ +---------------------------+
+
+The PTT device driver cannot be loaded if debugfs is not mounted.
+Each PTT device will be presented under /sys/kernel/debugfs/hisi_ptt
+as its root directory, with name of its BDF number.
+::
+
+ /sys/kernel/debug/hisi_ptt/<domain>:<bus>:<device>.<function>
+
+Tune
+====
+
+PTT tune is designed for monitoring and adjusting PCIe link parameters(events).
+Currently we support events 4 classes. The scope of the events
+covers the PCIe core with which the PTT device belongs to.
+
+Each event is presented as a file under $(PTT root dir)/$(BDF)/tune, and
+mostly this will be a simple open/read/write/close cycle to tune
+the event.
+::
+ $ cd /sys/kernel/debug/hisi_ptt/$(BDF)/tune
+ $ ls
+ buf_rx_cpld buf_rx_pd dllp_link_ack_freq link_credit_rx_cplh
+ link_credit_rx_ph qos_tx_dp qos_tx_dp
+ $ cat qos_tx_dp
+ 100
+ $ echo 50 > qos_tx_dp
+ $ cat qos_tx_dp
+ 50
+
+Current value(numerical value) of the event can be get by simply
+read the file, and write the desired value to the file to tune.
+Tune multiple events at the same time is not permitted, which means
+you cannot read or write more than one tune file at one time.
+
+1. Link credit control
+----------------------
+
+Following files are provided for tune the link credit events of the PCIe core.
+PCIe link uses credit to control the flow, refer to the PCIe Spec for further
+information.
+
+- link_credit_rx_nph: rx non-posted request headers' credit
+- link_credit_rx_npd: rx non-posted request data payload's credit
+- link_credit_rx_ph: rx posted request headers' credit
+- link_credit_rx_pd: rx posted request data payload's credit
+- link_credit_rx_cplh: rx completion headers' credit
+- link_credit_rx_cpld: rx completion data payload's credit
+
+Note that the event value is not accurate but a probable one to indicate
+the level of each event, for example, perhaps 100 for high level,
+50 for median and 0 for low.
+
+2. Link DLLP control
+--------------------
+
+Following files are provided for tune the link events of DLLP of the PCIe core.
+
+- dllp_link_ack_freq: frequency of DLLP ACKs
+- dllp_link_updatefc_freq: frequency of DLLP flow control updates
+- dllp_link_ssc: spread spectrum control of DLLP link
+
+Note that the event value just indicates a probable level, but not
+accurate.
+
+3. Buffer control
+-----------------
+
+Following files are provided for tune the rx/tx buffer depth of the PCIe core.
+
+- buf_tx_header: buffer depth for tx packets headers
+- buf_tx_data: buffer depth for tx packets data payloads
+- buf_rx_ph: buffer depth for rx posted request packets headers
+- buf_rx_pd: buffer depth for rx posted request packets data payloads
+- buf_rx_nph: buffer depth for rx non-posted request packets headers
+- buf_rx_npd: buffer depth for rx non-posted request packets data payloads
+- buf_rx_cplh: buffer depth for rx completion packets headers
+- buf_rx_cpld: buffer depth for rx completion packets data payloads
+
+Note that the event value just indicates a probable level, but not
+accurate.
+
+4. Data path QoS control
+------------------------
+
+Following files are provided for tune the QoS of the data path of the PCIe core.
+
+- qos_tx_dp: QoS for tx data path
+- qos_rx_dp: QoS for rx data path
+
+Note that the event value just indicates a probable level, but not
+accurate.
+
+Trace
+=====
+
+PTT trace is designed for dumping the TLP headers to the memory, which
+can be used to analyze the transactions and usage condition of the PCIe
+Link. You can choose to trace the headers either by its requester ID,
+or the headers from the link downstream certain root ports, which are
+on the same core of PTT device. It's also support to trace the headers
+of certain type and of certain direction.
+
+In order to start trace, you need to configure the parameters first.
+The parameters files is provided under $(PTT root dir)/$(BDF)/trace.
+::
+ $ cd /sys/kernel/debug/hisi_ptt/$(BDF)/trace
+ $ ls
+ free_buffer filter buflet_nums buflet_size
+ direction type data trace_on
+
+1. filter
+---------
+
+You can configure the filter of TLP headers through the file. The filter
+is provided as BDF numbers of either root port or subordinates, which
+belong to the same PCIe core. You can get the filters available and
+currently configure by read the file, and write the desired BDF to the
+file to set the filters. The default filter is the first root port on
+the core, and write invalid BDF(not in the available list) will return
+a failure.
+::
+ $ echo 0000:80:04.0 > filter
+ $ cat filter
+ 0000:80:00.0 [0000:80:04.0] 0000:81:00.0 0000:81:00.1 0000:82:00.0
+
+2. type
+-------
+
+You can trace the TLP headers of certain types by configure the file.
+Read the file will get available types and current setting, and write
+the desired type to the file to configure. The default type is
+`posted_request` and write types not in the available list will return
+a failure.
+::
+ $ echo completion > type
+ $ cat type
+ posted_request non-posted_request [completion] all
+
+3. direction
+------------
+
+You can trace the TLP headers from certain direction, which is relative
+to the root port or the PCIe core. Read the file to get available
+directions and current configurition, and write the desired direction
+to configure. The default value is `rx` and any invalid direction will
+return a failure. Note `rxtx_no_dma_p2p` means the headers of both
+directions, but not include P2P DMA access.
+::
+ $ echo rxtx > direction
+ $ cat direction
+ rx tx [rxtx] rxtx_no_dma_p2p
+
+4. buflet_size
+--------------
+
+The traced TLP headers will be written to the memory allocated
+by the driver. The hardware accept 4 DMA address with same size,
+and write the buflet sequetially like below. If DMA addr 3 is
+finished and the trace is still on, it will return to addr 0.
+Driver will allocated each DMA buffer (we call it buflet) and
+swap a preallocated one if it has been finished.
+::
+ +->[DMA addr 0]->[DMA addr 1]->[DMA addr 2]->[DMA addr 3]-+
+ +---------------------------------------------------------+
+
+You should both configure the buflet_size and buflet_nums to
+configure the `trace buffer` to receive the TLP headers. The
+total trace buffer size is buflet_size * buflet_nums. Note
+that the trace buffer will not be allocated immediately after you
+configure the parameters, but will be allocated right before
+the trace starts.
+
+This file configures the buflet size. Read the file will get
+available buflet size and size set currently, write the desired
+size to the file to configure. The default size is 2 MiB and any
+invalid size written will return a failure.
+::
+ $ cat buflet_size
+ [2 MiB] 4 MiB 6 MiB 8 MiB 10 MiB
+ $ echo 8 > buflet_size
+ $ cat buflet_size
+ 2 MiB 4 MiB 6 MiB [8 MiB] 10 MiB
+
+5. buflet_nums
+--------------
+
+You can write the desired buflet counts to the file to configure,
+and read the file to get current buflet counts. The default
+value is 64. And any positive value is valid. Note that big value
+may lead to DMA memory allocation failure, and you will not be
+able to start tracing. If it happens, you should consider adjusting
+buflet_nums or buflet_size.
+::
+ $ cat buflet_nums
+ 64
+ $ echo 128 > buflet_nums
+ $ cat buflet_nums
+ 128
+
+6. data
+-------
+
+The file to access the traced data. You can read the file to get the
+binary blob of traced TLP headers. The format of the headers is
+4 Dword length and is just as defined by the PCIe Spec r4.0,
+Sec 2.2.4.1, or 8 Dword length with additional 4 Dword extra
+information.
+
+echo "" > data will free all the trace buffers allocated as well as
+the traced datas.
+
+7. trace_on
+-----------
+
+Start or end the trace by simple writing to the file, and monitor the
+trace status by reading the file.
+::
+ $ echo 1 > trace_on # start trace
+ $ cat trace_on # get the trace status
+ 1
+ $ echo 0 > trace_on # manually end trace
+
+The read value of the trace_on will be auto cleared if the buffer
+allocated is full. 1 indicates the trace is running and 0 for
+stopped. Write any non-zero value to the file can start trace.
+
+8. free_buffer
+--------------
+
+File to indicate the trace buffer status and to manually free the
+trace buffer. The read value of 1 indicates the trace buffer has
+been allocated and exists in the memory, while 0 indicates there
+is no buffer allocated. Write 1 to the file to free the trace
+buffer as well as the traced datas.
+::
+ $ cat free_buffer
+ 1 # indicate the buffer exists
+ $ echo 1 > free_buffer # free the trace buffer
+ $ cat free_buffer
+ 0
diff --git a/drivers/hwtracing/Kconfig b/drivers/hwtracing/Kconfig
index 1308583..e3796b1 100644
--- a/drivers/hwtracing/Kconfig
+++ b/drivers/hwtracing/Kconfig
@@ -5,4 +5,6 @@ source "drivers/hwtracing/stm/Kconfig"
source "drivers/hwtracing/intel_th/Kconfig"
+source "drivers/hwtracing/hisilicon/Kconfig"
+
endmenu
diff --git a/drivers/hwtracing/hisilicon/Kconfig b/drivers/hwtracing/hisilicon/Kconfig
new file mode 100644
index 0000000..95e91b9
--- /dev/null
+++ b/drivers/hwtracing/hisilicon/Kconfig
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config HISI_PTT
+ tristate "HiSilicon PCIe Tune and Trace Device"
+ depends on PCI && HAS_DMA && HAS_IOMEM
+ help
+ HiSilicon PCIe Tune and Trace Device exist as a PCIe iEP
+ device, provides support for PCIe traffic tuning and
+ tracing TLP headers to the memory.
diff --git a/drivers/hwtracing/hisilicon/Makefile b/drivers/hwtracing/hisilicon/Makefile
new file mode 100644
index 0000000..908c09a
--- /dev/null
+++ b/drivers/hwtracing/hisilicon/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_HISI_PTT) += hisi_ptt.o
diff --git a/drivers/hwtracing/hisilicon/hisi_ptt.c b/drivers/hwtracing/hisilicon/hisi_ptt.c
new file mode 100644
index 0000000..c58a3cc
--- /dev/null
+++ b/drivers/hwtracing/hisilicon/hisi_ptt.c
@@ -0,0 +1,1172 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Driver for HiSilicon PCIe tune and trace device
+ *
+ * Copyright (c) 2020 HiSilicon Limited.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/debugfs.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/seq_file.h>
+#include <linux/pci.h>
+
+#define HISI_PTT_CTRL_STR_LEN 40
+#define HISI_PTT_DEFAULT_TRACE_BUF_CNT 64
+
+#define HISI_PTT_RESET_WAIT_MS 1000UL
+
+#define HISI_PTT_IRQ_NUMS 1
+#define HISI_PTT_DMA_IRQ 0
+#define HISI_PTT_DMA_NUMS 4
+
+#define HISI_PTT_TUNING_CTRL 0x0380
+#define HISI_PTT_TUNING_CTRL_CODE GENMASK(3, 0)
+#define HISI_PTT_TUNING_CTRL_EN GENMASK(23, 16)
+#define HISI_PTT_TUNING_DATA 0x0384
+#define HISI_PTT_TUNING_DATA_VAL GENMASK(15, 0)
+#define HISI_PTT_TRACE_ADDR_SIZE 0x0400
+#define HISI_PTT_TRACE_ADDR_BASE_LO_0 0x0410
+#define HISI_PTT_TRACE_ADDR_BASE_HI_0 0x0414
+#define HISI_PTT_TRACE_CTRL 0x0450
+#define HISI_PTT_TRACE_CTRL_EN BIT(0)
+#define HISI_PTT_TRACE_CTRL_RST BIT(1)
+#define HISI_PTT_TRACE_CTRL_RXTX_SEL GENMASK(3, 2)
+#define HISI_PTT_TRACE_CTRL_TYPE_SEL GENMASK(7, 4)
+#define HISI_PTT_TRACE_CTRL_DATA_FORMAT BIT(14)
+#define HISI_PTT_TRACE_CTRL_FILTER_MODE BIT(15)
+#define HISI_PTT_TRACE_CTRL_TARGET_SEL GENMASK(31, 16)
+#define HISI_PTT_TRACE_INT_STAT 0x0490
+#define HISI_PTT_TRACE_INT_STAT_MASK GENMASK(3, 0)
+#define HISI_PTT_TRACE_WR_STS 0x04a0
+#define HISI_PTT_TRACE_WR_STS_WRITE GENMASK(27, 0)
+#define HISI_PTT_TRACE_WR_STS_BUFFER GENMASK(29, 28)
+#define HISI_PTT_TRACE_STS 0x04b0
+#define HISI_PTT_TRACE_IDLE BIT(0)
+#define HISI_PTT_MAILBOX_0 0x07e0
+
+static struct dentry *hisi_ptt_debugfs_root;
+
+struct event_desc {
+ const char *name;
+ u32 event_code;
+};
+
+static struct event_desc tune_events[] = {
+ { "link_credit_rx_nph", 0x1 | (BIT(1) << 16) },
+ { "link_credit_rx_npd", 0x1 | (BIT(2) << 16) },
+ { "link_credit_rx_ph", 0x1 | (BIT(3) << 16) },
+ { "link_credit_rx_pd", 0x1 | (BIT(4) << 16) },
+ { "link_credit_rx_cplh", 0x1 | (BIT(5) << 16) },
+ { "link_credit_rx_cpld", 0x1 | (BIT(6) << 16) },
+ { "dllp_link_ack_freq", 0x2 | (BIT(1) << 16) },
+ { "dllp_link_updatefc_freq", 0x2 | (BIT(2) << 16) },
+ { "dllp_link_ssc", 0x2 | (BIT(2) << 16) },
+ { "buf_tx_header", 0x3 | (BIT(1) << 16) },
+ { "buf_tx_data", 0x3 | (BIT(2) << 16) },
+ { "buf_rx_ph", 0x3 | (BIT(3) << 16) },
+ { "buf_rx_pd", 0x3 | (BIT(4) << 16) },
+ { "buf_rx_nph", 0x3 | (BIT(5) << 16) },
+ { "buf_rx_npd", 0x3 | (BIT(6) << 16) },
+ { "buf_rx_cplh", 0x3 | (BIT(7) << 16) },
+ { "buf_rx_cpld", 0x3 | (BIT(8) << 16) },
+ { "qos_tx_dp", 0x4 | (BIT(1) << 16) },
+ { "qos_rx_dp", 0x4 | (BIT(1) << 16) },
+};
+
+static struct event_desc trace_rxtx[] = {
+ { "rx", 0 },
+ { "tx", 1 },
+ { "rxtx", 2 },
+ { "rxtx_no_dma_p2p", 3 },
+};
+
+static struct event_desc trace_events[] = {
+ { "posted_request", 0 },
+ { "non-posted_request", 2 },
+ { "completion", 4 },
+ { "all", 8 },
+};
+
+static const int available_buflet_size[] = {
+ 0x00200000, /* 2 MiB */
+ 0x00400000, /* 4 MiB */
+ 0x00600000, /* 6 MiB */
+ 0x00800000, /* 8 MiB */
+ 0x00a00000, /* 10 MiB */
+};
+
+struct debugfs_file_desc {
+ struct hisi_ptt *hisi_ptt;
+ const char *name;
+ const struct file_operations *fops;
+ int index;
+};
+
+struct dma_buflet {
+ struct list_head list;
+ dma_addr_t dma;
+ void *addr;
+ int index;
+ u64 size;
+};
+
+struct ptt_trace_ctrl {
+ struct list_head trace_buf;
+ struct dma_buflet *cur;
+ atomic_t status; /* 0:idle, 1:tracing */
+ u64 buflet_nums;
+ u32 buflet_size;
+ u32 tr_event;
+ u32 rxtx;
+};
+
+struct per_func_info {
+ struct list_head list;
+ struct pci_dev *pdev;
+};
+
+struct hisi_ptt {
+ struct ptt_trace_ctrl trace_ctrl;
+ struct per_func_info *target_func;
+ struct list_head avail_devfns;
+ struct dentry *debugfs_dir;
+ void __iomem *iobase;
+ struct pci_dev *pdev;
+ struct mutex mutex; /* protects the hisi_ptt structure */
+ const char *name;
+ u32 tune_event;
+ u32 domain;
+ u32 bus;
+};
+
+static u32 hisi_ptt_tune_data_read(struct hisi_ptt *hisi_ptt)
+{
+ u32 val;
+
+ writel(hisi_ptt->tune_event, hisi_ptt->iobase + HISI_PTT_TUNING_CTRL);
+
+ val = readl(hisi_ptt->iobase + HISI_PTT_TUNING_DATA);
+ val &= HISI_PTT_TUNING_DATA_VAL;
+
+ return val;
+}
+
+static void hisi_ptt_tune_data_write(struct hisi_ptt *hisi_ptt, u32 data)
+{
+ writel(hisi_ptt->tune_event, hisi_ptt->iobase + HISI_PTT_TUNING_CTRL);
+ data &= HISI_PTT_TUNING_DATA_VAL;
+ writel(data, hisi_ptt->iobase + HISI_PTT_TUNING_DATA);
+}
+
+static ssize_t hisi_ptt_tune_common_read(struct file *filp, char __user *buf,
+ size_t count, loff_t *pos)
+{
+ struct debugfs_file_desc *desc = filp->private_data;
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
+ char tbuf[HISI_PTT_CTRL_STR_LEN];
+ int len;
+ u32 val;
+
+ if (!mutex_trylock(&hisi_ptt->mutex))
+ return -EBUSY;
+ hisi_ptt->tune_event = tune_events[desc->index].event_code;
+ val = hisi_ptt_tune_data_read(hisi_ptt);
+ mutex_unlock(&hisi_ptt->mutex);
+
+ len = snprintf(tbuf, HISI_PTT_CTRL_STR_LEN,
+ "%d\n", val);
+
+ return simple_read_from_buffer(buf, count, pos, tbuf, len);
+}
+
+static ssize_t hisi_ptt_tune_common_write(struct file *filp,
+ const char __user *buf, size_t count, loff_t *pos)
+{
+ struct debugfs_file_desc *desc = filp->private_data;
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
+ char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
+ int len, val;
+
+ len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN - 1,
+ pos, buf, count);
+ if (len < 0)
+ return -EINVAL;
+ cp = strchr(tbuf, '\n');
+ if (cp)
+ *cp = '\0';
+ if (kstrtouint(tbuf, 0, &val))
+ return -EINVAL;
+
+ if (!mutex_trylock(&hisi_ptt->mutex))
+ return -EBUSY;
+ hisi_ptt->tune_event = tune_events[desc->index].event_code;
+ hisi_ptt_tune_data_write(hisi_ptt, val);
+ mutex_unlock(&hisi_ptt->mutex);
+
+ return count;
+}
+
+static const struct file_operations tune_common_fops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = hisi_ptt_tune_common_read,
+ .write = hisi_ptt_tune_common_write,
+ .llseek = no_llseek,
+};
+
+static void hisi_ptt_free_trace_buf(struct hisi_ptt *hisi_ptt)
+{
+ struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
+ struct device *dev = &hisi_ptt->pdev->dev;
+ struct dma_buflet *buflet, *tbuflet;
+
+ if (list_empty(&ctrl->trace_buf))
+ return;
+
+ list_for_each_entry_safe(buflet, tbuflet, &ctrl->trace_buf, list) {
+ dma_free_coherent(dev, buflet->size, buflet->addr, buflet->dma);
+ list_del(&buflet->list);
+ kfree(buflet);
+ }
+}
+
+static int hisi_ptt_alloc_trace_buf(struct hisi_ptt *hisi_ptt)
+{
+ struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
+ struct device *dev = &hisi_ptt->pdev->dev;
+ struct dma_buflet *buflet;
+ int i, ret;
+
+ /* Make sure the trace buffer is empty before allocating */
+ if (!list_empty(&ctrl->trace_buf))
+ hisi_ptt_free_trace_buf(hisi_ptt);
+
+ for (i = 0; i < ctrl->buflet_nums; ++i) {
+ buflet = kzalloc(sizeof(*buflet), GFP_KERNEL);
+ if (!buflet) {
+ ret = -ENOMEM;
+ goto err;
+ }
+ buflet->addr = dma_alloc_coherent(dev, ctrl->buflet_size,
+ &buflet->dma, GFP_KERNEL);
+ if (!buflet->addr) {
+ kfree(buflet);
+ ret = -ENOMEM;
+ goto err;
+ }
+ buflet->index = i;
+ buflet->size = ctrl->buflet_size;
+ list_add_tail(&buflet->list, &ctrl->trace_buf);
+ }
+
+ return 0;
+err:
+ hisi_ptt_free_trace_buf(hisi_ptt);
+ return ret;
+}
+
+static void hisi_ptt_trace_end(struct hisi_ptt *hisi_ptt)
+{
+ writel(0, hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
+ atomic_dec(&hisi_ptt->trace_ctrl.status);
+ hisi_ptt->trace_ctrl.cur = NULL;
+}
+
+static int hisi_ptt_trace_start(struct hisi_ptt *hisi_ptt)
+{
+ struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
+ struct pci_dev *pdev, *rp;
+ u32 val;
+ int i;
+
+ if (!hisi_ptt->target_func) {
+ pci_err(hisi_ptt->pdev, "No available root port/function\n");
+ return -ENODEV;
+ }
+ pdev = hisi_ptt->target_func->pdev;
+ rp = pcie_find_root_port(pdev);
+
+ val = readl(hisi_ptt->iobase + HISI_PTT_TRACE_STS);
+ if (!(val & HISI_PTT_TRACE_IDLE)) /* Trace has already started */
+ return -EBUSY;
+
+ /* reset the DMA before start tracing */
+ val = readl(hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
+ val |= HISI_PTT_TRACE_CTRL_RST;
+ writel(val, hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
+
+ msleep(HISI_PTT_RESET_WAIT_MS);
+
+ val = readl(hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
+ val &= ~HISI_PTT_TRACE_CTRL_RST;
+ writel(val, hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
+
+ /* clear the interrupt status */
+ writel(0, hisi_ptt->iobase + HISI_PTT_TRACE_INT_STAT);
+
+ for (i = 0; i < HISI_PTT_DMA_NUMS; ++i) {
+ if (!ctrl->cur)
+ ctrl->cur = list_first_entry(&ctrl->trace_buf, struct dma_buflet, list);
+ else
+ ctrl->cur = list_next_entry(ctrl->cur, list);
+
+ writel(ctrl->cur->dma,
+ hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_BASE_LO_0 + i * 8);
+ writel(ctrl->cur->dma >> 32,
+ hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_BASE_HI_0 + i * 8);
+ }
+ writel(ctrl->buflet_size, hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_SIZE);
+
+ /* set the trace control register */
+ val = 0;
+ val |= FIELD_PREP(HISI_PTT_TRACE_CTRL_TYPE_SEL, ctrl->tr_event);
+ val |= FIELD_PREP(HISI_PTT_TRACE_CTRL_RXTX_SEL, ctrl->rxtx);
+ /*
+ * The TLP headers can be filtered either by the root port,
+ * or by the requester ID.
+ */
+ if (pci_pcie_type(pdev) != PCI_EXP_TYPE_ROOT_PORT) {
+ val |= FIELD_PREP(HISI_PTT_TRACE_CTRL_TARGET_SEL, pdev->bus->number << 8 | pdev->devfn);
+ val |= HISI_PTT_TRACE_CTRL_FILTER_MODE;
+ } else {
+ val |= FIELD_PREP(HISI_PTT_TRACE_CTRL_TARGET_SEL, BIT(PCI_SLOT(rp->devfn)));
+ }
+
+ val |= HISI_PTT_TRACE_CTRL_EN;
+ writel(val, hisi_ptt->iobase + HISI_PTT_TRACE_CTRL);
+
+ atomic_inc(&ctrl->status);
+
+ return 0;
+}
+
+#define TRACE_ATTR(__name) \
+static int __name ## _open(struct inode *inode, struct file *filp) \
+{ \
+ struct debugfs_file_desc *desc = inode->i_private; \
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt; \
+ if (!mutex_trylock(&hisi_ptt->mutex)) \
+ return -EBUSY; \
+ return single_open(filp, __name ## _show, hisi_ptt); \
+} \
+static int __name ## _release(struct inode *inode, struct file *filp) \
+{ \
+ struct debugfs_file_desc *desc = inode->i_private; \
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt; \
+ mutex_unlock(&hisi_ptt->mutex); \
+ return seq_release(inode, filp); \
+} \
+static const struct file_operations __name ## _fops = { \
+ .owner = THIS_MODULE, \
+ .open = __name ## _open, \
+ .read = seq_read, \
+ .write = __name ## _write, \
+ .llseek = seq_lseek, \
+ .release = __name ## _release, \
+}
+
+static int set_filter_show(struct seq_file *m, void *v)
+{
+ struct hisi_ptt *hisi_ptt = m->private;
+ struct per_func_info *func;
+
+ list_for_each_entry(func, &hisi_ptt->avail_devfns, list) {
+ struct pci_dev *pdev = func->pdev;
+
+ if (func == hisi_ptt->target_func)
+ seq_printf(m, "[%04x:%02x:%02x.%d]\n",
+ pci_domain_nr(pdev->bus),
+ pdev->bus->number, PCI_SLOT(pdev->devfn),
+ PCI_FUNC(pdev->devfn));
+ else
+ seq_printf(m, "%04x:%02x:%02x.%d\n",
+ pci_domain_nr(pdev->bus),
+ pdev->bus->number, PCI_SLOT(pdev->devfn),
+ PCI_FUNC(pdev->devfn));
+ }
+
+ return 0;
+}
+
+static ssize_t set_filter_write(struct file *filp, const char __user *buf,
+ size_t count, loff_t *pos)
+{
+ struct seq_file *m = filp->private_data;
+ struct hisi_ptt *hisi_ptt = m->private;
+ char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
+ u32 domain, bus, dev, fn, devfn;
+ struct per_func_info *tfunc;
+ int len, num;
+ bool found = false;
+
+ if (list_empty(&hisi_ptt->avail_devfns))
+ return -EINVAL;
+ len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN - 1, pos, buf, count);
+ if (len < 0)
+ return -EINVAL;
+ cp = strchr(tbuf, '\n');
+ if (cp)
+ *cp = '\0';
+
+ /*
+ * the input should be like 0000:80:01.1, etc. Parse it
+ * and check whether it's in the available func list.
+ */
+ num = sscanf(tbuf, "%04x:%02x:%02x.%d", &domain, &bus, &dev, &fn);
+ if (num != 4)
+ return -EINVAL;
+
+ devfn = PCI_DEVFN(dev, fn);
+ list_for_each_entry(tfunc, &hisi_ptt->avail_devfns, list) {
+ struct pci_dev *pdev = tfunc->pdev;
+
+ if (domain == pci_domain_nr(pdev->bus) &&
+ bus == pdev->bus->number && devfn == pdev->devfn) {
+ hisi_ptt->target_func = tfunc;
+ found = true;
+ break;
+ }
+ }
+
+ if (!found)
+ return -EINVAL;
+
+ return count;
+}
+TRACE_ATTR(set_filter);
+
+static int set_direction_show(struct seq_file *m, void *v)
+{
+ struct hisi_ptt *hisi_ptt = m->private;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(trace_rxtx); ++i) {
+ if (hisi_ptt->trace_ctrl.rxtx == trace_rxtx[i].event_code)
+ seq_printf(m, "[%s] ", trace_rxtx[i].name);
+ else
+ seq_printf(m, "%s ", trace_rxtx[i].name);
+ }
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static ssize_t set_direction_write(struct file *filp, const char __user *buf,
+ size_t count, loff_t *pos)
+{
+ struct seq_file *m = filp->private_data;
+ struct hisi_ptt *hisi_ptt = m->private;
+ char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
+ bool set = false;
+ int len, i;
+
+ len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN, pos, buf, count);
+ if (len < 0)
+ return -EINVAL;
+ cp = strchr(tbuf, '\n');
+ if (cp)
+ *cp = '\0';
+
+ for (i = 0; i < ARRAY_SIZE(trace_rxtx); ++i) {
+ if (!strcmp(tbuf, trace_rxtx[i].name)) {
+ hisi_ptt->trace_ctrl.rxtx = trace_rxtx[i].event_code;
+ set = true;
+ break;
+ }
+ }
+
+ if (!set)
+ return -EINVAL;
+
+ return 0;
+}
+TRACE_ATTR(set_direction);
+
+static int set_trace_type_show(struct seq_file *m, void *v)
+{
+ struct hisi_ptt *hisi_ptt = m->private;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(trace_events); ++i) {
+ if (hisi_ptt->trace_ctrl.tr_event == trace_events[i].event_code)
+ seq_printf(m, "[%s] ", trace_events[i].name);
+ else
+ seq_printf(m, "%s ", trace_events[i].name);
+ }
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static ssize_t set_trace_type_write(struct file *filp, const char __user *buf,
+ size_t count, loff_t *pos)
+{
+ struct seq_file *m = filp->private_data;
+ struct hisi_ptt *hisi_ptt = m->private;
+ char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
+ bool set = false;
+ int len, i;
+
+ len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN, pos, buf, count);
+ if (len < 0)
+ return -EINVAL;
+ cp = strchr(tbuf, '\n');
+ if (cp)
+ *cp = '\0';
+
+ for (i = 0; i < ARRAY_SIZE(trace_events); ++i) {
+ if (!strcmp(tbuf, trace_events[i].name)) {
+ hisi_ptt->trace_ctrl.tr_event = trace_events[i].event_code;
+ set = true;
+ break;
+ }
+ }
+
+ if (!set)
+ return -EINVAL;
+
+ return count;
+}
+TRACE_ATTR(set_trace_type);
+
+static int trace_on_get(void *data, u64 *val)
+{
+ struct debugfs_file_desc *desc = data;
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
+
+ if (!mutex_trylock(&hisi_ptt->mutex))
+ return -EBUSY;
+
+ *val = atomic_read(&hisi_ptt->trace_ctrl.status);
+
+ mutex_unlock(&hisi_ptt->mutex);
+
+ return 0;
+}
+
+static int trace_on_set(void *data, u64 val)
+{
+ struct debugfs_file_desc *desc = data;
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
+ int ret = 0;
+
+ if (!mutex_trylock(&hisi_ptt->mutex))
+ return -EBUSY;
+
+ if (val) {
+ if (atomic_read(&hisi_ptt->trace_ctrl.status))
+ goto out;
+ if (hisi_ptt_alloc_trace_buf(hisi_ptt)) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ if (hisi_ptt_trace_start(hisi_ptt)) {
+ ret = -EBUSY;
+ goto out;
+ }
+ } else {
+ if (!atomic_read(&hisi_ptt->trace_ctrl.status))
+ goto out;
+ hisi_ptt_trace_end(hisi_ptt);
+ }
+
+out:
+ hisi_ptt_free_trace_buf(hisi_ptt);
+ mutex_unlock(&hisi_ptt->mutex);
+ return ret;
+}
+DEFINE_DEBUGFS_ATTRIBUTE(trace_on_fops, trace_on_get,
+ trace_on_set, "%lld\n");
+
+static int set_trace_buf_nums_get(void *data, u64 *val)
+{
+ struct debugfs_file_desc *desc = data;
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
+
+ if (!mutex_trylock(&hisi_ptt->mutex))
+ return -EBUSY;
+
+ *val = hisi_ptt->trace_ctrl.buflet_nums;
+
+ mutex_unlock(&hisi_ptt->mutex);
+ return 0;
+}
+
+static int set_trace_buf_nums_set(void *data, u64 val)
+{
+ struct debugfs_file_desc *desc = data;
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
+
+ if (!mutex_trylock(&hisi_ptt->mutex))
+ return -EBUSY;
+
+ hisi_ptt->trace_ctrl.buflet_nums = val;
+
+ mutex_unlock(&hisi_ptt->mutex);
+ return 0;
+}
+DEFINE_DEBUGFS_ATTRIBUTE(set_trace_buf_nums_fops, set_trace_buf_nums_get,
+ set_trace_buf_nums_set, "%lld\n");
+
+static int set_trace_buflet_size_show(struct seq_file *m, void *v)
+{
+ struct hisi_ptt *hisi_ptt = m->private;
+ struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(available_buflet_size); ++i) {
+ if (ctrl->buflet_size == available_buflet_size[i]) {
+ seq_printf(m, "[%dMiB] ",
+ available_buflet_size[i] >> 20);
+ continue;
+ }
+ seq_printf(m, "%dMiB ", available_buflet_size[i] >> 20);
+ }
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static ssize_t set_trace_buflet_size_write(struct file *filp,
+ const char __user *buf, size_t count, loff_t *pos)
+{
+ struct seq_file *m = filp->private_data;
+ struct hisi_ptt *hisi_ptt = m->private;
+ struct ptt_trace_ctrl *ctrl = &hisi_ptt->trace_ctrl;
+ char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
+ int i, len, size, set = 0;
+
+ len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN, pos, buf, count);
+ cp = strchr(tbuf, '\n');
+ if (cp)
+ *cp = '\0';
+ if (kstrtouint(tbuf, 0, &size))
+ return -EINVAL;
+ size <<= 20;
+
+ for (i = 0; i < ARRAY_SIZE(available_buflet_size); ++i) {
+ if (available_buflet_size[i] == size) {
+ ctrl->buflet_size = size;
+ set = 1;
+ break;
+ }
+ }
+
+ if (!set)
+ return -EINVAL;
+
+ return count;
+}
+TRACE_ATTR(set_trace_buflet_size);
+
+static int free_trace_buf_get(void *data, u64 *val)
+{
+ struct debugfs_file_desc *desc = data;
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
+
+ if (!mutex_trylock(&hisi_ptt->mutex))
+ return -EBUSY;
+
+ *val = list_empty(&hisi_ptt->trace_ctrl.trace_buf);
+
+ mutex_unlock(&hisi_ptt->mutex);
+ return 0;
+}
+
+static int free_trace_buf_set(void *data, u64 val)
+{
+ struct debugfs_file_desc *desc = data;
+ struct hisi_ptt *hisi_ptt = desc->hisi_ptt;
+ int ret = 0;
+
+ if (!mutex_trylock(&hisi_ptt->mutex))
+ return -EBUSY;
+
+ if (!list_empty(&hisi_ptt->trace_ctrl.trace_buf)) {
+ if (atomic_read(&hisi_ptt->trace_ctrl.status))
+ ret = -EBUSY;
+ else
+ hisi_ptt_free_trace_buf(hisi_ptt);
+ }
+
+ mutex_unlock(&hisi_ptt->mutex);
+ return ret;
+}
+DEFINE_DEBUGFS_ATTRIBUTE(free_trace_buf_fops, free_trace_buf_get,
+ free_trace_buf_set, "%lld\n");
+
+static void *trace_data_next(struct seq_file *m, void *v, loff_t *pos)
+{
+ struct hisi_ptt *hisi_ptt = m->private;
+ struct dma_buflet *buflet = v;
+
+ (*pos)++;
+
+ if (!list_is_last(&buflet->list, &hisi_ptt->trace_ctrl.trace_buf))
+ return list_next_entry(buflet, list);
+
+ return NULL;
+}
+
+static void *trace_data_start(struct seq_file *m, loff_t *pos)
+{
+ struct hisi_ptt *hisi_ptt = m->private;
+ struct dma_buflet *buflet;
+ loff_t off = 0;
+
+ buflet = list_first_entry(&hisi_ptt->trace_ctrl.trace_buf, struct dma_buflet, list);
+ while (off < *pos)
+ buflet = trace_data_next(m, buflet, &off);
+
+ return buflet;
+}
+
+static void trace_data_stop(struct seq_file *m, void *p)
+{
+ /* Nothing to do, only a stub */
+}
+
+static int trace_data_show(struct seq_file *m, void *v)
+{
+ struct dma_buflet *buflet = v;
+
+ if (buflet)
+ seq_write(m, buflet->addr, buflet->size);
+
+ return 0;
+}
+
+static const struct seq_operations trace_data_seq_ops = {
+ .start = trace_data_start,
+ .next = trace_data_next,
+ .stop = trace_data_stop,
+ .show = trace_data_show,
+};
+
+static int trace_data_open(struct inode *inode, struct file *filep)
+{
+ struct hisi_ptt *hisi_ptt = inode->i_private;
+ struct seq_file *m;
+ int ret;
+
+ /*
+ * Check the trace status, we cannot read the
+ * data if the trace is still on. Then hold the
+ * lock when reading the traced data.
+ */
+ if (atomic_read(&hisi_ptt->trace_ctrl.status) ||
+ !mutex_trylock(&hisi_ptt->mutex))
+ return -EBUSY;
+
+ if (list_empty(&hisi_ptt->trace_ctrl.trace_buf)) {
+ ret = -ENOTTY;
+ goto err;
+ }
+
+ ret = seq_open(filep, &trace_data_seq_ops);
+ if (ret)
+ goto err;
+
+ m = filep->private_data;
+ m->private = hisi_ptt;
+
+ return 0;
+err:
+ mutex_unlock(&hisi_ptt->mutex);
+ return ret;
+}
+
+static ssize_t trace_data_write(struct file *filp, const char __user *buf,
+ size_t count, loff_t *off)
+{
+ struct seq_file *m = filp->private_data;
+ struct hisi_ptt *hisi_ptt = m->private;
+ char tbuf[HISI_PTT_CTRL_STR_LEN], *cp;
+ int len;
+
+ len = simple_write_to_buffer(tbuf, HISI_PTT_CTRL_STR_LEN, off, buf, count);
+ cp = strchr(tbuf, '\n');
+ if (cp)
+ *cp = '\0';
+
+ /* Free the trace buffer when echo "" > trace_data */
+ if (!strlen(tbuf) && !list_empty(&hisi_ptt->trace_ctrl.trace_buf)) {
+ if (atomic_read(&hisi_ptt->trace_ctrl.status))
+ return -EBUSY;
+ hisi_ptt_free_trace_buf(hisi_ptt);
+ }
+
+ return count;
+}
+
+static int trace_data_release(struct inode *inode, struct file *filp)
+{
+ struct hisi_ptt *hisi_ptt = inode->i_private;
+
+ mutex_unlock(&hisi_ptt->mutex);
+
+ return seq_release(inode, filp);
+}
+
+static const struct file_operations trace_data_fops = {
+ .owner = THIS_MODULE,
+ .open = trace_data_open,
+ .read = seq_read,
+ .write = trace_data_write,
+ .llseek = no_llseek,
+ .release = trace_data_release,
+};
+
+static struct debugfs_file_desc trace_entries[] = {
+ { NULL, "filter", &set_filter_fops, 0 },
+ { NULL, "direction", &set_direction_fops, 0 },
+ { NULL, "type", &set_trace_type_fops, 0 },
+ { NULL, "trace_on", &trace_on_fops, 0 },
+ { NULL, "buf_nums", &set_trace_buf_nums_fops, 0 },
+ { NULL, "buflet_size", &set_trace_buflet_size_fops, 0 },
+ { NULL, "free_buffer", &free_trace_buf_fops, 0 },
+ { NULL, "data", &trace_data_fops, 0 },
+};
+
+irqreturn_t hisi_ptt_isr(int irq, void *context)
+{
+ struct hisi_ptt *hisi_ptt = context;
+ struct dma_buflet *next, *cur;
+ u32 val, buf_idx;
+
+ val = readl(hisi_ptt->iobase + HISI_PTT_TRACE_INT_STAT);
+ buf_idx = __ffs(val & HISI_PTT_TRACE_INT_STAT_MASK);
+ /*
+ * Check whether the trace buffer is full. Stop tracing
+ * when the last DMA buffer is finished. Otherwise, assign
+ * the address of next buflet to the DMA register.
+ */
+ cur = hisi_ptt->trace_ctrl.cur;
+ if (list_is_last(&cur->list, &hisi_ptt->trace_ctrl.trace_buf)) {
+ if ((val & HISI_PTT_TRACE_INT_STAT_MASK) == HISI_PTT_TRACE_INT_STAT_MASK)
+ hisi_ptt_trace_end(hisi_ptt);
+ } else {
+ next = list_next_entry(cur, list);
+ writel(next->dma, hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_BASE_LO_0 + buf_idx * 8);
+ writel(next->dma >> 32, hisi_ptt->iobase + HISI_PTT_TRACE_ADDR_BASE_HI_0 + buf_idx * 8);
+ hisi_ptt->trace_ctrl.cur = next;
+ val &= ~BIT(buf_idx);
+ writel(val, hisi_ptt->iobase + HISI_PTT_TRACE_INT_STAT);
+ }
+
+ return IRQ_HANDLED;
+}
+
+irqreturn_t hisi_ptt_irq(int irq, void *context)
+{
+ struct hisi_ptt *hisi_ptt = context;
+ u32 status;
+
+ status = readl(hisi_ptt->iobase + HISI_PTT_TRACE_INT_STAT);
+ if (!(status & HISI_PTT_TRACE_INT_STAT_MASK))
+ return IRQ_NONE;
+
+ return IRQ_WAKE_THREAD;
+}
+
+static int hisi_ptt_irq_register(struct hisi_ptt *hisi_ptt)
+{
+ struct pci_dev *pdev = hisi_ptt->pdev;
+ int ret;
+
+ ret = pci_alloc_irq_vectors(pdev, HISI_PTT_IRQ_NUMS, HISI_PTT_IRQ_NUMS,
+ PCI_IRQ_MSI);
+ if (ret < 0) {
+ pci_err(pdev, "allocate irq vector failed %d", ret);
+ return ret;
+ }
+
+ ret = request_threaded_irq(pci_irq_vector(pdev, HISI_PTT_DMA_IRQ),
+ hisi_ptt_irq, hisi_ptt_isr, IRQF_SHARED,
+ hisi_ptt->name, hisi_ptt);
+
+ if (ret) {
+ pci_err(pdev, "request irq %d failed",
+ pci_irq_vector(pdev, HISI_PTT_DMA_IRQ));
+ pci_free_irq_vectors(pdev);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void hisi_ptt_irq_unregister(struct hisi_ptt *hisi_ptt)
+{
+ struct pci_dev *pdev = hisi_ptt->pdev;
+
+ free_irq(pci_irq_vector(pdev, HISI_PTT_DMA_IRQ), hisi_ptt);
+ pci_free_irq_vectors(pdev);
+}
+
+static void hisi_ptt_init_ctrls(struct hisi_ptt *hisi_ptt)
+{
+ struct pci_dev *pdev = hisi_ptt->pdev, *tpdev;
+ struct pci_bus *child_bus;
+ unsigned long port_mask, bit;
+
+ hisi_ptt->domain = pci_domain_nr(pdev->bus);
+ hisi_ptt->bus = pdev->bus->number;
+ /*
+ * The mailbox register provides the information about the
+ * root ports which the RCiEP can control and monitor.
+ */
+ port_mask = readl(hisi_ptt->iobase + HISI_PTT_MAILBOX_0);
+
+ /*
+ * The ports traced by the RCiEP are masked by the register.
+ * Some ports may not exists even if they are masked. We'll check
+ * whether one port is enabled by finding its pci_dev structure
+ * in the device list, and add it and its subordinates
+ * in the available devfns list.
+ */
+ for_each_set_bit(bit, &port_mask, sizeof(port_mask) * 8) {
+ struct per_func_info *func;
+
+ tpdev = pci_get_domain_bus_and_slot(hisi_ptt->domain,
+ hisi_ptt->bus,
+ PCI_DEVFN(bit, 0));
+ /*
+ * If the root port is not existed in the system,
+ * just skip it and check next one.
+ */
+ if (!tpdev)
+ continue;
+
+ func = devm_kmalloc(&pdev->dev, sizeof(*func), GFP_KERNEL);
+ if (!func)
+ continue;
+ func->pdev = tpdev;
+ pci_dev_put(tpdev);
+
+ list_add_tail(&func->list, &hisi_ptt->avail_devfns);
+
+ /*
+ * The PTT can designate function for trace.
+ * Add the root port's subordinates in the list as we
+ * can specify certain function.
+ */
+ child_bus = tpdev->subordinate;
+ list_for_each_entry(tpdev, &child_bus->devices, bus_list) {
+ func = devm_kmalloc(&pdev->dev, sizeof(*func),
+ GFP_KERNEL);
+ if (!func)
+ continue;
+ func->pdev = tpdev;
+ list_add_tail(&func->list, &hisi_ptt->avail_devfns);
+ }
+ }
+
+ /* Initialize the target function */
+ if (!list_empty(&hisi_ptt->avail_devfns))
+ hisi_ptt->target_func = list_first_entry(&hisi_ptt->avail_devfns,
+ struct per_func_info, list);
+
+ /* Initialize trace controls */
+ INIT_LIST_HEAD(&hisi_ptt->trace_ctrl.trace_buf);
+ hisi_ptt->trace_ctrl.buflet_nums = HISI_PTT_DEFAULT_TRACE_BUF_CNT;
+ hisi_ptt->trace_ctrl.buflet_size = available_buflet_size[0];
+ hisi_ptt->trace_ctrl.tr_event = trace_events[0].event_code;
+ hisi_ptt->trace_ctrl.rxtx = trace_rxtx[0].event_code;
+}
+
+static int hisi_ptt_create_debugfs_entries(struct hisi_ptt *hisi_ptt)
+{
+ struct dentry *tdir;
+ int i;
+
+ hisi_ptt->debugfs_dir = debugfs_create_dir(hisi_ptt->name,
+ hisi_ptt_debugfs_root);
+ if (IS_ERR(hisi_ptt->debugfs_dir))
+ return -ENOENT;
+
+ tdir = debugfs_create_dir("tune", hisi_ptt->debugfs_dir);
+ if (IS_ERR(tdir))
+ goto err;
+ for (i = 0; i < ARRAY_SIZE(tune_events); ++i) {
+ struct debugfs_file_desc *tune_file;
+
+ tune_file = devm_kzalloc(&hisi_ptt->pdev->dev,
+ sizeof(*tune_file), GFP_KERNEL);
+ if (!tune_file)
+ goto err;
+ tune_file->hisi_ptt = hisi_ptt;
+ /* We use tune event string as control file name. */
+ tune_file->name = tune_events[i].name;
+ tune_file->fops = &tune_common_fops;
+ tune_file->index = i;
+ if (IS_ERR(debugfs_create_file(tune_events[i].name, 0600,
+ tdir, tune_file,
+ &tune_common_fops)))
+ goto err;
+ }
+
+ tdir = debugfs_create_dir("trace", hisi_ptt->debugfs_dir);
+ if (IS_ERR(tdir))
+ goto err;
+ for (i = 0; i < ARRAY_SIZE(trace_entries); ++i) {
+ trace_entries[i].hisi_ptt = hisi_ptt;
+ trace_entries[i].index = i;
+ if (IS_ERR(debugfs_create_file(trace_entries[i].name, 0600,
+ tdir, &trace_entries[i],
+ trace_entries[i].fops)))
+ goto err;
+ }
+
+ return 0;
+err:
+ pci_err(hisi_ptt->pdev, "create debugfs entries failed\n");
+ debugfs_remove_recursive(hisi_ptt->debugfs_dir);
+ return -ENOENT;
+}
+
+static int hisi_ptt_probe(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ struct hisi_ptt *hisi_ptt;
+ int ret;
+
+ hisi_ptt = devm_kzalloc(&pdev->dev, sizeof(*hisi_ptt), GFP_KERNEL);
+ if (!hisi_ptt)
+ return -ENOMEM;
+
+ mutex_init(&hisi_ptt->mutex);
+ INIT_LIST_HEAD(&hisi_ptt->avail_devfns);
+ hisi_ptt->pdev = pdev;
+ /*
+ * Lifetime of pci_dev is longer than hisi_ptt,
+ * so directly reference to the pci name string.
+ */
+ hisi_ptt->name = pci_name(pdev);
+ pci_set_drvdata(pdev, hisi_ptt);
+
+ ret = pcim_enable_device(pdev);
+ if (ret) {
+ pci_err(pdev, "fail to enable device\n");
+ return ret;
+ }
+
+ ret = pcim_iomap_regions(pdev, BIT(0), hisi_ptt->name);
+ if (ret) {
+ pci_err(pdev, "fail to remap io memory\n");
+ return ret;
+ }
+
+ ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
+ if (ret) {
+ pci_err(pdev, "fail to set 64 bit dma mask %d", ret);
+ return ret;
+ }
+ pci_set_master(pdev);
+
+ ret = hisi_ptt_irq_register(hisi_ptt);
+ if (ret)
+ return ret;
+
+ hisi_ptt_init_ctrls(hisi_ptt);
+
+ ret = hisi_ptt_create_debugfs_entries(hisi_ptt);
+ if (ret) {
+ hisi_ptt_irq_unregister(hisi_ptt);
+ return ret;
+ }
+
+ return 0;
+}
+
+void hisi_ptt_remove(struct pci_dev *pdev)
+{
+ struct hisi_ptt *hisi_ptt = pci_get_drvdata(pdev);
+
+ hisi_ptt_free_trace_buf(hisi_ptt);
+ debugfs_remove_recursive(hisi_ptt->debugfs_dir);
+ hisi_ptt_irq_unregister(hisi_ptt);
+}
+
+static pci_ers_result_t hisi_ptt_err_detected(struct pci_dev *pdev,
+ pci_channel_state_t state)
+{
+ /* The PTT device doesn't support error recovery */
+ return PCI_ERS_RESULT_RECOVERED;
+}
+
+static const struct pci_error_handlers hisi_ptt_err_handler = {
+ .error_detected = hisi_ptt_err_detected,
+};
+
+#define PCI_DEVICE_ID_HISI_PTT 0xa250
+static const struct pci_device_id hisi_ptt_id_tbl[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, PCI_DEVICE_ID_HISI_PTT) },
+ { 0, }
+};
+
+static struct pci_driver hisi_ptt_driver = {
+ .name = "hisi_ptt",
+ .id_table = hisi_ptt_id_tbl,
+ .probe = hisi_ptt_probe,
+ .remove = hisi_ptt_remove,
+ .err_handler = &hisi_ptt_err_handler,
+};
+
+static int hisi_ptt_register_debugfs(void)
+{
+ if (!debugfs_initialized()) {
+ pr_err("error: debugfs uninitialized\n");
+ return -ENOENT;
+ }
+
+ hisi_ptt_debugfs_root = debugfs_create_dir("hisi_ptt", NULL);
+ if (IS_ERR(hisi_ptt_debugfs_root)) {
+ pr_err("error: fail to create debugfs directory\n");
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
+static void hisi_ptt_unregister_debugfs(void)
+{
+ debugfs_remove_recursive(hisi_ptt_debugfs_root);
+}
+
+static int __init hisi_ptt_module_init(void)
+{
+ int ret;
+
+ /* The driver cannot work without debugfs entry */
+ ret = hisi_ptt_register_debugfs();
+ if (ret)
+ return ret;
+
+ ret = pci_register_driver(&hisi_ptt_driver);
+ if (ret) {
+ pr_err("error: fail to register hisi ptt driver\n");
+ hisi_ptt_unregister_debugfs();
+ return ret;
+ }
+
+ return 0;
+}
+
+static void __exit hisi_ptt_module_exit(void)
+{
+ pci_unregister_driver(&hisi_ptt_driver);
+ hisi_ptt_unregister_debugfs();
+}
+
+module_init(hisi_ptt_module_init);
+module_exit(hisi_ptt_module_exit);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Yicong Yang <yangyicong@xxxxxxxxxxxxx>");
+MODULE_DESCRIPTION("Driver for HiSilicon PCIe tune and trace device");
--
2.8.1
