On 11/20/2017 06:01 AM, Tvrtko Ursulin wrote:
Hi,
On 16/11/2017 20:42, Michael Sartain wrote:
On Wed, Sep 6, 2017, at 02:09 AM, Chris Wilson wrote:
Quoting Daniel Vetter (2017-09-06 08:46:50)
Hi Pierre,
On Tue, Sep 5, 2017 at 11:15 PM, Pierre-Loup A. Griffais
<pgriffais@xxxxxxxxxxxxxxxxx> wrote:
Hi Daniel,
In the past couple of months we've been working on gpuvis, a GPU
tracing
tool similar to GPUView on Windows. It's lower level than other
API-based
tracing tools and lets you debug system-wide GPU scheduling issues,
eg.
interaction between several processes using the GPU, which is pretty
critical for VR usecases.
It's all based on ftrace; we've initially developped it with
support for
amdgpu, and had to patch the kernel code there to change what
tracing events
are reported and how. Now that we have a good idea of what's needed
and it's
more or less proven in production, we were wondering if you had any
interest
in adding a similar set of events for Intel GPUs so we could read
them and
present them the same way? We have pretty specific requirements,
but this
work-in-progress documentation should give a good idea of what they
are:
https://github.com/mikesart/gpuvis/wiki/Overview
We already have those tracepoint equivs and a script to generate a
similar visualisation: intel-gpu-tools/scripts/trace.pl, but only
looking at the scheduling issue from the gpu pov. But it's really only a
dev toy atm, plugging the gap between userspace and the gpu has been on
the perennial wishlist.
-Chris
I added Intel event visualization to gpuvis based on your trace.pl
script. Screenshot at the top of the wiki page here:
https://github.com/mikesart/gpuvis/wiki/TechDocs-Intel
In that screenshot the mouse is hovering over the ctx=30,seqno=1900 bar
which selects those events in the event list and shows a tooltip with
the submit, execute, etc info.
It certainly looks immensely better than my browser based hack. But
unfortunately I still did not get round actually trying your tool.
How scalable it is - meaning - can it handle very busy and huge traces?
The typical SteamVR trace in our "DVR" plumbing (always tracing in the
background) has about 500k events over ~20 seconds. I can zoom and scrub
through it at 60fps without issues here.
Is there any outlook of it getting packaged in some distro?
The tool itself is standalone and very easy to package; Mike recently
wrote some plumbing example scripts around it that can get useful
captures, so I would think we're in better shape for packaging now than
before they existed. We bundle it with SteamVR so haven't really spent
any time looking into packaging.
For the amdgpu driver, we're able to get the submit information from
user space and associate those events to specific processes. Example of
that is here:
https://github.com/mikesart/gpuvis/wiki/TechDocs-AMDGpu
If you ever get a chance to try gpuvis and have any feedback, we'd love
to hear it. Also if you ever get userspace tracepoint data in, let me
know and I'd be happy to hook that up as well.
What kind of information is missing to wire up this missing bit? I mean
the thing you are referring to as user space submit data, what is that?
The main thing is to be able to associate a chunk of GPU work with a
userspace process. Currently, there's no tracepoint we're aware of in
the work submission ioctl, which means that while the GPU tasks are
properly displayed, they're anonymous. The main intent behind gpuvis is
to be able to debug multi-process GPU interaction and timing problems,
so having a tracepoint in the submit ioctl lets us associate the user
context of the application that submitted the work with the work itself
if it shares the same identifying seqnos/ids. gpuvis can then show you
thread information about that process, color-code for easy
disambiguation, etc.
Btw one new thing we are close to merging to i915 is the perf PMU
support. That will enable real-time monitoring of per-engine busyness,
waits, frequency, power, maybe more in the future like queue-depth. I
don't know if things like that would be interesting for gpuvis? Some of
it can be inferred from the tracepoints already in post-processing so
there is some overlap. I am not sure, but thought to mention it. This is
the series: https://patchwork.freedesktop.org/series/27488/. It is used
via existing perf userspace API.
Seems like the kind of thing gpuvis could happily overlay on top of the
trace with its plotting functionality.
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
