On Fri, 25 May 2012 10:51:19 -0700
Jesse Barnes <jbarnes at virtuousgeek.org> wrote:
> On Fri, 27 Apr 2012 17:40:21 -0700
> Ben Widawsky <ben at bwidawsk.net> wrote:
>
> > Dumb binary interfaces which allow root-only updates of the cache
> > remapping registers. As mentioned in a previous patch, software using
> > this interface needs to know about HW limits, and other programming
> > considerations as the kernel interface does no checking for these things
> > on the root-only interface.
> >
> > v1: Drop extra posting reads (Chris)
> > Return negative values in the sysfs interfaces on errors (Chris)
> >
> > Signed-off-by: Ben Widawsky <ben at bwidawsk.net>
> > ---
> > drivers/gpu/drm/i915/i915_sysfs.c | 128 ++++++++++++++++++++++++++++++++++++-
> > 1 file changed, 126 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/i915_sysfs.c b/drivers/gpu/drm/i915/i915_sysfs.c
> > index 79f8344..ed77cbf 100644
> > --- a/drivers/gpu/drm/i915/i915_sysfs.c
> > +++ b/drivers/gpu/drm/i915/i915_sysfs.c
> > @@ -29,6 +29,7 @@
> > #include <linux/module.h>
> > #include <linux/stat.h>
> > #include <linux/sysfs.h>
> > +#include "intel_drv.h"
> > #include "i915_drv.h"
> >
> > static u32 calc_residency(struct drm_device *dev, const u32 reg)
> > @@ -92,20 +93,143 @@ static struct attribute_group rc6_attr_group = {
> > .attrs = rc6_attrs
> > };
> >
> > +static int l3_access_valid(struct drm_device *dev, loff_t offset)
> > +{
> > + if (!IS_IVYBRIDGE(dev))
> > + return -EPERM;
>
> -ENODEV?
>
> > +
> > + if (offset % 4 != 0)
> > + return -EPERM;
>
> -EINVAL?
>
> > +
> > + if (offset >= GEN7_L3LOG_SIZE)
> > + return -ENXIO;
>
> -E2BIG or -ERANGE?
>
> And maybe some debug messages here so people can debug their daemons in
> case some of the return values are identical.
>
Good point, I've gone with EPERM, EINVAL, and ENXIO - so no more
duplication and added the debug message for good measure.
> > +
> > + return 0;
> > +}
> > +
> > +static ssize_t
> > +i915_l3_read(struct file *filp, struct kobject *kobj,
> > + struct bin_attribute *attr, char *buf,
> > + loff_t offset, size_t count)
> > +{
> > + struct device *dev = container_of(kobj, struct device, kobj);
> > + struct drm_minor *dminor = container_of(dev, struct drm_minor, kdev);
> > + struct drm_device *drm_dev = dminor->dev;
> > + struct drm_i915_private *dev_priv = drm_dev->dev_private;
> > + uint32_t misccpctl;
> > + int i, ret;
> > +
> > + ret = l3_access_valid(drm_dev, offset);
> > + if (ret)
> > + return ret;
> > +
> > + ret = i915_mutex_lock_interruptible(drm_dev);
> > + if (ret)
> > + return ret;
> > +
> > + misccpctl = I915_READ(GEN7_MISCCPCTL);
> > + I915_WRITE(GEN7_MISCCPCTL, misccpctl & ~GEN7_DOP_CLOCK_GATE_ENABLE);
> > +
> > + for (i = offset; count >= 4 && i < GEN7_L3LOG_SIZE; i += 4, count -= 4)
> > + *((uint32_t *)(&buf[i])) = I915_READ(GEN7_L3LOG_BASE + i);
> > +
> > + I915_WRITE(GEN7_MISCCPCTL, misccpctl);
> > +
> > + mutex_unlock(&drm_dev->struct_mutex);
> > +
> > + return i - offset;
> > +}
> > +
> > +static ssize_t
> > +i915_l3_write(struct file *filp, struct kobject *kobj,
> > + struct bin_attribute *attr, char *buf,
> > + loff_t offset, size_t count)
> > +{
> > + struct device *dev = container_of(kobj, struct device, kobj);
> > + struct drm_minor *dminor = container_of(dev, struct drm_minor, kdev);
> > + struct drm_device *drm_dev = dminor->dev;
> > + struct drm_i915_private *dev_priv = drm_dev->dev_private;
> > + u32 *temp = NULL;
> > + int i, ret;
> > +
> > + ret = l3_access_valid(drm_dev, offset);
> > + if (ret)
> > + return ret;
> > +
> > + ret = i915_mutex_lock_interruptible(drm_dev);
> > + if (ret)
> > + return ret;
> > +
> > + if (!dev_priv->mm.l3_remap_info) {
> > + temp = kzalloc(GEN7_L3LOG_SIZE, GFP_KERNEL);
> > + if (!temp) {
> > + mutex_unlock(&drm_dev->struct_mutex);
> > + return -ENOMEM;
> > + }
> > + }
>
> Grumble, lazy allocation. But if we did this at driver load time we'd
> also need a valid bit so we knew whether to try to load it or not...
>
Just curious what the grumble is about? It is a sysfs entry after all.
> > +
> > + ret = i915_gpu_idle(drm_dev, true);
> > + if (ret) {
> > + kfree(temp);
> > + mutex_unlock(&drm_dev->struct_mutex);
> > + return ret;
> > + }
> > +
> > + /* TODO: Ideally we really want a GPU reset here to make sure errors
> > + * aren't propagated Since I cannot find a stable way to reset the GPU
> > + * at this point it is left as a TODO.
> > + */
>
> Do we want a reset? Or just to restart the app that consumed the
> error? If the latter, then the current uevent ought to be enough,
> potentially with a configurable SIGBUS or something.
>
I've thought about this a great deal. I think as long as every app
referencing the buffer is killed, and we do a full pipeline flush after
the remapping, we should be able to avoid a reset. I like the idea of
SIGBUS, but it's a bit of work to track which buffer is actually bad,
and with buffer re-use, potentially harder to find the application that
cared. Therefore, I think reset is the easiest way to just make every
app reset.
> > +
> > + if (dev_priv->mm.l3_remap_info)
> > + temp = dev_priv->mm.l3_remap_info;
> > +
> > + dev_priv->mm.l3_remap_info = temp;
>
> I'd combine these assignments with the kzalloc block above just to keep
> things together.
>
Leftover cruft from when I had reset code. Thanks.
> > +
> > + for (i = offset; count >= 4 && i < GEN7_L3LOG_SIZE; i += 4, count -= 4) {
> > + u32 remap = I915_READ(GEN7_L3LOG_BASE + i);
> > + if (remap && remap != *temp)
> > + DRM_ERROR("0x%x was already programmed to %x\n",
> > + GEN7_L3LOG_BASE + i, remap);
> > + *temp++ = *(uint32_t *)(&buf[i]);
> > + }
>
> Is this an error? Or just a "userspace is schizo" and therefore
> just a DEBUG?
>
You're right. Thanks.
> > +
> > + i915_gem_l3_remap(drm_dev);
>
> You could add the warning/debug info to l3_remap instead, right?
>
Yes. Fixed.
> > +
> > + mutex_unlock(&drm_dev->struct_mutex);
> > +
> > + return offset - i;
> > +}
> > +
> > +static struct bin_attribute dpf_attrs = {
> > + .attr = {.name = "l3_parity", .mode = (S_IRUSR | S_IWUSR)},
> > + .size = GEN7_L3LOG_SIZE,
> > + .read = i915_l3_read,
> > + .write = i915_l3_write,
> > + .mmap = NULL
> > +};
> > +
> > void i915_setup_sysfs(struct drm_device *dev)
> > {
> > int ret;
> >
> > - /* ILK doesn't have any residency information */
> > + /* ILK and below don't yet have relevant sysfs files */
> > if (INTEL_INFO(dev)->gen < 6)
> > return;
> >
> > ret = sysfs_merge_group(&dev->primary->kdev.kobj, &rc6_attr_group);
> > if (ret)
> > - DRM_ERROR("sysfs setup failed\n");
> > + DRM_ERROR("RC6 residency sysfs setup failed\n");
> > +
> > + if (!IS_IVYBRIDGE(dev))
> > + return;
> > +
> > + ret = device_create_bin_file(&dev->primary->kdev, &dpf_attrs);
> > + if (ret)
> > + DRM_ERROR("l3 parity sysfs setup failed\n");
> > }
> >
> > void i915_teardown_sysfs(struct drm_device *dev)
> > {
> > + device_remove_bin_file(&dev->primary->kdev, &dpf_attrs);
> > sysfs_unmerge_group(&dev->primary->kdev.kobj, &rc6_attr_group);
> > }
>
> I think Daniel covered the above.
Yep.
>
> With the above addressed:
> Reviewed-by: Jesse Barnes <jbarnes at virtuousgeek.org>
>
Thanks.
--
Ben Widawsky, Intel Open Source Technology Center
