On Fri, Apr 21, 2017 at 06:14:31PM +0200, Gerd Hoffmann wrote: > Hi, > > > > My personal opinion is that formats in drm_fourcc.h should be > > > independent of the CPU byte order and the function > > > drm_mode_legacy_fb_format() and drivers depending on that incorrect > > > assumption be fixed instead. > > > > The problem is this isn't a kernel-internal thing any more. With the > > addition of the ADDFB2 ioctl the fourcc codes became part of the > > kernel/userspace abi ... > > Ok, added some printk's to the ADDFB and ADDFB2 code paths and tested a > bit. Apparently pretty much all userspace still uses the ADDFB ioctl. > xorg (modesetting driver) does. gnome-shell in wayland mode does. > Seems the big transition to ADDFB2 didn't happen yet. > > I guess that makes changing drm_mode_legacy_fb_format + drivers a > reasonable option ... Yeah, I came to the same conclusion after chatting with some folks on irc. So my current idea is that we change any driver that wants to follow the CPU endianness to declare support for big endian formats if the CPU is big endian. Presumably these are mostly the virtual GPU drivers. Additonally we'll make the mapping performed by drm_mode_legacy_fb_format() driver controlled. That way drivers that got changed to follow CPU endianness can return a framebuffer that matches CPU endianness. And drivers that expect the GPU endianness to not depend on the CPU endianness will keep working as they do now. The downside is that users of the legacy addfb ioctl will need to magically know which endianness they will get, but that is apparently already the case. And users of addfb2 will keep on specifying the endianness explicitly with DRM_FORMAT_BIG_ENDIAN vs. 0. Does that sound like a workable solution? -- Ville Syrjälä Intel OTC
