On Wed, Dec 26, 2018 at 12:55 PM Billy Laws <blaws05@xxxxxxxxx> wrote: > > > > On 26 December 2018 17:02:33 GMT, Roderick Colenbrander <thunderbird2k@xxxxxxxxx> wrote: > >On Tue, Dec 25, 2018 at 2:42 PM Billy Laws <blaws05@xxxxxxxxx> wrote: > >> > >> > >> > >> On 23 December 2018 04:45:13 GMT, Roderick Colenbrander > ><thunderbird2k@xxxxxxxxx> wrote: > >> >On Fri, Dec 21, 2018 at 8:16 AM Billy Laws <blaws05@xxxxxxxxx> > >wrote: > >> >> > >> >> > >> >> > >> >> On 21 December 2018 09:56:10 GMT, Benjamin Tissoires > >> ><benjamin.tissoires@xxxxxxxxx> wrote: > >> >> >Hi Billy, > >> >> > > >> >> >On Tue, Dec 18, 2018 at 11:16 AM Billy Laws <blaws05@xxxxxxxxx> > >> >wrote: > >> >> >> > >> >> >> Hi, > >> >> >> I'm looking into making a driver for the Nintendo switches > >joycon, > >> >> >they are two separate controllers that can be used together or > >> >> >separately but most of the time they are used together as one > >> >> >controller. > >> >> >> Each of them is presented as a separate Bluetooth hid device so > >> >I'm > >> >> >not sure the best way to represent them to user space. Should I > >have > >> >a > >> >> >new input device for each separate joycon or create a new input > >> >device > >> >> >for every pair of l+r joycon? > >> >> > > >> >> >I'd say if they are 2 distincts bluetooth devices, it makes sense > >to > >> >> >have them separated. However, in the same way we have touch > >panels > >> >> >that are doing pen and touch and that are exported as 2 separate > >> >> >device nodes, you should be able to provide the info to the user > >> >space > >> >> >that those 2 devices are supposed to be used together. > >> >> >For pen/touch devices, userspace (udev) relies on the physical > >path > >> >> >and assign LIBINPUT_DEVICE_GROUP to group those 2 devices > >> >altogether. > >> >> > > >> >> >So I think if you provide a way from the serial to be able to > >match > >> >> >those 2, user space should be OK. > >> >> > > >> >> >However, please note that I am not in the SDL community (I am > >more a > >> >> >regular input devices developer) so their advice might differ. > >> >> > > >> >> Also, what about programs that access the controller directly > >without > >> >libinput, i guess they would need to be fixed? > >> >> How would I tell userspace which one is left and which one is > >right? > >> > > >> >Do the left and the right joycon have a different product id? > >Ideally > >> >applications use vendor and product id. Worst case you can put Left > >/ > >> >Right in the device string... > >> > > >> Yeah, they do. > >> I've been discussing with some switch Linux folks and we have come to > >the conclusion that doing it this way isn't really going to be good for > >the intended use of them. There are lots of games that will enumerate > >each controller as a separate player and that would be pretty hard to > >change for all if them. > >> Could there be a sysfs knob to unify the inputs to one? Then there > >can be an application in user space can switch between modes. > >> If not, then the only solution we could think of was to create a fake > >input device in userspace that joins both and somehow prevent games > >from using split ones, which I find really ugly. > > > >I'm well aware of these kind of challenges for "composite devices". To > >some extend it is a kernel issue, but also a userspace issue where > >many applications / libraries on the gaming side assume "1 evdev > >device = 1 player". This is not correct. There are many devices for > >which the kernel reports multiple evdev devices e.g. Wacom tablets, > >keyboards, Wii controller, Sony DualShock 3 / 4 and many others. Most > >of these devices are non-gaming devices and handled through libinput, > >which hides these challenges and most people are not even aware of the > >issues behind the scenes. > > > >On the gaming side many applications use evdev directly or through > >SDL2, but there is an assumption about "1 evdev device = 1 player". A > >few years ago we (=Sony) refactored the DualShock 3 /4 driver > >(hid-sony). We had to expose multiple evdev devices 1 for "gamepad", 1 > >for "motion sensors" and 1 for "touchpad". Each device has different > >evdev capabilities, some games are kind of aware now. > > > >That said handling composite devices from userspace is a pain. > >Basically right now you need to scrape sysfs and stitch devices > >together. We had some threads before with Peter Hutterer among others > >in which we proposed evdev improvements for composite device > >enumeration. We never sorted it out. Part of the challenge is that > >enumeration improvements, easily have to span multiple frameworks. > >Gaming peripherals often have other features e.g. speakers, > >microphones, motion sensors to weird ones like heart rate sensors and > >many others. > > > >Back to the Joycon. A single Joycon to me looks like a composite > >device as well. You have a stick and a few buttons in addition to a > >accelerometer + gyroscope. This at least requires 2x evdev device per > >Joycon (1 for buttons/sticks and 1 for motion sensors). In addition > >the right Joycon has a depth sensor as well. Not sure how to expose > >this sensor, whether it can be done through evdev in a new way or > >should use IIO (we need to bridge IIO to evdev better). > > > It isn't a depth sensor, its an entire near infrared camera, but it seems really hard to implement and there is no v4l hid integration. There is also an NFC reader in the left joycon. Thanks, there other cases of cross-api in particular audio and weird sensors exposed through IIO. The idea we had a while ago was maybe to add a new evdev API for querying composite devices. For example it could have returned a list of devices e.g. major/minor of character device + API like v4l, alsa or something like that. There were edge cases as some features are only exposed through sysfs e.g. LEDs or sometimes IIO sensors. I discussed extensively within my team at work as well. We couldn't figure out something good. We almost felt the "composite device problem" should be solved outside of evdev. Maybe have some helpers for sysfs to bundle devices together in "device groups". At some point I would love to help sort this problem out as it is a big challenge for us and I'm sure for many others. > >Bundling of 2x Joycon left + right almost feels like a "dynamic > >composite device". I don't know where such combining should be done. > >My gut feeling tells me at the userspace side. The kernel side feels > >like it would easily become really complicated to implement. > > > >For example if each Joycon was initially separate, there would be at 1 > >least 1 evdev node per device (likely more due to sensors). Suddenly > >"bundling them", might mean changing capabilities of a live evdev > >device (adding more buttons/sticks) and removing evdev node for the > >now redundant other evdev device. (There might be other approaches) It > >feels quite messy also for applications, who see devices come/go and > >capabilities change. > > > >I recall the Wii driver supported adding of peripherals as well at > >runtime. I forgot how it was done there. I suspect it are separate > >evdev nodes.. > > > >All in all, I'm not sure what the best solution is. I'm thinking more > >and more (also because of DS4 discussions) that it is a job for > >userspace and that games and gaming libraries haven't caught up with > >modern evdev usages. Though it is too hard for every game to catch up > >and the sysfs/evdev logic is complicated. Ideally a higher level > >library should be used. The kernel could probably do a slightly better > >job making device enumeration easier to help as well. > Openxr seems like it will help greatly for this issue as it was built with this in mind but my concern is only vr games will see the importance of it and others will do whatever they used to do. Maybe OpenXR can do it, I haven't seen the exact specs. Though honestly I'm a bit sceptical if they will solve this stuff, they probably just offer a wrapper around evdev and may not really solve this problem. It really depends on how vendors decide to bubble up their input peripherals. I have seen a lot of custom device specific APIs e.g. Daydream on Android even Android has a fine input and sensors system. > What library's are you referencing for ds4? I guess I should have a look at that. What can I do about old games that will never be updated? Is user space emulation of evdev the best idea then Well many Linux games used evdev directly for DS4 and I guess added some special handling. The main challenge were the motion sensors, but the motion sensors evdev node sets "INPUT_PROP_ACCELEROMETER" which they are using for filtering. SDL2 applications kind of work fine as most use the SDL2 gamepad API, which does some filtering based on capabilities. It kind of "worked by luck" though we did provide various SDL2 patches to improve DS4 button / stick mapping. The main button which can't be reached through SDL2 gamepad API is the "click button" under the touchpad as it is part of the "touchpad evdev node". In an ideal world their world be a "higher level" API just like libinput, but more geared towards gaming. Applications and libraries like SDL2 could interface with this. > >Thanks, > >Roderick
