Hello,
I'm currently working on designing an interface for controlling "forcepads";
that is, touchpads with force sensors and haptic actuators. Below is my
proposal for the protocol at both the userspace and HID interfaces. I would
appreciate any feedback you might have.
Thank you,
Sean O'Brien
Chromium OS Touch/Input Team
Background
==========
There are multiple independent projects to develop a touchpad with force sensors
and haptic actuators, instead of a traditional button. These “forcepads” have
several advantages and potential uses; they allow clicking across the entire
touchpad surface, adjusting the force requirement for clicks, haptic feedback
initiated by UI, etc.
Objective
=========
Develop a standard protocol to allow userspace applications to communicate with
forcepads, and minimize duplicated code and effort.
Requirements:
1. Support UI-initiated haptic feedback.
2. Allow userspace to control when button press and button release haptic
effects are triggered. (Useful when detecting a false click, changing force
thresholds, or sending context-dependent effects)
3. Allow a backward-compatible mode where the forcepad produces button press and
button release effects autonomously.
4. Reveal force sensor readings to userspace applications.
Proposal
========
I propose standardized forcepad support in the linux kernel at both the HID and
userspace interface.
Userspace Interface
-------------------
Multitouch
..........
The linux kernel has a well defined protocol [0] for multitouch devices,
including touchpads. The protocol defines a field called ABS_MT_PRESSURE as a
measure of the pressure applied by a contact. Unfortunately, it is not used
that way in practice- instead it is generally used as an approximate measure of
contact area. I will distinguish these concepts by calling them “true force” and
“traditional pressure”.
Instead of using ABS_MT_PRESSURE to report the “traditional pressure,” forcepads
would send “true force” values in this field. For each contact, this field would
report the estimated force applied by that contact.
The resolution of ABS_MT_PRESSURE should also be defined and reported, so that
userspace consumers can translate to force units. By defining the resolution, we
also differentiate it from how it is used to report “traditional pressure”,
where it has no resolution. Userspace consumers will be able to use this to
detect that this is a forcepad, and treat the pressure field accordingly.
ABS_PRESSURE may be optionally reported as the total force applied to the
forcepad.
The device/driver shouldn’t detect button clicks, this is left to the userspace
gesture library. Accordingly, the driver should not sent BTN_* events to
userspace in normal operating mode. However it should still report the ability
to produce such events, for use in autonomous mode.
Haptic Control
..............
The force feedback protocol [1] should be used to request predefined effects.
Typical use of the force feedback protocol requires loading effects to the
driver by describing the output waveform, and then requesting those effects
using an ID provided by the driver. We don’t want to describe the output
waveform explicitly, but instead use a set of predefined IDs for the desired
effect types. The device/driver would be responsible for having the effects
loaded and ready to respond to a request for the predefined IDs.
The force feedback protocol will need to be extended to allow requests for
predefined IDs. This requires a new feedback effect type:
/**
* struct ff_predefined_effect
* @level: strength of the effect
* @vendor: ID of the vendor who defined the effect
* @id: ID of the effect among those defined by the vendor
*/
struct ff_predefined_effect {
__s16 level;
__u16 vendor;
__u16 id;
}
Vendors can define specifications for the waveforms and assign them IDs. They
could then be requested using their vendor ID and the waveform ID, as defined in
the simple haptic HID protocol [2].
To allow a standard way to trigger press and release effects, all forcepads
should support the WAVEFORM_PRESS and WAVEFORM_RELEASE through this interface.
Since the standard waveform id namespace doesn’t overlap with the vendor
waveform id namespace, the vendor id can be ignored for these waveforms.
Autonomous mode
---------------
In order to facilitate an operating system which cannot handle force feedback,
the forcepad should start up in “autonomous mode”, meaning it acts as a normal
touchpad. This means it should perform the press and release haptic feedback
autonomously at predefined force thresholds, and send the appropriate BTN_*
events.
After verifying that all of the required haptic effects are available through
the force feedback protocol, the OS can enable host-controlled mode. This could
be done by writing to a sysfs node “host_feedback_enabled”.
When the host enters suspend mode, the OS will not be able to respond quickly
enough to input from the touchpad to tell it to perform haptic feedback, making
the touchpad feel unresponsive. When the host suspends, the touchpad should
enter autonomous mode.
HID Interface
-------------
Multitouch
..........
The HID API for multitouch reports is mostly unchanged except:
1. The tip pressure field [3] should be used to report “true force” instead of
“traditional pressure”. The physical unit type, exponent, and limits should
be reported in the report descriptor for the “true force” field [4].
2. The device will always report it’s button as being unpressed, except in
autonomous mode, when it will report the button state according to its
predefined force thresholds.
Haptic control
..............
The simple haptic HID protocol [2] should be used.
The following waveforms should be supported:
| WAVEFORM_NONE | Implicit waveforms required by protocol |
| WAVEFORM_STOP | |
| ------------------------ | ------------------------------------------------- |
| WAVEFORM_PRESS | To be used in autonomous mode and host-controlled |
| WAVEFORM_RELEASE | mode to simulate button press and release. |
| ------------------------ | ------------------------------------------------- |
| Vendor-defined waveforms | Optional waveforms to be used in host-controlled |
| | mode, subject to vendor specification. |
All waveforms will have an associated duration; continuous waveforms will not be
supported.
Only manual triggering will be supported through this interface. Autonomous
triggering of waveforms is enabled by putting the device in autonomous mode.
Retriggering (queueing multiple triggers of the same waveform) is not supported.
If intensity modification for waveforms is supported by the device, the
intensity control should be included in the manual trigger output report. This
allows modification of the intensity on a per-waveform basis.
Alternatives Considered
=======================
Add a “true force” field to the multi-touch protocol
----------------------------------------------------
This allows us to send “traditional pressure” in addition to “true force”. It
also allows another possible protocol in addition to sending the force per
contact:
* Sending overall force and center of force: Should be easier to calculate
depending on force sensor layout, and provides potentially useful extra info
As mentioned before, there is already a concept of pressure in the multi-touch
protocol, generally used as a proxy for contact area. We would add another field
to represent force. However, the pressure field is defined by the protocol as
actual pressure, and I feel this would only add more confusion. The touch_major
and touch_minor fields can be used to report contact area explicitly, so we're
not losing that capability.
Report overall force, but not per-contact force
-----------------------------------------------
Instead of reporting the “traditional pressure,” send “true force” values in the
ABS_PRESSURE field.The ABS_MT_PRESSURE field would still be used to send
“traditional pressure” for each contact.
I'm not convinced this is necessary, or the best idea. It makes backward
compatibility easy, but other input libraries could detect forcepads by the
resolution associated with the ABS_MT_PRESSURE field, and ignore that field,
using touch_major and touch_minor instead.
In addition, it adds even more confusion than the above option, so I think I'd
prefer a clean break.
Use driver-wide gain for force feedback
---------------------------------------
The force feedback protocol also has a mechanism to set driver-wide gain, which
could be used to set global effect strength level. However, allowing different
strength levels for each predefined effect would allow more flexibility e.g.: if
the user wants to have low strength for UI feedback effects and high strength
for button click effects.
[0]: https://www.kernel.org/doc/Documentation/input/multi-touch-protocol.txt
[1]: https://www.kernel.org/doc/Documentation/input/ff.txt
[2]: https://www.usb.org/sites/default/files/hutrr63b_-_haptics_page_redline_0.pdf
[3]: Usage ID 0x30 of HID usage table 0x0D. See chapter 16:
https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf
[4]: See section 6.2.2.7 of the HID specification:
https://www.usb.org/sites/default/files/documents/hid1_11.pdf
