The vt code translates UTF-8 strings into glyph index values and stores those glyph values in the screen buffer. Because there can only be at most 512 glyphs at the moment, it is impossible to represent most unicode characters, in which case a default glyph (often '?') is displayed instead. The original unicode value is then lost. The 512-glyph limitation is inherent to text-mode VGA displays after which the core console code was modelled. This also means that the /dev/vcs* devices only provide user space with glyph index values, and then user applications must get hold of the unicode-to-glyph table the kernel is using in order to back-translate those into actual characters. It is not possible to get back the original unicode value when multiple unicode characters map to the same glyph, especially for the vast majority that maps to the default replacement glyph. Users of /dev/vcs* shouldn't have to be restricted to a narrow unicode space from lossy screen content because of that. This is especially true for accessibility applications such as BRLTTY that rely on /dev/vcs to render screen content onto braille terminals. It was also argued that the VGA-centric glyph buffer should eventually go entirely. The current design made sense when hardware was slow and managing the screen directly into the VGA memory made a difference (i.e. 25 years ago). Modern console display drivers no longer have to be limited to 512 glyphs. Quoting Alan Cox: |The only driver that it suits is the VGA text mode driver, which at |2GHz+ is going to be fast enough whatever format you convert from. We |have the memory, the processor power and the fact almost all our |displays are bitmapped (or more complex still) all in favour of |throwing away that limit. This patch series introduces unicode screen support to the core console code with /dev/vcs* as a first user. Memory is allocated, and possible CPU overhead introduced, only if /dev/vcsu is read at least once. For now both the glyph and unicode buffers are maintained in parallel to allow for a smooth transition. I'm a prime user of this new /dev/vcsu interface, as well as the BRLTTY maintainer Dave Mielke who implemented support for this in BRLTTY. There is therefore a vested interest in maintaining this feature as necessary. And this received extensive testing as well at this point. This is also available on top of v4.18-rc2 here: git://git.linaro.org/people/nicolas.pitre/linux vt-unicode Changes from v2: - Dropped patch #4 as it was useful only for initial debugging and it attracted all the review comments so far -- actually more than the patch is worth. - Added Adam Borowski's ACK. Changes from v1: - Rebased to v4.18-rc1. - Dropped first patch (now in mainline as commit 4b4ecd9cb8). - Removed a printk instance from an error path easily triggerable from user space. - Minor cleanup. Diffstat: drivers/tty/vt/vc_screen.c | 90 ++++++++-- drivers/tty/vt/vt.c | 308 +++++++++++++++++++++++++++++++-- include/linux/console_struct.h | 2 + include/linux/selection.h | 5 + 4 files changed, 380 insertions(+), 25 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-console" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
