From: Geert Uytterhoeven <geert@xxxxxxxxxxxxxx> Sent: Thursday, March 20, 2025 3:46 AM
>
> Hi Michael,
>
> On Wed, 19 Mar 2025 at 21:29, Michael Kelley <mhklinux@xxxxxxxxxxx> wrote:
> > From: Helge Deller <deller@xxxxxx> Sent: Tuesday, March 18, 2025 1:16 AM
> > > On 3/18/25 03:05, Michael Kelley wrote:
> > > > I've been trying to get mmap() working with the hyperv_fb.c fbdev driver, which
> > > > is for Linux guests running on Microsoft's Hyper-V hypervisor. The hyperv_fb driver
> > > > uses fbdev deferred I/O for performance reasons. But it looks to me like fbdev
> > > > deferred I/O is fundamentally broken when the underlying framebuffer memory
> > > > is allocated from kernel memory (alloc_pages or dma_alloc_coherent).
> > > >
> > > > The hyperv_fb.c driver may allocate the framebuffer memory in several ways,
> > > > depending on the size of the framebuffer specified by the Hyper-V host and the VM
> > > > "Generation". For a Generation 2 VM, the framebuffer memory is allocated by the
> > > > Hyper-V host and is assigned to guest MMIO space. The hyperv_fb driver does a
> > > > vmalloc() allocation for deferred I/O to work against. This combination handles mmap()
> > > > of /dev/fb<n> correctly and the performance benefits of deferred I/O are substantial.
> > > >
> > > > But for a Generation 1 VM, the hyperv_fb driver allocates the framebuffer memory in
> > > > contiguous guest physical memory using alloc_pages() or dma_alloc_coherent(), and
> > > > informs the Hyper-V host of the location. In this case, mmap() with deferred I/O does
> > > > not work. The mmap() succeeds, and user space updates to the mmap'ed memory are
> > > > correctly reflected to the framebuffer. But when the user space program does munmap()
> > > > or terminates, the Linux kernel free lists become scrambled and the kernel eventually
> > > > panics. The problem is that when munmap() is done, the PTEs in the VMA are cleaned
> > > > up, and the corresponding struct page refcounts are decremented. If the refcount goes
> > > > to zero (which it typically will), the page is immediately freed. In this way, some or all
> > > > of the framebuffer memory gets erroneously freed. From what I see, the VMA should
> > > > be marked VM_PFNMAP when allocated memory kernel is being used as the
> > > > framebuffer with deferred I/O, but that's not happening. The handling of deferred I/O
> > > > page faults would also need updating to make this work.
>
> I assume this is triggered by running any fbdev userspace that uses
> mmap(), e.g. fbtest?
Yes. I have my own little program, but it is similar to what I see fbtest does.
>
> > > > The fbdev deferred I/O support was originally added to the hyperv_fb driver in the
> > > > 5.6 kernel, and based on my recent experiments, it has never worked correctly when
> > > > the framebuffer is allocated from kernel memory. fbdev deferred I/O support for using
> > > > kernel memory as the framebuffer was originally added in commit 37b4837959cb9
> > > > back in 2008 in Linux 2.6.29. But I don't see how it ever worked properly, unless
> > > > changes in generic memory management somehow broke it in the intervening years.
> > > >
> > > > I think I know how to fix all this. But before working on a patch, I wanted to check
> > > > with the fbdev community to see if this might be a known issue and whether there
> > > > is any additional insight someone might offer. Thanks for any comments or help.
> > >
> > > I haven't heard of any major deferred-i/o issues since I've jumped into fbdev
> > > maintenance. But you might be right, as I haven't looked much into it yet and
> > > there are just a few drivers using it.
> >
> > Thanks for the input. In the fbdev directory, there are 9 drivers using deferred I/O.
> > Of those, 6 use vmalloc() to allocate the framebuffer, and that path works just fine.
> > The other 3 use alloc_pages(), dma_alloc_coherent(), or __get_free_pages(), all of
> > which manifest the underlying problem when munmap()'ed. Those 3 drivers are:
> >
> > * hyperv_fb.c, which I'm working with
> > * sh_mobile_lcdcfb.c
> > * ssd1307fb.c
>
> Nowadays sh_mobile_lcdcfb is used only on various SuperH boards
> (I have no hardware to test).
>
> sh_mobile_lcdcfb was used on ARM-based SH/R-Mobile SoCs until DT
> support was added to the DRM driver for the corresponding hardware.
> The platform using it was migrated to DRM in commit 138588e9fa237f97
> ("ARM: dts: renesas: r8a7740: Add LCDC nodes") in v6.8). At the time
> of the conversion, fbtest worked fine with sh_mobile_lcdcfb.
OK, good to know. sh_mobile_lcdcfb gets its framebuffer using
dma_alloc_coherent(). Do you recall how big the framebuffer was?
If over 4 MiB, dma_alloc_coherent() would have allocated from CMA,
and that works OK.
>
> Deferred I/O is also used in DRM drivers for displays that are connected
> using I2C or SPI. Last time I tried the st7735r driver, it worked fine
> with fbtest. That was also on arm32, though.
The st7735r driver appears to use fbtft-core.c, and that does a vmalloc()
for the screen buffer, so it won't have the problem I'm seeing.
Thanks for the help ....
Michael
