Hi, On 2025-01-27 20:20:41 +0100, David Hildenbrand wrote: > On 27.01.25 18:25, Andres Freund wrote: > > On 2025-01-27 15:09:23 +0100, David Hildenbrand wrote: > > Unfortunately for the VMs with those disks I don't have access to hardware > > performance counters :(. > > > > > > Maybe there is a link to the report you could share, thanks. > > > > A profile of the "original" case where I hit this, without the patch that > > Willy linked to: > > > > Note this is a profile *not* using hardware perf counters, thus likely to be > > rather skewed: > > https://gist.github.com/anarazel/304aa6b81d05feb3f4990b467d02dabc > > (this was on Debian Sid's 6.12.6) > > > > Without the patch I achieved ~18GB/s with 1GB pages and ~35GB/s with 2MB > > pages. > > Out of interest, did you ever compare it to 4k? I didn't. Postgres will always do at least 8kB (unless compiled with non-default settings). But I also don't think I tested just doing 8kB on that VM. I doubt I'd have gotten close to the max, even with 2MB huge pages. At least not without block-layer-level merging of IOs. If it's particularly interesting, I can bring a similar VM up and run that comparison. > > This time it's actual hardware perf counters... > > > > Relevant details about the c2c report, excerpted from IRC: > > > > andres | willy: Looking at a bit more detail into the c2c report, it looks > > like the dirtying is due to folio->_pincount and folio->_refcount in > > about equal measure and folio->flags being modified in > > gup_fast_fallback(). The modifications then, unsurprisingly, cause a > > lot of cache misses for reads (like in bio_set_pages_dirty() and > > bio_check_pages_dirty()). > > > > willy | andres: that makes perfect sense, thanks > > willy | really, the only way to fix that is to split it up > > willy | and either we can split it per-cpu or per-physical-address-range > > As discussed, even better is "not repeatedly pinning/unpinning" at all :) Indeed ;) > I'm curious, are multiple processes involved, or is this all within a single > process? In the test case here multiple processes are involved, I was testing a parallel sequential scan, with a high limit to the paralellism. There are cases in which a fair bit of read IO is done from a single proccess (e.g. to prerewarm the buffer pool after a restart, that's currently done by a single process), but it's more common for high throughput to happen across multiple processes. With modern drives a single task won't be able to execute non-trivial queries at full disk speed. Greetings, Andres Freund
