On Sat, Aug 05, 2023 at 12:03:59AM +0200, Frederic Weisbecker wrote:
> On Tue, Jun 27, 2023 at 05:08:15PM -0300, Marcelo Tosatti wrote:
> >
> > For certain types of applications (for example PLC software or
> > RAN processing), upon occurrence of an event, it is necessary to
> > complete a certain task in a maximum amount of time (deadline).
> >
> > One way to express this requirement is with a pair of numbers,
> > deadline time and execution time, where:
> >
> > * deadline time: length of time between event and deadline.
> > * execution time: length of time it takes for processing of event
> > to occur on a particular hardware platform
> > (uninterrupted).
> >
> > The particular values depend on use-case. For the case
> > where the realtime application executes in a virtualized
> > guest, an IPI which must be serviced in the host will cause
> > the following sequence of events:
> >
> > 1) VM-exit
> > 2) execution of IPI (and function call)
> > 3) VM-entry
> >
> > Which causes an excess of 50us latency as observed by cyclictest
> > (this violates the latency requirement of vRAN application with 1ms TTI,
> > for example).
> >
> > invalidate_bh_lrus calls an IPI on each CPU that has non empty
> > per-CPU cache:
> >
> > on_each_cpu_cond(has_bh_in_lru, invalidate_bh_lru, NULL, 1);
> >
> > The performance when using the per-CPU LRU cache is as follows:
> >
> > 42 ns per __find_get_block
> > 68 ns per __find_get_block_slow
> >
> > Given that the main use cases for latency sensitive applications
> > do not involve block I/O (data necessary for program operation is
> > locked in RAM), disable per-CPU buffer_head caches for isolated CPUs.
Hi Frederic,
> So what happens if they ever do I/O then? Like if they need to do
> some prep work before entering an isolated critical section?
Then instead of going through the per-CPU LRU buffer_head cache
(__find_get_block), isolated CPUs will work as if their per-CPU
cache is always empty, going through the slowpath
(__find_get_block_slow). The algorithm is:
/*
* Perform a pagecache lookup for the matching buffer. If it's there, refresh
* it in the LRU and mark it as accessed. If it is not present then return
* NULL
*/
struct buffer_head *
__find_get_block(struct block_device *bdev, sector_t block, unsigned size)
{
struct buffer_head *bh = lookup_bh_lru(bdev, block, size);
if (bh == NULL) {
/* __find_get_block_slow will mark the page accessed */
bh = __find_get_block_slow(bdev, block);
if (bh)
bh_lru_install(bh);
} else
touch_buffer(bh);
return bh;
}
EXPORT_SYMBOL(__find_get_block);
I think the performance difference between the per-CPU LRU cache
VS __find_get_block_slow was much more significant when the cache
was introduced. Nowadays its only 26ns (moreover modern filesystems
do not use buffer_head's).
> Thanks.
Thank you for the review.
> >
> > Signed-off-by: Marcelo Tosatti <mtosatti@xxxxxxxxxx>
> >
> > diff --git a/fs/buffer.c b/fs/buffer.c
> > index a7fc561758b1..49e9160ce100 100644
> > --- a/fs/buffer.c
> > +++ b/fs/buffer.c
> > @@ -49,6 +49,7 @@
> > #include <trace/events/block.h>
> > #include <linux/fscrypt.h>
> > #include <linux/fsverity.h>
> > +#include <linux/sched/isolation.h>
> >
> > #include "internal.h"
> >
> > @@ -1289,7 +1290,7 @@ static void bh_lru_install(struct buffer_head *bh)
> > * failing page migration.
> > * Skip putting upcoming bh into bh_lru until migration is done.
> > */
> > - if (lru_cache_disabled()) {
> > + if (lru_cache_disabled() || cpu_is_isolated(smp_processor_id())) {
> > bh_lru_unlock();
> > return;
> > }
> > @@ -1319,6 +1320,10 @@ lookup_bh_lru(struct block_device *bdev, sector_t block, unsigned size)
> >
> > check_irqs_on();
> > bh_lru_lock();
> > + if (cpu_is_isolated(smp_processor_id())) {
> > + bh_lru_unlock();
> > + return NULL;
> > + }
> > for (i = 0; i < BH_LRU_SIZE; i++) {
> > struct buffer_head *bh = __this_cpu_read(bh_lrus.bhs[i]);
> >
> >
>
>
