Thanks Honza and Brauner,
For the question moving f_ref to 2nd cache line, in rocksdb readwhilewriting test, f_ref contends with f_path, f_pos_lock and f_security; while in 3rd cache line, it contends with prev_pos of file_ra_state; the score of placing in 2nd cache line(exchange f_security) is observed ~10% worse than in 3rd cache line.
Moving f_owner to 1st cache line(in vfs-6.15.misc) keeps the same performance according to rocksdb test.
Best Regards
Pan
> -----Original Message-----
> From: Jan Kara <jack@xxxxxxx>
> Sent: Tuesday, March 4, 2025 6:36 PM
> To: Christian Brauner <brauner@xxxxxxxxxx>
> Cc: Jan Kara <jack@xxxxxxx>; Deng, Pan <pan.deng@xxxxxxxxx>;
> viro@xxxxxxxxxxxxxxxxxx; linux-fsdevel@xxxxxxxxxxxxxxx; Li, Tianyou
> <tianyou.li@xxxxxxxxx>; tim.c.chen@xxxxxxxxxxxxxxx; Zhu, Lipeng
> <lipeng.zhu@xxxxxxxxx>
> Subject: Re: [PATCH] fs: place f_ref to 3rd cache line in struct file to resolve false
> sharing
>
> On Sat 01-03-25 11:07:12, Christian Brauner wrote:
> > > > @@ -1127,6 +1127,7 @@ struct file {
> > > > struct file_ra_state f_ra;
> > > > freeptr_t f_freeptr;
> > > > };
> > > > + file_ref_t f_ref;
> > > > /* --- cacheline 3 boundary (192 bytes) --- */ }
> > > > __randomize_layout
> > >
> > > This keeps struct file within 3 cachelines but it actually grows it
> > > from
> > > 184 to 192 bytes (and yes, that changes how many file structs we can
> > > fit in a slab). So instead of adding 8 bytes of padding, just pick
> > > some read-mostly element and move it into the hole - f_owner looks
> > > like one possible candidate.
> >
> > This is what I did. See vfs-6.15.misc! Thanks!
>
> Thanks. Looks good! BTW now I've realized that with struct file having 184
> bytes, the cacheline alignment of the structure need not be the one described
> by the struct definition due to the allocation starting in the middle of the
> cacheline. It will introduce some noise in the results but I guess overall this
> should still be a win.
>
> Honza
> --
> Jan Kara <jack@xxxxxxxx>
> SUSE Labs, CR
