On Tue, 2012-05-22 at 17:26 +0200, Thomas Gleixner wrote:
> On Tue, 15 May 2012, Steven Rostedt wrote:
> > +struct rw_semaphore {
> > + int initialized;
> > + struct __rw_semaphore lock[NR_CPUS];
>
> So that will blow up every rw_semaphore user by
>
> NR_CPUS * sizeof(struct __rw_semaphore)
>
> With lockdep off thats: NR_CPUS * 48
>
> With lockdep on thats: NR_CPUS * 128 + NR_CPUS * 8 (__key)
>
> So for NR_CPUS=64 that's 3072 or 8704 Bytes.
For a box that has 64 CPUS, 8k should be nothing (even for every task).
But then again, NR_CPUS is compile time option. It would be nice if we
could make NR_CPUS just what was actually available :-/
>
> That'll make e.g. XFS happy. xfs_inode has two rw_sems.
>
> sizeof(xfs_inode) in mainline is: 856 bytes
>
> sizeof(xfs_inode) on RT is: 1028 bytes
>
> But with your change it would goto (NR_CPUS = 64):
>
> 1028 - 96 + 2 * 3072 = 7076 bytes
>
> That's almost an order of magnitude!
>
> NFS has an rwsem in the inode as well, and ext4 has two.
>
> So we trade massive memory waste for how much performance?
We could always make this an option. I may be able to also do linker
tricks to make it a boot time option where the memory is allocated in
sections that can be freed if the option is not enabled. Just a thought,
I know this is making it more complex than necessary.
>
> We really need numbers for various scenarios. There are applications
> which are pretty mmap heavy and it would really surprise me when
> taking NR_CPUS locks in one go is not going to cause a massive
> overhead.
Well, it doesn't take NR_CPUS locks, it takes possible_cpus() locks,
which may be much smaller. As a compiled time NR_CPUS=64 running on a
box with just 4 cpus will do a loop of 4 and not 64.
I'm all for benchmarks. But right now, making all readers pass through a
single mutex is a huge bottle neck for a lot of loads. Yes, they are
mostly Java loads, but for some strange reason, our customers seems to
like to run Java on our RT kernel :-p
-- Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
