On Wed, Sep 29, 2010 at 12:04:25AM -0400, Christoph Hellwig wrote: > On Wed, Sep 29, 2010 at 10:51:40AM +1000, Dave Chinner wrote: > > From: Dave Chinner <dchinner@xxxxxxxxxx> > > > > Under heavy parallel unlink workloads, the incore superblock lock is > > heavily trafficed in xfs_mod_incore_sb_batch(). This is despite the > > fact that the counters being modified are typically the counters > > that are per-cpu and do not require the lock. IOWs, we are locking > > and unlocking the superblock lock needlessly, and the result is that > > it is third most heavily contended lock in the system under these > > workloads. > > > > Fix this by only locking the superblock lock when we are modifying a > > counter protected by it. This completely removes the m_sb_lock from > > lock_stat traces during create/remove workloads. > > God spot of the idiocy there, but I really don't like the patch. Fair enough - it is a rather quick hack. ;) > I've started writing a small patches series solving the issue slightly > better by cleaning up this area a bit. Oh, cool. That code is quite a tangle. > After this we will never use xfs_mod_incore_sb/xfs_mod_incore_sb_batch > for the percpu counters but rather make those always go through > xfs_icsb_modify_counters. I'll need to quickly finish it up and will > send it out soon. FWIW, I've got a prototype that converts the per-cpu counters to the generic per-cpu counter infrastructure. It chops out almost all the xfs_icsb_* stuff (including xfs_icsb_modify_counters()) and has a diffstat of: 6 files changed, 317 insertions(+), 709 deletions(-) It needs a significant cleanup of xfs_mod_incore_sb() before/after the conversion which I haven't done yet because I haven't quite got my new percpu_counter_test_and_add_delta() function working correctly yet. I spotted this locking problem when testing the patch... That said, there's no reason why my percpu counter code needs to run through xfs_mod_incore_sb() at all. If we have a separate path for per-cpu counters then I can rework my code on top of that.... Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
