On Mon, Jan 12, 2015 at 05:09:01PM +0100, Carsten Aulbert wrote: > Hi Brian > > On 01/12/2015 04:52 PM, Brian Foster wrote: > > I can't see any symbols associated with the perf output. I suspect > > because I'm not running on your kernel. It might be better to run 'perf > > report -g' and copy/paste the stack trace for some of the larger > > consumers. > > > > Sorry, I rarely need to use perf and of course forgot that the > intermediate output it tightly coupled to the running kernel. Attaching > the output of perf report -g here. It does look like we're spending most of the time down in xfs_diallog_ag(), which is the algorithm for finding a free inode in the btree for a particular AG when we know that existing records have some free. > > > > Sounds good. FWIW, something like the following should tell us how many > > free inodes are available in each ag, and thus whether we have to search > > for free inodes in existing records rather than allocate new ones: > > > > for i in $(seq 0 15); do > > xfs_db -c "agi $i" -c "p freecount" <dev> > > done > > > Another metric :) > > freecount = 53795884 > freecount = 251 > freecount = 45 > freecount = 381 > freecount = 11009 > freecount = 6748 > freecount = 663 > freecount = 595 > freecount = 693 > freecount = 9089 > freecount = 37122 > freecount = 2657 > freecount = 60497 > freecount = 1790275 > freecount = 54544 > > That looks... not really uniform to me. > No, but it does show that there are a bunch of free inodes scattered throughout the existing records in most of the AGs. The finobt should definitely help avoid the allocation latency when this occurs. It is interesting that you have so many more free inodes in ag 0 (~53m as opposed to several hundreds/thousands in others). What does 'p count' show for each ag? Was this fs grown to the current size over time? Brian > Cheers > > Carsten > > _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
