On Tue, Dec 01, 2015 at 11:08:47AM +0200, Avi Kivity wrote: > On 11/30/2015 06:14 PM, Brian Foster wrote: > >On Mon, Nov 30, 2015 at 04:29:13PM +0200, Avi Kivity wrote: > >> > >>On 11/30/2015 04:10 PM, Brian Foster wrote: ... > >The agsize/agcount mkfs-time heuristics change depending on the type of > >storage. A single AG can be up to 1TB and if the fs is not considered > >"multidisk" (e.g., no stripe unit/width is defined), 4 AGs is the > >default up to 4TB. If a stripe unit is set, the agsize/agcount is > >adjusted depending on the size of the overall volume (see > >xfsprogs-dev/mkfs/xfs_mkfs.c:calc_default_ag_geometry() for details). > > We'll experiment with this. Surely it depends on more than the amount of > storage? If you have a high op rate you'll be more likely to excite > contention, no? > Sure. The absolute optimal configuration for your workload probably depends on more than storage size, but mkfs doesn't have that information. In general, it tries to use the most reasonable configuration based on the storage and expected workload. If you want to tweak it beyond that, indeed, the best bet is to experiment with what works. > > > >>Are those locks held around I/O, or just CPU operations, or a mix? > >I believe it's a mix of modifications and I/O, though it looks like some > >of the I/O cases don't necessarily wait on the lock. E.g., the AIL > >pushing case will trylock and defer to the next list iteration if the > >buffer is busy. > > > > Ok. For us sleeping in io_submit() is death because we have no other thread > on that core to take its place. > The above is with regard to metadata I/O, whereas io_submit() is obviously for user I/O. io_submit() can probably block in a variety of places afaict... it might have to read in the inode extent map, allocate blocks, take inode/ag locks, reserve log space for transactions, etc. It sounds to me that first and foremost you want to make sure you don't have however many parallel operations you typically have running contending on the same inodes or AGs. Hint: creating files under separate subdirectories is a quick and easy way to allocate inodes under separate AGs (the agno is encoded into the upper bits of the inode number). Reducing the frequency of block allocation/frees might also be another help (e.g., preallocate and reuse files, 'mount -o ikeep,' etc.). Beyond that, you probably want to make sure the log is large enough to support all concurrent operations. See the xfs_log_grant_* tracepoints for a window into if/how long transaction reservations might be waiting on the log. Brian > _______________________________________________ > xfs mailing list > xfs@xxxxxxxxxxx > http://oss.sgi.com/mailman/listinfo/xfs _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
