On Mon, Oct 09, 2017 at 11:05:56AM +0300, Avi Kivity wrote: > On 10/07/2017 01:21 AM, Eric Sandeen wrote: > >On 10/6/17 5:20 PM, Dave Chinner wrote: > >>On Fri, Oct 06, 2017 at 11:18:39AM -0500, Eric Sandeen wrote: > >>>On 10/6/17 10:38 AM, Darrick J. Wong wrote: > >>>>On Fri, Oct 06, 2017 at 10:46:20AM +0200, Gandalf Corvotempesta wrote: > >>>>Semirelated question: for a solid state disk on a machine with high CPU > >>>>counts do we prefer agcount == cpucount to take advantage of the > >>>>high(er) iops and lack of seek time to increase parallelism? > >>>> > >>>>(Not that I've studied that in depth.) > >>>Interesting question. :) Maybe harder to answer for SSD black boxes? > >>Easy: switch to multidisk mode if /sys/block/<dev>/queue/rotational > >>is zero after doing all the other checks. Then SSDs will get larger > >>AG counts automatically. > >The "hard part" was knowing just how much parallelism is actually inside > >the black box. > > It's often > 100. Sure, that might be the IO concurrency the SSD sees and handles, but you very rarely require that much allocation parallelism in the workload. Only a small amount of the IO submission path is actually allocation work, so a single AG can provide plenty of async IO parallelism before an AG is the limiting factor. i.e. A single AG can typically support tens of thousands of free space manipulations per second before the AG locks become the bottleneck. Hence by the time you get to 16 AGs there's concurrency available for (runs a concurrent workload and measures) at least 350,000 allocation transactions per second on relatively slow 5 year old 8-core server CPUs. And that's CPU bound (16 CPUs all at >95%), so faster, more recent CPUs will run much higher numbers. IOws, don't confuse allocation concurrency with IO concurrency or application concurrency. It's not the same thing and it is rarely a limiting factor for most workloads, even the most IO intensive ones... > > But "multidisk mode" doesn't go too overboard, so yeah > >that's probably fine. > > Is there a penalty associated with having too many allocation groups? Yes. You break up the large contiguous free spaces into many smaller free spaces and so can induce premature onset of filesystem aging related performance degradations. And for spinning disks, more than 4-8AGs per spindle causes excessive seeks in mixed workloads and degrades performance that way.... Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
