On Sat, Oct 25, 2014 at 01:12:48PM -0500, Stan Hoeppner wrote: > I recall reading a while back something about disabling the filestreams > allocator, or at least changing its behavior, but I'm unable to find that. > > What I'm trying to do is use parallel dd w/O_DIRECT to write 44 files in > parallel to 44 directories, thus all 44 AGs, in one test, then write 44 > files to one dir, one AG, in another test. The purpose of this > quick/dirty exercise is to demonstrate throughput differences due to > full platter seeking in the former case and localized seeking in the > latter case. > > But of course the problem I'm running into in the single directory case > is that the filestreams allocator starts writing all of the 44 files > into the appropriate AG, but then begins allocating extents for each > file in other AGs. This is of course defeating the purpose of the tests. That's caused by allocator contention. When you try to write 44 files to the same dir in parallel, they'll all start with the same target AG, but then when one thread is allocating into AG 43 and has the AG locked, a second attempt to allocate to than AG will see the AG locked and so it will move to find the next AG that is not locked. Remember, AGs were not originally designed for confining physical locality - they are designed to allow allocator parallelism. Hence once the file has jumped to a new AG it will try to allocate sequentially from that point onwards in that same AG, until either ENOSPC or further contention. Hence with a workload like this, if the writes continue for long enough each file will end up finding it's own uncontended AG and hence mostly end up contiguous on disk and not getting blocked waiting for allocation on other files. When you have as many writers as there are AGs, however, such a steady state is generally not possible as there will always be files trying to write into the same AG. As it is, filestreams is not designed for this sort of parallel workload. filestreams is designed to separate single threaded streams of IO into different locations, not handle concurrent writes into multiple files in the same directory. As it is, the inode64 will probably demonstrate exactly the same behaviour because it will start by trying to write all the files to the same AG and hence hit allocator contention, too. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
