On Mon, May 11, 2015 at 02:07:39PM -0700, Shrinand Javadekar wrote: > In case you're curious, the solutions to this problem are being > discussed at [1] on the Swift side. > > O_TMPFILE and linkat() aren't available in all kernels (and python) > and therefore not a viable option right away. Among few others, one of > the proposals is to shard the tmp directory into say 256 dirs and > create these files inside those dirs. That way files won't get created > in a single AG. Are there any other problems if this is done? Yes, you don't get any locality between files in the same directory once you move them to their final location in the hash tree. You'll end up with the same problems you are trying to solve now as the object count goes up - you'll simply have taken a different path to the same destination.... Create your temp files in their eventual destination directory - doing anything else is pretty much guaranteed to compromise all the locality algorithms the filesystem uses. Without good locality within the data set, performance of lookups and writeback will degrade to large seek time random IO pretty quickly. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
