On Donnerstag, 21. Juli 2011 Dave Chinner wrote: > If you are writing files that grow like this, then you are doing > something wrong. If the app can't do it's IO differently, then this > is exactly the reason we have userspace-controlled preallocation > interfaces. > > Filesystems cannot prevent user stupidity from screwing something > up.... This can happen if you copy a syslog server over to a new disk, then let it start it's work again. Many files that start small and grow. Luckily, the logs are rotated latest monthly, so it shouldn't be too bad. > > And files >64KiB are immediately fragmented > > then. At this time, there are only 16384 * 2KiB = 32MiB used, which > > is 3,125% of the disk. I can't believe my numbers, are they true? > > No, because most filesystems have a 4k block size. I just meant pure disk usage. Of 1GB, only 32MB are used, and this worst case example hits us badly. > Not to mention > that fragmentation is likely to be limited to the single AG the files > in the directory belong to. i.e. even if we can't allocation a sunit > aligned chunk in an AG, we won't switch to another AG just to do > sunit aligned allocation. This is good to know also, thanks. > > OK, this is a worst case scenario, and as you've said before, any > > filesystem can be considered full at 85% fill grade. But it's > > incredible how quickly you could fuck up a filesystem when using > > su/sw and writing small files. > > Well, don't use a filesystem that is optimised for storing large > sizes, large files and high bandwidth for storing lots of small > files, then. Indeed, the point of not packing the files is so they > -don't fragemnt as they grow-. XFS is not designed to be optimal > for small filesystems or small files. In most cases it will deal > with them just fine, so in reality your concerns are mostly > unfounded... Yes, I just wanted to know about the corner cases, and how XFS behaves. Actually, we're changing over to using NetApps, and with their WAFL anyway I should drop all su/sw usage and just use 4KB blocks. And even when XFS is optimized for large files, there are often small ones. Think of a mysql server with hundreds of DBs and innodb_file_per_table set. Even when some DBs are large, there are many small files. But this thread has drifted a bit. XFS does great work, and now I understand the background a bit more. Thanks, Dave. -- mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services: Protéger http://proteger.at [gesprochen: Prot-e-schee] Tel: +43 660 / 415 6531 // Haus zu verkaufen: http://zmi.at/langegg/
Attachment:
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
