On 10.05.2011 23:17, Dave Chinner wrote: > On Tue, May 10, 2011 at 12:57:00PM +0200, Matthias Schniedermeyer wrote: > > Hi > > > > > > Since a few weeks i'm experiencing an annoying 'thing' where files are > > often too big in `du` and directory totals are to high in `ls -l`. > > > > I appears that files, which are in the process of beeing > > copied/downloaded/whatever, grow in large chunks ahead of time, while > > the actual file-content is beeing copied into the files. > > It's supposed to work like this. It's called speculative allocation > beyond end of file. XFS has always done this, but we've recently > made it more aggressive to prevent excessive fragmentation on > concurrent large file workloads when there is lots of disk space > free. OK. > > And then it > > appears that the last chunk isn't shrunk after the process is finished. > > It should be truncated away when the file descriptor is closed and > the last reference goes away. > > > Neither xfs_bmap (Version 3.1.5) nor filefrag show anything beyond the > > extent that compromises the actual file-content. > > what is the output of xfs_bmap -vvp on a file that apparently hasn't > been shrunk? How do you know it hasn't been shrunk? Does it persist du > forever in this state, or does doing something like dropping caches > (echo 3 > /proc/sys/vm/drop_caches) cause the specualtive > preallocation to disappear? This works: sync ; echo 3 > /proc/sys/vm/drop_caches At least in several tries the `du` output shrunk to the size of the original. > > Any idea how to debug this, or is this a known bug and waiting a few > > days for 2.6.39 should fix this? > > It doesn't appear to be doing anything wrong from your description. > Remember that XFS is optimised for high end storage and server > configurations and workloads, not typical desktop usage... I would call it a regression. I reguarly follow copying/downloading with `du`, the speculative preallocation makes that more or less useless. Especially downloading someting big from the internet which @ 231kb/s isn't exactly fast and shows identical `du`s for increasingly longer periods of time. (Or "--apparent-size" should be made default, but that falls short with sparse-files) IMHO `du`/`ls -l` should not be able to 'see' the speculative preallocation. Bis denn -- Real Programmers consider "what you see is what you get" to be just as bad a concept in Text Editors as it is in women. No, the Real Programmer wants a "you asked for it, you got it" text editor -- complicated, cryptic, powerful, unforgiving, dangerous. _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
