Re: gorged harddrive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, 2008-04-02 at 07:29 +0800, Ed Greshko wrote:
> Patrick O'Callaghan wrote:
> > On Tue, 2008-04-01 at 20:50 +0800, Ed Greshko wrote:
> >> Patrick O'Callaghan wrote:
> >>> On Tue, 2008-04-01 at 10:57 +0800, Ed Greshko wrote:
> >>>> charles f. zeitler wrote:
> >>>>> --- Ed Greshko <Ed.Greshko@xxxxxxxxxxx> wrote:
> >>>>>
> >>>>>> charles f. zeitler wrote:
> >>>>>>> i've been pruning my "downloads" disk,
> >>>>>>> rather drastically, and not making a dent.
> >>>>>>>
> >>>>>>> today some more, less drastic but still
> >>>>>>> hefty, same result.
> >>>>>>>
> >>>>>>> revisited du- checked it twice - three
> >>>>>>> times- yup, it reports one directory at
> >>>>>>> 800+ gb- on a 400gb disk!
> >>>>>>>
> >>>>>>> fsck (forced) failed to report any problems,
> >>>>>>> there don't seem to be any symlinks,
> >>>>>>> and the sub-direcory sizes are sane...
> >>>>>>>
> >>>>>>> any ideas welcome, and appreciated.
> >>>>>> Instead of telling people what you are seeing it would be better to show the 
> >>>>>> actual commands and output.
> >>>>>>
> >>>>> good point.
> >>>>>
> >>>>>
> >>>>> [fedora_8@Nyarlethotep ~]$ df
> >>>>> Filesystem           1K-blocks      Used Available Use% Mounted on
> >>>>> /dev/sda8             13250836  11459264   1107608  92% /
> >>>>> /dev/sda9              1898468    825572    974904  46% /tmp
> >>>>> /dev/sda11           270882768 259964688   5414052  98% /home
> >>>>> /dev/sda10             1898468   1156484    643992  65% /var
> >>>>> /dev/sdc1            480719056 370452080 105383136  78% /home/fedora_8/music_vids
> >>>>> /dev/sda2               101105     17986     77898  19% /boot
> >>>>> tmpfs                  1037552       248   1037304   1% /dev/shm
> >>>>> /dev/sdb1            384578164 330445976  34596748  91% /home/fedora_8/torrents_isos
> >>>>>
> >>>>> /dev/sdb1 is the drive under discussion.
> >>>>>
> >>>>>
> >>>>> [fedora_8@Nyarlethotep ~]$ du -sb t*s/*
> >>>>> 34256010522     torrents_isos/backup
> >>>>> 883393808812    torrents_isos/data
> >>>>> 58352749159     torrents_isos/finished
> >>>>> 75197043648     torrents_isos/finnished
> >>>>> 18222558607     torrents_isos/isos
> >>>>> 4781438 torrents_isos/logs
> >>>>> 16384   torrents_isos/lost+found
> >>>>> 4096    torrents_isos/lost_meta
> >>>>> 193903286       torrents_isos/meta
> >>>>> 1402434610      torrents_isos/new
> >>>>> 75585799469     torrents_isos/porn
> >>>>> 4096    torrents_isos/rar
> >>>>> 1318803 torrents_isos/shas
> >>>>> 4096    torrents_isos/tmp
> >>>>> 97996487        torrents_isos/total_meta.tar.bz2
> >>>>> 4096    torrents_isos/zip
> >>>>>
> >>>>> somethings wrong with t*s/data  ....
> >>>>>
> >>>> OK....  I believe I know what the problem is.  The torrents_isos/porn 
> >>>> directory makes things seem larger than what they really are....
> >>>>
> >>>> No, just kidding.....
> >>>>
> >>>> I believe you may have a bunch of non-completed torrent downloads.  When you 
> >>>> start a torrent download the client will reserve the space and it will be 
> >>>> reflected in the output of "du" but *not* in the output of "df".  Thus with 
> >>>> "du" you can have a situation where it "thinks" more disk space is being 
> >>>> used than it actually is.  FWIW, this is normal.
> >>> No.
> >> What do you mean "no"?
> >>
> >> When starting the download of a torrent the output of "du" shows the disk 
> >> space has been used.  But, in reality, it hasn't.  df reflects that is 
> >> hasn't been actually used.
> > 
> > There are two scenarios, according to whether your torrent client
> > preallocates space or not. I use Ktorrent, which can be set to do
> > either. In the default case, no preallocation is done. After starting a
> > torrent an 'ls -l' of the file shows its full size, but a 'du' shows
> > only what has actually been downloaded. I checked this before my
> > previous post and I just checked it again to be sure.
> > 
> > If preallocation is on then of course 'du' will show the allocated
> > space, but 'df' will account for it. I also checked this BTW. The
> > numbers add up.
> > 
> > In other words 'df' and 'du' are mutually consistent.
> 
> While it may be the case on your system, it isn't on mine and apparently 
> isn't on the OP's.  And if you use azureus on your system you could prove it 
> to yourself too.

Just installed Azureus and tested it. It behaves exactly the same as
Ktorrent, both with and without preallocation.

This is hardly surprising as we're talking about how the filesystem
works, not about about a specific application. As someone else pointed
out, a inconsistency between the results of 'du' and 'df' can occur when
a running process has an open file which has been unlinked, meaning the
space occupied by the file has not yet been reclaimed even though the
file no longer has a name. However, any inconsistency arising from this
would be in the *opposite* direction to what the OP is seeing, i.e. less
actual free space than accounted for by the filesystem size less the
space occupied by (visible) files.

In all of this I've been assuming that preallocation means what it has
always meant traditionally in Unix/Linux: write garbage into the file to
make sure disk space is allocated. It turns out the system can now do
this for you, with the fallocate() call. However this is a very recent
addition and people were still arguing about its semantics less than a
year ago -- see http://lwn.net/Articles/240571. One detail stands out:
with the correct parameter, the apparent size of the file (as reported
by the stat() call) does not change, even if space is allocated beyond
its end. The effect of this on 'df' and 'du' doesn't seem to be
documented anywhere. Furthermore, although it's not explicitly stated,
one presumes that the unused space is reclaimed when the file is closed,
so the OP's question still stands: how can 'du' report more space than
is being used, even if no processes have open files? For that matter,
how can the system preallocate more space than the size of the
filesystem? Doesn't make sense.

BTW, it might be worth knowing what filesystem the OP is using. In my
case it's ext3.

poc

-- 
fedora-list mailing list
fedora-list@xxxxxxxxxx
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [Fedora Magazine]     [Fedora News]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Maintainers]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Legacy]     [Fedora Desktop]     [Fedora Fonts]     [ATA RAID]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [SSH]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Centos]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Tux]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Asterisk PBX]     [Fedora Sparc]     [Fedora Universal Network Connector]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux