On 03/13/2018 04:59 PM, Rick Stevens wrote: > On 03/13/2018 02:21 PM, Stephen Morris wrote: >> On 13/3/18 8:46 pm, Patrick O'Callaghan wrote: >>> On Mon, 2018-03-12 at 18:25 -0700, Rick Stevens wrote: >>>> On 03/12/2018 03:37 PM, Patrick O'Callaghan wrote: >>>>> On Tue, 2018-03-13 at 07:26 +1100, Stephen Morris wrote: >>>>>>> 'du' with no parameters recursively lists all the subdirectories and >>>>>>> their sizes, along with the grand total. When applied to my home >>>>>>> directory, I get over 30,000 lines of output. That's almost never >>>>>>> what >>>>>>> I want. My usual call is 'du -hs'. >>>>>>> >>>>>>> poc >>>>>> Thanks Patrick, taking this a step further, it seems to me that the >>>>>> only >>>>>> parameter for du that, to me, provides the correct file size is -b as >>>>>> shown below. >>>>> I think you have a misconception here. 'du' does not give file sizes, >>>>> it gives disk usage. A 1-byte file takes up at least 1 disk block, so >>>>> that's the size 'du' will give. I seem to remember that it also counts >>>>> indirect blocks and other housekeeping that corresponds to the file >>>>> without being included in the file's content, but I could be mistaken >>>>> (though I'm fairly sure early versions did do that). >>>> du (with no flags) gives disk usage. As Patrick says, a 1-byte file >>>> uses one disk block (which is generally 4KiB) and that's what du is >>>> reporting (after all, "du" means "disk usage"). The "-b" flag means >>>> "set the block size to 1 byte and show the apparent size", which is what >>>> "ls -l" would report (there may be differences between du and ls if >>>> sparse files are involved). >>> Good point about sparse files, which I'd forgotten in this context. A >>> sparse file can be 10TB in size yet occupy only 1 disk block. 'du' will >>> give the usage as 1 block and 'ls' will say it's 10TB. >>> >>>> Also, du walks down the entire current directory unless you give it >>>> arguments to tell it what to look at. Note that the arguments you pass >>>> it are interpreted by the _shell_, not "du" (even the man page says >>>> "PATTERN is a shell pattern (not a regular expression)"). >>>> >>>> This is a common confusion point with many people. Unless you enclose >>>> shell metacharacters in quotes (and dots and splats are metacharacters), >>>> the shell WILL interpret them--sometimes in ways you aren't expecting! >>>> By default, shell globbing does NOT expand filenames that start with a >>>> dot (to the shell, a dot means "current directory"). >>> Slight nit: '.' (on its own) means 'current directory' to the kernel, >>> not to the Shell, i.e. it's wired into the system and doesn't depend on >>> the command interpreter (in the same way that '/' is the path separator >>> and cannot be changed). The use of dot-files as a way of hiding certain >>> names is IIRC originally just a convention of 'ls' which the Shell >>> adopted for consistency. >>> >>> poc >> >> Thanks guys, I fully understood the -b parameter was giving an apparent >> size of the file (which approximately matches what ls -l provides), that >> sparse files can impact the apparent size, and that the physical size of >> a file is governed by the sector size, but my usage of du, especially >> with the -s parameter, is the amount of "apparent" space occupied by >> files and directories (where possible I usually use a sector size of >> 512) to identify space hogs. I'm also fully aware of the sector size >> trade off between efficient space utilization and efficient I/O and the >> apparent space used by files. >> >> What I hadn't realized until Patrick mentioned it, was the significance >> of the * in the path specification. I hadn't realized that with using >> the * not only did it cause files prefixed with a '.' to be ignored but >> it also causes directories prefixed with a '.' to be ignored also. > > No. What you have to get straight is that when you use "*", you're > asking the shell to create a list of files and return it, and by default > the shell doesn't list files starting with a dot. > > To clarify, assume you have a directory containing the files "file1", > "file2", "file3" and ".dotfile". If you "cd" to it and do an "ls *" > (asking the shell to expand the glob), you'll get a list of "file1", > "file2" and "file3". The same is true if the command was "du" and not > "ls". So the command: > > du -a * > > is exactly equivalent to: > > du -a file1 file2 file3 > > If you simply do: > > du -a > > then du will walk down the _directory_ you're in and display all of the > files, including ".dotfile", since du does not ignore files starting > with a dot. Here's an example: [root@golem4 ~]# mkdir /tmp/testdir [root@golem4 ~]# cd /tmp/testdir [root@golem4 testdir]# touch file1 file2 file3 .dotfile [root@golem4 testdir]# echo "Without shopt:";ls *;echo "With shopt:";shopt -s dotglob;ls *;echo "Without shopt again:";shopt -u dotglob;ls * Without shopt: file1 file2 file3 With shopt: .dotfile file1 file2 file3 Without shopt again: file1 file2 file3 [root@golem4 testdir]# du * 0 file1 0 file2 0 file3 [root@golem4 testdir]# du -a * 0 file1 0 file2 0 file3 [root@golem4 testdir]# du -a 0 ./file2 0 ./file3 0 ./.dotfile 0 ./file1 4 . Hope that clarifies it a bit. And when I say "files that start with a dot", that includes directories, devices, whatever (remember, essentially in Unixish systems, EVERYTHING is some type of a file). ---------------------------------------------------------------------- - Rick Stevens, Systems Engineer, AllDigital ricks@xxxxxxxxxxxxxx - - AIM/Skype: therps2 ICQ: 22643734 Yahoo: origrps2 - - - - To err is human, to moo bovine. - ---------------------------------------------------------------------- _______________________________________________ users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
