Re: File limit inside a single directory

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



If you are using ext3 then you will get an EMLINK error at 32000 subdirs. 
If you are using ext4 with the "dir_nlink" feature enabled then you
can create several million subdirs if you want.

As for performance, if you are not regularly creating and deleting the
directory I don't think you would even notice the performance difference
of a large directory.  The create and unlink rate get somewhat slower at
100k files (from 25k/sec to 10k/sec) and 1M entries or so (to 5k/sec).
For lookups the whole directory will easily fit into cache (at about 40MB
for 1M entries) so the lookup rate is not affected much by size. 

Cheers, Andreas

> On Feb 14, 2015, at 03:17, suntrop@xxxxxx wrote:
> 
> Sorry for confusion, but now I am confused :-) My setup will be like
> /files/
> /files/1/
> /files/2/
> /files/3/
> (...)
> /files/24244/
> (...)
> /files/113524/
> 
> Thus all folders reside within the /files/ directory (can't change that). 
> What happens when the max link count reach 32,000 and I create more folders? Will this result in any errors or problems? I’m using ext3. Is the only consequence that the links aren’t counting up any more, like ext4?
>  
>  
> 
> Gesendet: Freitag, 13. Februar 2015 um 18:28 Uhr
> Von: "Eric Sandeen" <sandeen@xxxxxxxxxx>
> An: suntrop@xxxxxx, "Andreas Dilger" <adilger@xxxxxxxxx>
> Cc: "linux-ext4@xxxxxxxxxxxxxxx" <linux-ext4@xxxxxxxxxxxxxxx>
> Betreff: Re: File limit inside a single directory
>> On 2/13/15 10:49 AM, suntrop@xxxxxx wrote:
>> Thanks guys. I was afraid of having a couple 100K (at most). The
>> server support team told me not to have more than 10 to 20K. There
>> seems to be a misconception (for me and people from the CMS) about
>> the 32K subdirectory limit, but this ins't for files/folders within a
>> single directory but rather nested directories like /1/2/3.../32000
> 
> You've confused things a bit here, FWIW.
> 
> The 32k (well, 32000 because, sure) limit on ext3 is max link count;
> each subdirectory increases the link count on its parent, but only
> its parent. It's not about deep nesting, or about files in a dir.
> It's only about subdirs in a parent dir.
> 
> # mkdir dir
> # stat dir | grep Links
> Device: fd06h/64774d Inode: 2490391 Links: 2 // . and .. entries
> # mkdir dir/subdir1 dir/subdir2 dir/subdir3
> # stat dir | grep Links
> Device: fd06h/64774d Inode: 2490391 Links: 5
> # mkdir dir/subdir1/subsubdir1 dir/subdir1/subsubdir2 dir/subdir1/subsubdir3
> # stat dir | grep Links
> Device: fd06h/64774d Inode: 2490391 Links: 5
> 
> ext4 bumped that max to 64000, and just stops counting if that number
> gets exceeded...
> 
> -Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Reiser Filesystem Development]     [Ceph FS]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite National Park]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Device Mapper]     [Linux Media]

  Powered by Linux