Re: [PATCH v3 4/4] mm: Adaptive hash table scaling

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu 04-05-17 14:23:24, Pasha Tatashin wrote:
> Hi Michal,
> 
> I do not really want to impose any hard limit, because I do not know what it
> should be.
> 
> The owners of the subsystems that use these large hash table should make a
> call, and perhaps pass high_limit, if needed into alloc_large_system_hash().

Some of surely should. E.g. mount_hashtable resp. mountpoint_hashtable
really do not need a large hash AFAIU. On the other hand it is somehow
handy to scale dentry and inode hashes according to the amount of
memory. But the scale factor should be much slower than the current
upstream implementation. As I've said I do not want to judge your
scaling change. All I am saying that making it explicit is just _wrong_
because it a) doesn't cover all cases just the two you have noticed and
b) new users will most probably just copy&paste existing users so
chances are they will introduce the same large hashtables without a good
reason. I would even say that user shouldn't care about how the scaling
is implemented. There is a way to limit it and if there is no limit set
then just do whatever is appropriate.

> 
> Previous growth rate was unacceptable, because in addition to allocating
> large tables (which is acceptable if we take a total system memory size), we
> also needed to zero that, and zeroing while we have only one CPU available
> was significantly reducing the boot time.
> 
> Now, on 32T the hash table is 1G instead of 32G, so the call is 32 times
> faster to finish. While it is not a good idea to waste memory, both 1G and
> 32G is insignificant amount of memory compared to the total amount of such
> 32T systems (0.09% and 0.003% accordingly).

Try to think in terms of hashed objects. How many objects would we need
to hash? Also this might be not a significant portion of the memory but
it is still a memory which can be used for other purposes.
-- 
Michal Hocko
SUSE Labs



[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [Samba]     [Device Mapper]     [CEPH Development]
  Powered by Linux