On Fri, Jul 07 2017, Phil Kauffman wrote:
>> I can imagine /etc/mtab being read once for every line in /etc/exports,
>> but unless your /etc/exports is very big, I can't easily see why it
>> would be read 900 times.
>> Maybe lots of different mount points are being accessed by something and
>> each one triggers a few reads...
>>
>> Can you show your /etc/exports file?
>
> These get generated via puppet and dropped into /etc/exports.d/
...
>
> Here we can see that there are 960 files in /etc/exports.d
> root@storage1:~# find /etc/exports.d/ -type f | wc
> 960 960 88701
Ahhh.. 960 exports. That could do it.
$ grep -c crossmnt all_etc_exports.d_in_one_file.txt
957
I get 957 - much the same number.
$ grep open redacted_strace.txt | uniq -c
2 open("/var/lib/nfs/etab", O_RDONLY) = 6
942 open("/etc/mtab", O_RDONLY|O_CLOEXEC) = 6
1 open("/proc/net/rpc/nfsd.export/channel", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 6
/etc/mtab was ready 942 for a single access. I would have expected 957,
but maybe the system is dynamic and something changed between the two
samples.
This makes it fairly clear what is happening. Now we just need to fix
it.
One option would be to cache some of the details extracted from
/etc/mtab, but that could get messy.
Another is to move the code around. In your case there are really just
3 exports to each of 300+ clients (I assume "client.cs.uchicago.edu" in
the combined exports file is really different in different files).
So any one client only needs to consider 3 exports, not 300.
There is room to optimize this code further than the below, but let's
start simple. Could you test with this patch applied and see what
difference it makes?
Thanks,
NeilBrown
diff --git a/utils/mountd/cache.c b/utils/mountd/cache.c
index ca6c84f4d93d..e712cc166157 100644
--- a/utils/mountd/cache.c
+++ b/utils/mountd/cache.c
@@ -727,6 +727,17 @@ static void nfsd_fh(int f)
for (exp = exportlist[i].p_head; exp; exp = next_exp) {
char *path;
+ if (!is_ipaddr_client(dom)
+ && !namelist_client_matches(exp, dom)) {
+ next_exp = exp->m_next;
+ continue;
+ }
+ if (is_ipaddr_client(dom)
+ && !ipaddr_client_matches(exp, ai)) {
+ next_exp = exp->m_next;
+ continue;
+ }
+
if (exp->m_export.e_flags & NFSEXP_CROSSMOUNT) {
static nfs_export *prev = NULL;
static void *mnt = NULL;
@@ -751,9 +762,6 @@ static void nfsd_fh(int f)
next_exp = exp->m_next;
}
- if (!is_ipaddr_client(dom)
- && !namelist_client_matches(exp, dom))
- continue;
if (exp->m_export.e_mountpoint &&
!is_mountpoint(exp->m_export.e_mountpoint[0]?
exp->m_export.e_mountpoint:
@@ -762,9 +770,6 @@ static void nfsd_fh(int f)
if (!match_fsid(&parsed, exp, path))
continue;
- if (is_ipaddr_client(dom)
- && !ipaddr_client_matches(exp, ai))
- continue;
if (!found || subexport(&exp->m_export, found)) {
found = &exp->m_export;
free(found_path);
Attachment:
signature.asc
Description: PGP signature
