Hi Robert,
interesting. It seems resolved is not expecting so heavy usage. Consider
other cache until this is fixed, unbound or dnsmasq might be a good choice.
Please create an issue for it on https://github.com/systemd/systemd.
Especially when it can use the fact it is a deamon, it should be able to
use notifications of /etc/resolv.conf symlink changes. It should not
require stat of that file on each query. It definitely should save that
when the resolve NSS plugin is the first one, even before files.
It should watch the file with some filesystem notification system.
Especially when it already knows it is the service supposed to maintain
that file. I don't understand why it is checking both resolv.conf and
stub-resolv.conf. Only one of them should be under active use.
Even if it could not use filesystem monitoring, I guess it could check
those files only once per second or so. Should not depend on number of
done queries.
Please share your distribution and systemd version in it, it is unclear
what version are we talking about. resolvectl status command output
would be also useful.
On 3/14/23 19:25, Robert Ayrapetyan wrote:
Hello, I'm using systemd-resolved on the server which performs a lot
of DNS queries (~20K per second) and systemd-resolved helps a lot
providing a cache:
Cache
Current Cache Size: 263
Cache Hits: 30928976
Cache Misses: 2961
However, systemd-resolved process almost constantly utilizes 100% of
CPU (I don't have any other dns services like dnsmasq installed).
strace shows this:
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
48.73 0.966765 4 194141 stat
15.06 0.298804 12 24106 sendmsg
13.59 0.269703 8 32346 32346 openat
6.42 0.127381 5 24118 recvmsg
5.49 0.108829 4 24118 recvfrom
5.34 0.106037 4 24118 epoll_wait
5.33 0.105838 4 24118 gettid
0.01 0.000151 6 24 epoll_ctl
And in particular for stat, it queries just 3 files in a loop:
stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=738, ...}) = 0
<0.000010>
stat("/run/systemd/resolve/resolv.conf", {st_mode=S_IFREG|0644,
st_size=624, ...}) = 0 <0.000009>
stat("/run/systemd/resolve/stub-resolv.conf", {st_mode=S_IFREG|0644,
st_size=738, ...}) = 0 <0.000013>
My question is: why does it "stats" so often? Is this a result of my
misconfiguration?
/etc/systemd/resolved.conf:
...
FallbackDNS=1.1.1.1 1.0.0.1
/etc/nsswitch.conf:
...
hosts: resolve [!UNAVAIL=return] files dns
--
Petr Menšík
Software Engineer, RHEL
Red Hat, https://www.redhat.com/
PGP: DFCF908DB7C87E8E529925BC4931CA5B6C9FC5CB
