Re: disk i/o: very high write rates and poor search performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 08/15/2018 10:13 AM, David Boreham wrote:

in strace.log:

[pid  8088] 12:55:39.739539 poll([{fd=435, events=POLLOUT}], 1, 1800000
<unfinished ...>
[pid  8058] 12:55:39.739573 <... write resumed> ) = 1 <0.000087>
[pid  8088] 12:55:39.739723 <... poll resumed> ) = 1 ([{fd=435,
revents=POLLOUT}]) <0.000168>
[pid  8058] 12:55:39.739754 write(426, "dn: cn=ntU"..., 344 <unfinished ...>
[pid  8088] 12:55:39.739824 sendto(435, "0<\2\1\2d7\0043u"..., 62, 0,
NULL, 0 <unfinished ...>

Usually you need to do a few iterations on the strace data to get to the bottom of things.

I took a look at the full file you uploaded. Couple of things :

1. With a truncated strace it helps to know the identity of the FDs. You can use "lsof" or dump the data out of /proc /sys...somewhere.

2. You can tell strace to only output certain classes of syscalls. For this problem, where filesystem I/O is the target, you can use : strace -e trace=file and -e trace=desc. More details here : https://linux-audit.com/the-ultimate-strace-cheat-sheet/ and also in the man page. This will remove the extra noise (gettimeofday() etc) in the output.

That said, I saw this, which looks possibly like a smoking gun:

[pid  8058] 12:55:40.563393 open("/etc/dirsrv/slapd-ldap0/dse.ldif.tmp", O_RDWR|O_CREAT|O_TRUNC, 0600 <unfinished ...> [pid  3186] 12:55:40.563826 <... select resumed> ) = 0 (Timeout) <0.009394>
[pid  8058] 12:55:40.563855 <... open resumed> ) = 426 <0.000431>

It looks to have opened dse.ldif, then assuming that FD = 426 is from the open() call, it goes on to write a bunch of data to that file :

[pid  8058] 12:55:40.567465 write(426, "dn: cn=mon"..., 413) = 413 <0.000209>
[pid  8058] 12:55:40.567826 write(426, "\n", 1) = 1 <0.000356>
[pid  8058] 12:55:40.568601 write(426, "dn: cn=cha"..., 318) = 318 <0.000058>
[pid  8058] 12:55:40.568727 write(426, "\n", 1) = 1 <0.000045>
[pid  8058] 12:55:40.568937 write(426, "dn: cn=enc"..., 321) = 321 <0.000042>
[pid  8058] 12:55:40.569040 write(426, "\n", 1) = 1 <0.000041>
[pid  8058] 12:55:40.569182 write(426, "dn: cn=fea"..., 100) = 100 <0.000042>
[pid  8058] 12:55:40.569281 write(426, "\n", 1) = 1 <0.000040>
[pid  8058] 12:55:40.569427 write(426, "dn: cn=kol"..., 409) = 409 <0.000042>
[pid  8058] 12:55:40.569528 write(426, "\n", 1) = 1 <0.000041>

dse.ldif is supposed to hold seldom-changing config data for the server.

So...one theory is that something the server is doing, or you are unknowingly asking it to do, is causing dse.ldif to be re-written constantly.

Updating the csn generator and the uuid generator will cause a lot of churn in dse.ldif.  There are other housekeeping tasks which will write dse.ldif


As you point out, the high filesystem I/O may not be the cause for the poor search performance, but certainly the server should not be constantly re-writing dse.ldif.




_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx/message/4GAY2YSXU34GG5AKNG5Z5KHOBQXSP3MC/

_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx/message/R7I3LQRL7KEMP3BBIITUYC3DNGCIHHJG/




[Index of Archives]     [Fedora User Discussion]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [Fedora News]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Maintainers]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Legacy]     [Fedora Desktop]     [Fedora Fonts]     [ATA RAID]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Centos]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora QA]     [Fedora Triage]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Tux]     [Yosemite News]     [Yosemite Photos]     [Linux Apps]     [Maemo Users]     [Gnome Users]     [KDE Users]     [Fedora Tools]     [Fedora Art]     [Fedora Docs]     [Maemo Users]     [Asterisk PBX]     [Fedora Sparc]     [Fedora Universal Network Connector]     [Fedora ARM]

  Powered by Linux