Re: ldapsearch performance problem

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 06/15/2018 11:50 PM, Mark Reynolds wrote:


On 06/15/2018 05:41 PM, Jan Kowalsky wrote:
Hi Marc,

thanks for help.

Am 15.06.2018 um 22:50 schrieb Mark Reynolds:
You did not run logconv.pl the way I requested, can you please run it
again this way:

logconv.pl -ulatn  <access logs>
I omitted the detailed searches because there are user-data in it...

but this here does't look like this is a problem:

----- Top 20 Most Frequent etimes -----

64902           etime=0
157             etime=1
34              etime=3
27              etime=2
18              etime=4

I think the next step, besides gathering the right logconv output, would
be to get several pstacks of the ns-slapd process at the same time the
"hang" occurs.  Then we can at least see what all the threads are doing
and hopefully get better information.  I would try and get the
389-ds-base-debuginfo package installed first though (this should give
us more readable pstacks).
hm, I installed 389-ds-base-dbg and 389-ds-base-libs-dbg. What else do I
have to do?

pstack gives me:

pstack 13463

13463: /usr/sbin/ns-slapd -D /etc/dirsrv/slapd-ldap0 -i
/var/run/dirsrv/slapd-ldap0.pid -w /var/run/dirsrv/slapd-ldap0.startpid
(No symbols found)
0x7f808a7717bc: ???? (7f803c03cf20, 0, 7f803c0421f0, 0, 7f808c0dc5d6,
5615f1d4bc80) + ffffd695834bb170
0x300000157: ???? (0, 200000, 0, 31, 3, 0) + ffffa9ea0c848bd3
crawl: Input/output error
Error tracing through process 13463

Does this help:
Are you sure the "debuginfo" packages are the same version as 389-ds-base/libs?

Run:  rpm -qa | grep 389-ds-base

The other option is attach gdb to the process and get a stack trace that way:

# gdb -p PID
(gdb) thread apply all bt
(gdb) quit



If everything runs fast:

root@ldap0:~# cat /proc/13463/stack
[<ffffffff810eaa9a>] hrtimer_start_range_ns+0x1aa/0x3d0
[<ffffffff810ea660>] hrtimer_init+0x110/0x110
[<ffffffff8121ce26>] poll_schedule_timeout+0x46/0x70
[<ffffffff8121e50f>] do_sys_poll+0x44f/0x560
[<ffffffff814aab95>] loopback_xmit+0x65/0xa0
[<ffffffff815785e5>] tcp_rate_gen+0x105/0x180
[<ffffffff81565fc7>] tcp_ack+0xd97/0x16e0
[<ffffffff8121cff0>] poll_select_copy_remaining+0x150/0x150
[<ffffffff8121cff0>] poll_select_copy_remaining+0x150/0x150
[<ffffffff8121cff0>] poll_select_copy_remaining+0x150/0x150
[<ffffffff8121cff0>] poll_select_copy_remaining+0x150/0x150
[<ffffffff8121cff0>] poll_select_copy_remaining+0x150/0x150
[<ffffffff8121cff0>] poll_select_copy_remaining+0x150/0x150
[<ffffffff8121cff0>] poll_select_copy_remaining+0x150/0x150
[<ffffffff8121cff0>] poll_select_copy_remaining+0x150/0x150
[<ffffffff81209bcd>] __fput+0x17d/0x220
[<ffffffff8102f530>] recalibrate_cpu_khz+0x10/0x10
[<ffffffff810efbf6>] ktime_get_ts64+0x46/0xf0
[<ffffffff8121e707>] SyS_poll+0x67/0x120
[<ffffffff81003bd1>] do_syscall_64+0x91/0x1a0
[<ffffffff816137ce>] entry_SYSCALL_64_after_swapgs+0x58/0xc6
[<ffffffffffffffff>] 0xffffffffffffffff


When it hangs:

root@ldap0:~# cat /proc/13463/stack
[<ffffffff810fa433>] futex_wait_queue_me+0xd3/0x140
[<ffffffff810fb2cc>] futex_wait+0xfc/0x260
[<ffffffff8121cff0>] poll_select_copy_remaining+0x150/0x150
[<ffffffff810fce17>] do_futex+0x2b7/0xb40
[<ffffffff810e81e7>] lock_timer_base+0x67/0x80
[<ffffffff810e839a>] internal_add_timer+0x1a/0x70
[<ffffffff81202d3b>] __check_object_size+0x10b/0x1dc
[<ffffffff814eeece>] move_addr_to_user+0xbe/0xe0
[<ffffffff814ef160>] SYSC_getsockname+0x80/0xd0
[<ffffffff810fd723>] SyS_futex+0x83/0x180
[<ffffffff814efb17>] SyS_setsockopt+0xb7/0xf0
[<ffffffff81003bd1>] do_syscall_64+0x91/0x1a0
[<ffffffff816137ce>] entry_SYSCALL_64_after_swapgs+0x58/0xc6
[<ffffffffffffffff>] 0xffffffffffffffff
Its hard to say for sure if this relates or not to the issue. This looks to be more of OS issue (polling) than a DS issue, but I can't say be sure. Lets try and get some useful stack traces using gdb and go from there.
these stacks don't look like slapd stacks, also for ns-slapd process there should be many threads.
Are you sure you run pstack on the right process ?

Thanks,
Mark

This is reproducable.

Regards
Jan
_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx/message/AH4EBT7XK6T6F6GHOIW4V46IVOAWXTRN/
_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx/message/HPUTRKEP7OUVUEBDEUVTUU2YPKGABJA3/

--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander
_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx/message/USROEUTDCMWYYS5CY32GQO5KLZ2AHQSP/




[Index of Archives]     [Fedora User Discussion]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [Fedora News]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Maintainers]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Legacy]     [Fedora Desktop]     [Fedora Fonts]     [ATA RAID]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Centos]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora QA]     [Fedora Triage]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Tux]     [Yosemite News]     [Yosemite Photos]     [Linux Apps]     [Maemo Users]     [Gnome Users]     [KDE Users]     [Fedora Tools]     [Fedora Art]     [Fedora Docs]     [Maemo Users]     [Asterisk PBX]     [Fedora Sparc]     [Fedora Universal Network Connector]     [Fedora ARM]

  Powered by Linux