Re: Unindexed search even on indexed database

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 3/4/21 7:24 AM, Pierre Rogier wrote:
Hi Jan, 

IMHO the allids threshold has been reached for the "issued" value for entryStatus  index
 (in other words: too many entries have  entryStatus: issued: )
You may want to grep for "conn=115 op=1" in access log and check how many entries where returned by the search
If the value is higher than allids threshold then it is normal  

Note: you may increase the allids threshold and reindex

Actually in 389 here is no need to reindex after increasing the nsslapd-idlistscanlimit.  In 389, vs DSEE, we actually index everything then we use the idlistscanlimit to limit how far we go into the index.  So you can adjust the limit while the server is running and no further actions are required.   Just an FYI.

So yes in this case increasing this value should solve the problem, you just need to set it high enough like Pierre mentioned.

Mark

   but that is not always the best thing to do. 
(It depends of the ratio of entries having entryStatus: issued versus the total number of entries in the backend:
  if most of the entries are in issued state it is better to keep the search unindexed:
  there is no real interest to manage huge index when we have to return
   most of the entries)

Regards
   Pierre 


On Thu, Mar 4, 2021 at 10:18 AM Jan Tomasek <jan@xxxxxxxxxx> wrote:
Hello,

I'm worrying about log lines:

[04/Mar/2021:10:08:47.982170561 +0100] - NOTICE - ldbm_back_search -
Unindexed search: search base="o=tcs2,o=apps,dc=cesnet,dc=cz" scope=2
filter="(entryStatus=issued)" conn=115 op=1

Index is defined:

# dsconf -D "cn=Directory Manager" -w "$pswd" ldap://localhost backend
index get TCS2_apps_cesnet_cz --attr entryStatus
dn: cn=entryStatus,cn=index,cn=TCS2_apps_cesnet_cz,cn=ldbm
database,cn=plugins,cn=config
cn: entryStatus
nsIndexType: eq
nsIndexType: pres
nsSystemIndex: False
objectClass: top
objectClass: nsIndex

Database is freshly reindexed:

# dsconf -D "cn=Directory Manager" -w "$pswd" ldap://localhost backend
index reindex TCS2_apps_cesnet_cz --attr entryStatus
Index task index_attrs_03042021_100813 completed successfully
Successfully reindexed database
# tail /var/log/dirsrv/slapd-cml3/errors
[04/Mar/2021:10:08:19.181006893 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 43000 entries (83%).
[04/Mar/2021:10:08:19.304566154 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 44000 entries (85%).
[04/Mar/2021:10:08:19.430861272 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 45000 entries (86%).
[04/Mar/2021:10:08:19.554529568 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 46000 entries (88%).
[04/Mar/2021:10:08:19.671814136 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 47000 entries (90%).
[04/Mar/2021:10:08:19.791473662 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 48000 entries (92%).
[04/Mar/2021:10:08:19.911157930 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 49000 entries (94%).
[04/Mar/2021:10:08:20.032595700 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 50000 entries (96%).
[04/Mar/2021:10:08:20.153813121 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 51000 entries (98%).
[04/Mar/2021:10:08:20.244942556 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Finished indexing.

But server is still complaining:

# time ldapsearch -H ldap://localhost -x -b
o=TCS2,o=apps,dc=cesnet,dc=cz '(entryStatus=issued)'
# extended LDIF
#
# LDAPv3
# base <o=TCS2,o=apps,dc=cesnet,dc=cz> with scope subtree
# filter: (entryStatus=issued)
# requesting: ALL
#

# search result
search: 2
result: 0 Success

# numResponses: 1

real    0m0.920s
user    0m0.014s
sys     0m0.001s
# tail /var/log/dirsrv/slapd-cml3/errors
...
[04/Mar/2021:10:08:20.153813121 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Indexed 51000 entries (98%).
[04/Mar/2021:10:08:20.244942556 +0100] - INFO - bdb_db2index -
TCS2_apps_cesnet_cz: Finished indexing.
[04/Mar/2021:10:08:47.982170561 +0100] - NOTICE - ldbm_back_search -
Unindexed search: search base="o=tcs2,o=apps,dc=cesnet,dc=cz" scope=2
filter="(entryStatus=issued)" conn=115 op=1

Some DB were created during reindex:

# ls -l /var/lib/dirsrv/slapd-cml3/db/TCS2_apps_cesnet_cz/
total 358212
-rw------- 1 dirsrv dirsrv     16384 Feb 22 10:08 aci.db
-rw------- 1 dirsrv dirsrv   2760704 Feb 22 10:09 ancestorid.db
-rw------- 1 dirsrv dirsrv  19668992 Mar  3 16:53 cn.db
-rw------- 1 dirsrv dirsrv        51 Mar  4 09:54 DBVERSION
-rw------- 1 dirsrv dirsrv     24576 Mar  3 16:53 dc.db
-rw------- 1 dirsrv dirsrv  13254656 Feb 22 10:09 entryrdn.db
-rw------- 1 dirsrv dirsrv   1114112 Mar  4 10:08 entryStatus.db
-rw------- 1 dirsrv dirsrv     16384 Feb 22 10:07 entryusn.db
-rw------- 1 dirsrv dirsrv   3063808 Mar  3 16:54 givenName.db
-rw------- 1 dirsrv dirsrv 285548544 Mar  4 10:09 id2entry.db
-rw------- 1 dirsrv dirsrv  16056320 Mar  3 16:54 mail.db
-rw------- 1 dirsrv dirsrv     16384 Feb 22 10:08 nscpEntryDN.db
-rw------- 1 dirsrv dirsrv   3891200 Feb 22 10:09 nsuniqueid.db
-rw------- 1 dirsrv dirsrv     24576 Feb 22 10:09 numsubordinates.db
-rw------- 1 dirsrv dirsrv   1466368 Feb 22 10:09 objectclass.db
-rw------- 1 dirsrv dirsrv    811008 Feb 22 10:09 parentid.db
-rw------- 1 dirsrv dirsrv    258048 Mar  4 10:00 replication_changelog.db
-rw------- 1 dirsrv dirsrv   3735552 Mar  3 16:55 sn.db
-rw------- 1 dirsrv dirsrv    335872 Mar  4 09:26 tcs2certificate.db
-rw------- 1 dirsrv dirsrv     32768 Mar  3 16:55 tcs2cesnetorgdn.db
-rw------- 1 dirsrv dirsrv   2826240 Mar  3 16:55 tcs2crtserialnumber.db
-rw------- 1 dirsrv dirsrv   3809280 Mar  3 16:55 tcs2crtsubject.db
-rw------- 1 dirsrv dirsrv     16384 Mar  3 16:55 tcs2idpentityid.db
-rw------- 1 dirsrv dirsrv   3276800 Mar  3 16:56 tcs2requesterdn.db
-rw------- 1 dirsrv dirsrv    393216 Mar  3 16:56 tcs2role.db
-rw------- 1 dirsrv dirsrv   3219456 Mar  3 16:56 telephoneNumber.db
-rw------- 1 dirsrv dirsrv    516096 Mar  3 16:56 uid.db
-rw------- 1 dirsrv dirsrv    647168 Mar  3 16:57 unstructuredname.db

Any ideas how to fix the problem?
--
-----------------------
Jan Tomasek aka Semik
http://www.tomasek.cz/
_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure


--
--

389 Directory Server Development Team

_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- 

389 Directory Server Development Team
_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Index of Archives]     [Fedora User Discussion]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [Fedora News]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Maintainers]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Legacy]     [Fedora Desktop]     [Fedora Fonts]     [ATA RAID]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Centos]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora QA]     [Fedora Triage]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Tux]     [Yosemite News]     [Yosemite Photos]     [Linux Apps]     [Maemo Users]     [Gnome Users]     [KDE Users]     [Fedora Tools]     [Fedora Art]     [Fedora Docs]     [Maemo Users]     [Asterisk PBX]     [Fedora Sparc]     [Fedora Universal Network Connector]     [Fedora ARM]

  Powered by Linux