On Thu, Jan 24, 2019 at 10:17 PM Mark Reynolds <mreynolds@xxxxxxxxxx> wrote: > > > On 1/24/19 1:54 PM, Mihai Carabas wrote: > > Hello, > > [root@ldap ~]# dsctl slapd-ldap db2ldif --replication cursdb cursdb.ldif > > cursdb.ldif doesn't seem to contain any replication related info. What > should contain related to replication? > > You should at least see an entry with "dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff", if you don't see this entry in the LDIF then it might not be working (sigh). You can then use the legacy tool: > Is there. All is ok. After exporting LDIF and importing again, the search still is buggy. [root@ldap ~]# dsctl slapd-ldap ldif2db cursdb /var/log/dirsrv/slapd-ldap/cursdb.ldif [24/Jan/2019:22:30:01.996505122 +0200] - INFO - ldbm_instance_config_cachememsize_set - force a minimal value 512000 [24/Jan/2019:22:30:02.018018715 +0200] - INFO - ldbm_instance_config_cachememsize_set - force a minimal value 512000 [24/Jan/2019:22:30:02.185170761 +0200] - INFO - dblayer_instance_start - Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database [24/Jan/2019:22:30:02.189742659 +0200] - INFO - check_and_set_import_cache - pagesize: 4096, available bytes 7582248960, process usage 24457216 [24/Jan/2019:22:30:02.191828258 +0200] - INFO - check_and_set_import_cache - Import allocates 2961816KB import cache. [24/Jan/2019:22:30:02.357426205 +0200] - INFO - import_main_offline - import cursdb: Beginning import job... [24/Jan/2019:22:30:02.359609704 +0200] - INFO - import_main_offline - import cursdb: Index buffering enabled with bucket size 100 [24/Jan/2019:22:30:02.560334339 +0200] - INFO - import_producer - import cursdb: Processing file "/var/log/dirsrv/slapd-ldap/cursdb.ldif" [24/Jan/2019:22:30:22.486506161 +0200] - INFO - import_monitor_threads - import cursdb: Processed 72879 entries -- average rate 3643.6/sec, recent rate 3643.6/sec, hit ratio 0% [24/Jan/2019:22:30:24.804083508 +0200] - INFO - import_producer - import cursdb: Finished scanning file "/var/log/dirsrv/slapd-ldap/cursdb.ldif" (81772 entries) [24/Jan/2019:22:30:25.204219378 +0200] - INFO - import_monitor_threads - import cursdb: Workers finished; cleaning up... [24/Jan/2019:22:30:25.407087712 +0200] - INFO - import_monitor_threads - import cursdb: Workers cleaned up. [24/Jan/2019:22:30:25.444768200 +0200] - INFO - import_main_offline - import cursdb: Cleaning up producer thread... [24/Jan/2019:22:30:25.447046899 +0200] - INFO - import_main_offline - import cursdb: Indexing complete. Post-processing... [24/Jan/2019:22:30:25.452563797 +0200] - INFO - import_main_offline - import cursdb: Generating numsubordinates (this may take several minutes to complete)... [24/Jan/2019:22:30:25.498537382 +0200] - INFO - import_main_offline - import cursdb: Generating numSubordinates complete. [24/Jan/2019:22:30:25.500966981 +0200] - INFO - ldbm_get_nonleaf_ids - import cursdb: Gathering ancestorid non-leaf IDs... [24/Jan/2019:22:30:25.503749080 +0200] - INFO - ldbm_get_nonleaf_ids - import cursdb: Finished gathering ancestorid non-leaf IDs. [24/Jan/2019:22:30:25.509673078 +0200] - INFO - ldbm_ancestorid_new_idl_create_index - import cursdb: Creating ancestorid index (new idl)... [24/Jan/2019:22:30:26.432287878 +0200] - INFO - ldbm_ancestorid_new_idl_create_index - import cursdb: Created ancestorid index (new idl). [24/Jan/2019:22:30:26.444136275 +0200] - INFO - import_main_offline - import cursdb: Flushing caches... [24/Jan/2019:22:30:26.447897673 +0200] - INFO - import_main_offline - import cursdb: Closing files... [24/Jan/2019:22:30:37.306490844 +0200] - INFO - dblayer_pre_close - All database threads now stopped [24/Jan/2019:22:30:37.309482243 +0200] - INFO - import_main_offline - import cursdb: Import complete. Processed 81772 entries in 35 seconds. (2336.34 entries/sec) ldif2db successful Here is the buggy search: # extended LDIF # # LDAPv3 # base <ou=DPPD,ou=People,dc=curs,dc=pub,dc=ro> with scope oneLevel # filter: ou=Profesori # requesting: ALL # # Profesori, DPPD, People, curs.pub.ro dn: ou=Profesori,ou=DPPD,ou=People,dc=curs,dc=pub,dc=ro ou: Profesori objectClass: top objectClass: organizationalunit # Asistenti-Man, DPPD, People, curs.pub.ro dn: ou=Asistenti-Man,ou=DPPD,ou=People,dc=curs,dc=pub,dc=ro objectClass: top objectClass: organizationalunit ou: ou=Asistenti-Man ou: Asistenti-Man # Profesori-Man, DPPD, People, curs.pub.ro dn: ou=Profesori-Man,ou=DPPD,ou=People,dc=curs,dc=pub,dc=ro objectClass: top objectClass: organizationalunit ou: ou=Profesori-Man ou: Profesori-Man # Externi, DPPD, People, curs.pub.ro dn: ou=Externi,ou=DPPD,ou=People,dc=curs,dc=pub,dc=ro objectClass: top objectClass: organizationalunit ou: ou=Externi ou: Externi # Auxiliari, DPPD, People, curs.pub.ro dn: ou=Auxiliari,ou=DPPD,ou=People,dc=curs,dc=pub,dc=ro objectClass: top objectClass: organizationalunit ou: Auxiliari # search result search: 2 result: 0 Success # numResponses: 6 # numEntries: 5 > db2ldif -r -n userroot -a LDIF_FILE > > Regards, > > Mark > > Thank you, > Mihai > > On Thu, Jan 24, 2019 at 5:53 PM Mark Reynolds <mreynolds@xxxxxxxxxx> wrote: > > On 1/24/19 10:26 AM, Mihai Carabas wrote: > > Also I used db2index on the stopped Directory Server. Should I have > used db2index.pl on the running one? > > IMO, it's best to stop to the server when doing these types of operations. > > Mark > > Thanks, > Mihai > > On Thu, Jan 24, 2019 at 5:24 PM Mihai Carabas <mihai.carabas@xxxxxxxxx> wrote: > > Hello Mark, > > On Thu, Jan 24, 2019 at 4:35 PM Mark Reynolds <mreynolds@xxxxxxxxxx> wrote: > > On 1/24/19 8:37 AM, Mihai Carabas wrote: > > On Thu, Jan 24, 2019 at 12:52 AM William Brown <wbrown@xxxxxxx> wrote: > > On 24 Jan 2019, at 00:50, Mihai Carabas <mihai.carabas@xxxxxxxxx> wrote: > > =profesor > 42815 42818 42821 42824 42827 42830 42833 42836 42839 42842 > 42845 42848 43445 43450 45720 52253 > =profesori > 7 68 3232 3648 5457 6259 6268 6270 6275 6278 6280 6282 6304 > 6308 6323 7098 21827 > > Can you also send me the output of : > > ldapsearch -b …. -s sub ‘(ou=Profesori)’ entryid > > # extended LDIF > # > # LDAPv3 > # base <ou=DPPD,ou=People,dc=curs,dc=pub,dc=ro> with scope subtree > # filter: ou=Profesori > # requesting: entryid > # > > # Profesori, DPPD, People, curs.pub.ro > dn: ou=Profesori,ou=DPPD,ou=People,dc=curs,dc=pub,dc=ro > entryid: 3648 > > # search result > search: 2 > result: 0 Success > > I issued db2index and I have the following output: > the 50% threshold. > [24/Jan/2019:15:14:31.578343786 +0200] - INFO - import_monitor_threads > - reindex cursdb: Ending pass number 1 ... > [24/Jan/2019:15:14:31.781708885 +0200] - INFO - import_monitor_threads > - reindex cursdb: Foreman is done; waiting for workers to finish... > [24/Jan/2019:15:14:32.055444491 +0200] - INFO - import_monitor_threads > - reindex cursdb: Workers finished; cleaning up... > [24/Jan/2019:15:14:32.259020991 +0200] - INFO - import_monitor_threads > - reindex cursdb: Workers cleaned up. > [24/Jan/2019:15:14:32.262209501 +0200] - INFO - > import_sweep_after_pass - reindex cursdb: Sweeping files for merging > later... > [24/Jan/2019:15:14:33.674826562 +0200] - INFO - dblayer_instance_start > - Import is running with nsslapd-db-private-import-mem on; No other > process is allowed to access the database > [24/Jan/2019:15:14:33.975430948 +0200] - INFO - > import_sweep_after_pass - reindex cursdb: Sweep done. > [24/Jan/2019:15:14:33.982718069 +0200] - INFO - import_main_offline - > reindex cursdb: Beginning pass number 2 > [24/Jan/2019:15:14:54.035041942 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 204521866.1/sec, recent rate 0.0/sec, hit ratio 0% > > [24/Jan/2019:15:15:14.086428912 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 104755102.1/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:15:34.138688984 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 70409167.0/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:15:54.189761953 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 53024187.5/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:16:14.239427918 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 42524348.4/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:16:34.483660956 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 35495530.5/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:16:54.533827623 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 30460703.5/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:17:14.586325096 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 26676765.1/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:17:34.636106261 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 23729056.3/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:17:54.686551928 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 21367956.2/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:18:14.812847819 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 19434204.5/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:18:34.883738846 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 17821407.4/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:18:54.938178725 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 16455782.3/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:19:14.995408557 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 15284552.3/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:19:35.047817446 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 2) -- average rate > 14221719.2/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:19:35.051470957 +0200] - INFO - import_throw_in_towel > - reindex cursdb: Decided to end this pass because the progress rate > has dropped below the 50% threshold. > [24/Jan/2019:15:19:35.054926468 +0200] - INFO - import_monitor_threads > - reindex cursdb: Ending pass number 2 ... > [24/Jan/2019:15:19:35.258101594 +0200] - INFO - import_monitor_threads > - reindex cursdb: Foreman is done; waiting for workers to finish... > [24/Jan/2019:15:19:35.261243303 +0200] - INFO - import_monitor_threads > - reindex cursdb: Workers finished; cleaning up... > [24/Jan/2019:15:19:35.464851131 +0200] - INFO - import_monitor_threads > - reindex cursdb: Workers cleaned up. > [24/Jan/2019:15:19:35.467968441 +0200] - INFO - > import_sweep_after_pass - reindex cursdb: Sweeping files for merging > later... > [24/Jan/2019:15:19:35.506390559 +0200] - INFO - dblayer_instance_start > - Import is running with nsslapd-db-private-import-mem on; No other > process is allowed to access the database > [24/Jan/2019:15:19:35.731292752 +0200] - INFO - > import_sweep_after_pass - reindex cursdb: Sweep done. > [24/Jan/2019:15:19:35.734380561 +0200] - INFO - import_main_offline - > reindex cursdb: Beginning pass number 3 > [24/Jan/2019:15:19:55.782461337 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 214747959.4/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:20:15.834380724 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 107373979.7/sec, recent rate 0.0/sec, hit ratio 0% > > > [24/Jan/2019:15:20:35.886774313 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 71582653.1/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:20:55.938228899 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 53686989.9/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:21:15.988693682 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 42949591.9/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:21:40.661184607 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 34359673.5/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:22:00.820373724 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 29620408.2/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:22:20.872956514 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 26030055.7/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:22:40.928481812 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 23215995.6/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:23:00.983057008 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 20951020.4/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:23:21.398151214 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 19004244.2/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:23:41.466437252 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 17459183.7/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:24:01.517539137 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 16146463.1/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:24:21.568626321 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 15017339.8/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:24:41.726888036 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 3) -- average rate > 14035814.3/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:24:41.731112549 +0200] - INFO - import_throw_in_towel > - reindex cursdb: Decided to end this pass because the progress rate > has dropped below the 50% threshold. > [24/Jan/2019:15:24:41.734400859 +0200] - INFO - import_monitor_threads > - reindex cursdb: Ending pass number 3 ... > [24/Jan/2019:15:24:41.837514477 +0200] - INFO - import_monitor_threads > - reindex cursdb: Foreman is done; waiting for workers to finish... > [24/Jan/2019:15:24:41.841081788 +0200] - INFO - import_monitor_threads > - reindex cursdb: Workers finished; cleaning up... > [24/Jan/2019:15:24:41.944805808 +0200] - INFO - import_monitor_threads > - reindex cursdb: Workers cleaned up. > [24/Jan/2019:15:24:41.948172218 +0200] - INFO - > import_sweep_after_pass - reindex cursdb: Sweeping files for merging > later... > [24/Jan/2019:15:24:41.972718294 +0200] - INFO - dblayer_instance_start > - Import is running with nsslapd-db-private-import-mem on; No other > process is allowed to access the database > [24/Jan/2019:15:24:42.277287032 +0200] - INFO - > import_sweep_after_pass - reindex cursdb: Sweep done. > [24/Jan/2019:15:24:42.524714295 +0200] - INFO - import_main_offline - > reindex cursdb: Beginning pass number 4 > [24/Jan/2019:15:25:02.589790122 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 4) -- average rate > 214747959.4/sec, recent rate 0.0/sec, hit ratio 0% > [24/Jan/2019:15:25:22.648607831 +0200] - INFO - import_monitor_threads > - reindex cursdb: Processed 8108 entries (pass 4) -- average rate > 107373979.7/sec, recent rate 0.0/sec, hit ratio 0% > > And it takes a long time. What should I do? > > This is very odd that the index task is not proceeding. Are there more > than 8108 entries in the database? Or is that the size of the db in > entries? > > I have more de 70000 users. So I do not think that is the size of the db. > > If the task is still running can you grab a stack traces using this process: > > http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs > > Grab a few stack traces with that gdb command, and send them to us please. > > Unfortunatelly I've closed the db2index command and I've restored the > Directory Server from a previous backup. I couldn' afford to wait much > longer. > > Then to fix it I think we need to do an export & import of the > database. That will regenerate everything. If you are using > replication then use the "--replication" option with db2ldif, otherwise > do not use that option: > > dsctl <INSTANCE> stop ---> You might have to kill the process > if the reindex is hanging :-( > > dsctl <INSTANCE> db2ldif userroot <LDIF file name> --replication > > Then import that LDIF > > dsctl <INSTANCE> ldif2db userroot <LDIF file name> > > dsctl <INSTANCE> start > > > Then retry your search tests. > > This should be done on my restored running instace? > > I have a replication with Active Directory. This will be affected? > > Regards, > > Mark > > > This is not contingent on db2index, so you can run the db2index and see if that corrects the issue now (if you have not already). > > Thanks > > — > Sincerely, > > William Brown > Software Engineer, 389 Directory Server > SUSE Labs > _______________________________________________ > 389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx > > _______________________________________________ > 389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx > > _______________________________________________ > 389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx > > _______________________________________________ > 389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx _______________________________________________ 389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx
