Re: 389 crash libdb: PANIC: fatal region error detected

[Ludwig Krispenz <lkrispen@xxxxxxxxxx>]

If the database panics, it has detected a state of inconsistency it cannot resolve and sets a panic flag to the database environment to prevents its use. Just restarting the ds will detect the flag and abort.
You can try to remove the __db.00* files and then start DS again, if this fails you need to rebuild the database either by a bak backup or ldif import.

Why it did panic, I think we can no longer find out.

Ludwig

On 08/02/2013 09:30 AM, Manel Gimeno Zaragozá wrote:

Hello,

Yesterday afternoon my LDAP server crashed without doing any modification, just consulting.

My environment is an openvz container:

# cat /etc/issue
CentOS release 6.4 (Final)
Kernel \r on an \m

# uname -a
Linux newldap.test.es 2.6.32-042stab053.5 #1 SMP Tue Mar 27 11:42:17 MSD 2012 x86_64 x86_64 x86_64 GNU/Linux

# rpm -qa | grep 389
389-console-1.1.7-1.el6.noarch
389-admin-1.1.29-1.el6.x86_64
389-admin-console-1.1.8-1.el6.noarch
389-ds-base-1.2.11.15-14.el6_4.x86_64
389-ds-console-1.2.6-1.el6.noarch
389-ds-base-libs-1.2.11.15-14.el6_4.x86_64
389-ds-base-devel-1.2.11.15-14.el6_4.x86_64
389-adminutil-1.1.15-1.el6.x86_64


Please find below the log when it crashes:

errors:10:[01/Aug/2013:14:09:44 +0200] configure_pr_socket - Unable to move socket file descriptor 62 above 64: OS error 24 (Too many open files)
errors:11:[01/Aug/2013:14:09:44 +0200] configure_pr_socket - Unable to move socket file descriptor 63 above 64: OS error 24 (Too many open files)
errors:12:[01/Aug/2013:14:09:44 +0200] - PR_Accept() failed, Netscape Portable Runtime error -5971 (Process open FD table is full.)
errors:13:[01/Aug/2013:14:09:44 +0200] - PR_Accept() failed, Netscape Portable Runtime error -5971 (Process open FD table is full.)
errors:14:[01/Aug/2013:14:09:44 +0200] - PR_Accept() failed, Netscape Portable Runtime error -5971 (Process open FD table is full.)
...
[01/Aug/2013:14:10:01 +0200] - PR_Accept() failed, Netscape Portable Runtime error -5971 (Process open FD table is full.)
[01/Aug/2013:14:10:01 +0200] - PR_Accept() failed, Netscape Portable Runtime error -5971 (Process open FD table is full.)
[01/Aug/2013:14:10:01 +0200] - libdb: PANIC: fatal region error detected; run recovery
[01/Aug/2013:14:10:01 +0200] - Serious Error---Failed in deadlock detect (aborted at 0x0), err=-30974 (DB_RUNRECOVERY: Fatal error, run database recovery)
[01/Aug/2013:14:10:01 +0200] - PR_Accept() failed, Netscape Portable Runtime error -5971 (Process open FD table is full.)
[01/Aug/2013:14:10:01 +0200] - PR_Accept() failed, Netscape Portable Runtime error -5971 (Process open FD table is full.)
...
[01/Aug/2013:14:12:27 +0200] - libdb: PANIC: fatal region error detected; run recovery
[01/Aug/2013:14:12:27 +0200] - Serious Error---Failed in deadlock detect (aborted at 0x0), err=-30974 (DB_RUNRECOVERY: Fatal error, run database recovery)
[01/Aug/2013:14:12:28 +0200] - libdb: PANIC: fatal region error detected; run recovery
[01/Aug/2013:14:12:28 +0200] - Serious Error---Failed in deadlock detect (aborted at 0x0), err=-30974 (DB_RUNRECOVERY: Fatal error, run database recovery)
[01/Aug/2013:14:12:28 +0200] - libdb: PANIC: fatal region error detected; run recovery
[01/Aug/2013:14:12:28 +0200] - Serious Error---Failed to checkpoint database, err=-30974 (DB_RUNRECOVERY: Fatal error, run database recovery)

when I noticed that the ldap had crashed, I tried to restart the process and the following errors show up:

[01/Aug/2013:14:20:15 +0200] - 389-Directory/1.2.11.15 B2013.105.2259 starting up
[01/Aug/2013:14:20:15 +0200] - WARNING: userRoot: entry cache size 10485760B is less than db size 14729216B; We recommend to increase the entry cache size nsslapd-cachememsize.
[01/Aug/2013:14:20:15 +0200] - libdb: PANIC: fatal region error detected; run recovery
[01/Aug/2013:14:20:15 +0200] - Opening database environment (/var/lib/dirsrv/slapd-ldap_kolab/db) failed. err=-30974: DB_RUNRECOVERY: Fatal error, run database recovery
[01/Aug/2013:14:20:15 +0200] - start: Failed to init database, err=-30974 DB_RUNRECOVERY: Fatal error, run database recovery
[01/Aug/2013:14:20:15 +0200] - Failed to start database plugin ldbm database
[01/Aug/2013:14:20:15 +0200] - WARNING: ldbm instance userRoot already exists
[01/Aug/2013:14:20:15 +0200] - ldbm_config_read_instance_entries: failed to add instance entry cn=userRoot,cn=ldbm database,cn=plugins,cn=config
[01/Aug/2013:14:20:15 +0200] - ldbm_config_load_dse_info: failed to read instance entries
[01/Aug/2013:14:20:15 +0200] - start: Loading database configuration failed
[01/Aug/2013:14:20:15 +0200] - Failed to start database plugin ldbm database
[01/Aug/2013:14:20:15 +0200] - Error: Failed to resolve plugin dependencies
[01/Aug/2013:14:20:15 +0200] - Error: preoperation plugin 7-bit check is not started
[01/Aug/2013:14:20:15 +0200] - Error: preoperation plugin Account Usability Plugin is not started
[01/Aug/2013:14:20:15 +0200] - Error: accesscontrol plugin ACL Plugin is not started
[01/Aug/2013:14:20:15 +0200] - Error: preoperation plugin ACL preoperation is not started
[01/Aug/2013:14:20:15 +0200] - Error: preoperation plugin Auto Membership Plugin is not started
[01/Aug/2013:14:20:15 +0200] - Error: object plugin Class of Service is not started
[01/Aug/2013:14:20:15 +0200] - Error: preoperation plugin deref is not started
[01/Aug/2013:14:20:15 +0200] - Error: preoperation plugin HTTP Client is not started
[01/Aug/2013:14:20:15 +0200] - Error: database plugin ldbm database is not started
[01/Aug/2013:14:20:15 +0200] - Error: object plugin Legacy Replication Plugin is not started
[01/Aug/2013:14:20:15 +0200] - Error: preoperation plugin Linked Attributes is not started
[01/Aug/2013:14:20:15 +0200] - Error: preoperation plugin Managed Entries is not started
[01/Aug/2013:14:20:15 +0200] - Error: object plugin Multimaster Replication Plugin is not started
[01/Aug/2013:14:20:15 +0200] - Error: object plugin Roles Plugin is not started
[01/Aug/2013:14:20:15 +0200] - Error: object plugin Views is not started

Can any one help me?

Thanks for your help

Manel

--
389 users mailing list
389-users@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-users

--
389 users mailing list
389-users@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-users