Re: How to tell when database backup has finished?

["MATON Brett" <Brett.Maton@xxxxxx>]

Thanks for the suggestions.

 

  For the record, status 32 was an error on my part, I hadn’t quoted the bind DN so ldapsearch was actually complaining about the bind DN not being found, not the backup task.

 

  Noriko, I’ve searched “cn=backup,cn=tasks,cn=config” and can’t find nstaskstatus anywhere.  I’m assuming that it also gets deleted once the task is complete...

cn=backup,cn=tasks,cn=config is the parent entry for all backup task entries.  When you create an entry under this parent, it starts the backup task.   It is the backup task entry that has the nsTaskStatus attribute, not the parent entry "cn=backup,cn=tasks,cn=config".  When the backup task is complete, and the ttl is expired, the backup task entry is automatically removed.

Hi Rich,  I appreciate your clarification.  I did however mean that I was searching “cn=backup,cn=tasks,cn=config” with no additional filters which should have returned everything, nsTaskStatus included for any backup job?

Exactly what search parameters (e.g. ldapsearch command line arguments) did you use?

 

  I might just be having a dumb moment but I couldn’t think of an efficient way to check the error log for a Backup Finished entry, anyone with suggestions?

The best way is to do the ldapsearch as described above.

  I’m still having issues, $taskDN for example would be something like cn=backup_2012_4_5_13_36_23, cn=backup, cn=tasks, cn=config

COUNT=1

while [ ${COUNT} -ne 0 ]; do

#

#  OOPS:  -b ${taskDN}

#         -b "cn=backup,cn=tasks,cn=config" "cn=<task>"

#

        COUNT=$(ldapsearch -x -H ${HOST} -D "${BINDAS}" -y ${PWFILE} -b "cn=backup,cn=tasks,cn=config" "(${taskDN})" dn | grep -c '^dn')

so this would expand to something like
COUNT=$(ldapsearch -x -H ldap.example.com -D "cn=directory manager" -y pwfile.txt -b "cn=backup,cn=tasks,cn=config" "(cn=backup_2012_4_5_13_36_23, cn=backup, cn=tasks, cn=config)" dn | grep -c '^dn')

This is incorrect in the following ways:
1) the argument to -H must be a valid LDAP URL - so -H ldap://ldap.example.com   $HOST is just a variable name, it’s actually the URL
2) "(cn=backup_2012_4_5_13_36_23, cn=backup, cn=tasks, cn=config)" is not a valid LDAP search filter - if you want to search for the backup entry, do one of the following
a) ldapsearch ... -s base -b "cn=backup_2012_4_5_13_36_23, cn=backup, cn=tasks, cn=config" ...
or
b) ldapsearch ... -b "cn=backup, cn=tasks, cn=config" "(cn=backup_2012_4_5_13_36_23)" ...

 

OK, search terms corrected.

 

The command expands to and returns this:

ldapsearch -x -H ldaps://<host fqdn>:636 -D "cn=Directory Manager" -y /root/.389ds-pw -s base -b "cn=backup_2012_4_11_7_28_40, cn=backup, cn=tasks, cn=config"

# extended LDIF

#

# LDAPv3

# base <cn=backup_2012_4_11_7_28_40, cn=backup, cn=tasks, cn=config> with scope baseObject

# filter: (objectclass=*)

# requesting: ALL

#

 

# search result

search: 2

result: 32 No such object

 

# numResponses: 1

I tweaked the db2bak.pl script to include ttl: 600 as I wanted some time to examine the backup task entry and search results:

...

$dn = "dn: cn=$taskname, cn=backup, cn=tasks, cn=config\n";

$misc = "changetype: add\nobjectclass: top\nobjectclass: extensibleObject\nttl: 600\n";

$cn = "cn: $taskname\n";
...

The entry gets added, but still disappears near instantaneously (db2bak.pl called standalone for output):

 

Back up directory: /var/lib/dirsrv/slapd-<host>/bak/<host>-2012_4_11_7_42_4

ldap_initialize( <DEFAULT> )

add objectclass:

        top

        extensibleObject

add ttl:

        600

add cn:

        backup_2012_4_11_7_42_4

add nsArchiveDir:

        /var/lib/dirsrv/slapd-<host>/bak/<host>-2012_4_11_7_42_4

add nsDatabaseType:

        ldbm database

adding new entry "cn=backup_2012_4_11_7_42_4, cn=backup, cn=tasks, cn=config"

modify complete

 

As you can see from the ls output below, tar is still being called before the task has completed:

 

[root@<host> backup]# ls -ltr

-rw-r--r--. 1 root   root     6788 Apr 11 07:28 20120411072840-<host>.tgz

drwx------. 5 nobody nobody   4096 Apr 11 07:28 20120411072840-<host>

 

Thanks for your time....

ok - I'd like you to try two more things:
1) alter your search parameters - do this instead:
ldapsearch -x -H ldaps://<host fqdn>:636 -D "cn=Directory Manager" -y /root/.389ds-pw -b "cn=backup, cn=tasks, cn=config"
do you see anything?

 

Ok this is a bit freaky:

 

Executing db2pk.pl manually from the command line I get this in /var/log/dirsrv/<instance>/error:

 

[11/Apr/2012:15:33:11 +0200] - Beginning backup of 'ldbm database'

[11/Apr/2012:15:33:12 +0200] - Backing up file 35 (/var/lib/dirsrv/slapd-<instance>/bak/<instance>-2012_4_11_15_33_12/.repl_changelog_backup/changelogdb/6b85db82-5df911e1-ab138798-28541438_4f45faa60000004b0000.db4)

[11/Apr/2012:15:33:12 +0200] - Backing up file 36 (/var/lib/dirsrv/slapd-<instance>/bak/<instance>-2012_4_11_15_33_12/.repl_changelog_backup/changelogdb/DBVERSION)

[11/Apr/2012:15:33:12 +0200] - Backup finished.

 

ldapsearch -x -H ldaps://<instance>.nrb.be:636 -D "cn=Directory Manager" -y /root/.389ds-pw -b "cn=backup_2012_4_11_15_33_12,cn=backup,cn=tasks,cn=config"                 # extended LDIF

#

# LDAPv3

# base <cn=backup_2012_4_11_15_33_12,cn=backup,cn=tasks,cn=config> with scope subtree

# filter: (objectclass=*)

# requesting: ALL

#

 

# backup_2012_4_11_15_33_12, backup, tasks, config

dn: cn=backup_2012_4_11_15_33_12,cn=backup,cn=tasks,cn=config

objectClass: top

objectClass: extensibleObject

ttl: 600

cn: backup_2012_4_11_15_33_12

nsarchivedir: /var/lib/dirsrv/slapd-<instance>/bak/<instance>-2012_4_11_15_33_12

nsdatabasetype: ldbm database

nstaskcurrentitem: 0

nstasktotalitems: 1

nstasklog::   <snipped>

nstaskstatus: Backup finished.

nstaskexitcode: 0

 

# search result

search: 2

result: 0 Success

 

# numResponses: 2

# numEntries: 1

 

 

Scripted version:

/var/log/dirsrv/<instance>/error:

 

[11/Apr/2012:15:35:06 +0200] - Beginning backup of 'ldbm database'

[11/Apr/2012:15:35:07 +0200] - Backing up file 1 (/usr/backup/20120411153507-nrbnrl0075/userRoot/telephoneNumber.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 2 (/usr/backup/20120411153507-nrbnrl0075/userRoot/ntUserDomainId.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 3 (/usr/backup/20120411153507-nrbnrl0075/userRoot/numsubordinates.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 4 (/usr/backup/20120411153507-nrbnrl0075/userRoot/objectclass.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 5 (/usr/backup/20120411153507-nrbnrl0075/userRoot/mail.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 6 (/usr/backup/20120411153507-nrbnrl0075/userRoot/ntUniqueId.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 7 (/usr/backup/20120411153507-nrbnrl0075/userRoot/parentid.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 8 (/usr/backup/20120411153507-nrbnrl0075/userRoot/aci.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 9 (/usr/backup/20120411153507-nrbnrl0075/userRoot/cn.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 10 (/usr/backup/20120411153507-nrbnrl0075/userRoot/seeAlso.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 11 (/usr/backup/20120411153507-nrbnrl0075/userRoot/entryrdn.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 12 (/usr/backup/20120411153507-nrbnrl0075/userRoot/uid.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 13 (/usr/backup/20120411153507-nrbnrl0075/userRoot/ancestorid.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 14 (/usr/backup/20120411153507-nrbnrl0075/userRoot/givenName.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 15 (/usr/backup/20120411153507-nrbnrl0075/userRoot/sn.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 16 (/usr/backup/20120411153507-nrbnrl0075/userRoot/uniquemember.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 17 (/usr/backup/20120411153507-nrbnrl0075/userRoot/nsuniqueid.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 18 (/usr/backup/20120411153507-nrbnrl0075/userRoot/nscpEntryDN.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 19 (/usr/backup/20120411153507-nrbnrl0075/userRoot/id2entry.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 20 (/usr/backup/20120411153507-nrbnrl0075/userRoot/DBVERSION)

[11/Apr/2012:15:35:07 +0200] - Backing up file 21 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/numsubordinates.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 22 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/objectclass.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 23 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/parentid.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 24 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/aci.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 25 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/cn.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 26 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/entryrdn.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 27 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/uid.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 28 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/ancestorid.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 29 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/givenName.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 30 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/sn.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 31 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/uniquemember.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 32 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/nsuniqueid.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 33 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/id2entry.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 34 (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/DBVERSION)

[11/Apr/2012:15:35:07 +0200] - Backing up file 35 (/usr/backup/20120411153507-nrbnrl0075/.repl_changelog_backup/changelogdb/6b85db82-5df911e1-ab138798-28541438_4f45faa60000004b0000.db4)

[11/Apr/2012:15:35:07 +0200] - Backing up file 36 (/usr/backup/20120411153507-nrbnrl0075/.repl_changelog_backup/changelogdb/DBVERSION)

[11/Apr/2012:15:35:07 +0200] - Backing up file 37 (/usr/backup/20120411153507-nrbnrl0075/log.0000000018)

[11/Apr/2012:15:35:07 +0200] - Backing up file 38 (/usr/backup/20120411153507-nrbnrl0075/DBVERSION)

[11/Apr/2012:15:35:07 +0200] - Backup finished.

 

Search result:

 

ldapsearch -x -H ldaps://nrbnrl0076.nrb.be:636 -D "cn=Directory Manager" -y /root/.389ds-pw -s base -b "cn=backup_2012_4_11_15_35_7, cn=backup, cn=tasks, cn=config"

# extended LDIF

#

# LDAPv3

# base <cn=backup_2012_4_11_15_35_7, cn=backup, cn=tasks, cn=config> with scope baseObject

# filter: (objectclass=*)

# requesting: ALL

#

 

# search result

search: 2

result: 32 No such object

 

# numResponses: 1

 

Search without taskid:

 

ldapsearch -x -H ldaps://nrbnrl0076.nrb.be:636 -D "cn=Directory Manager" -y /root/.389ds-pw -s base -b "cn=backup,cn=tasks,cn=config"

# extended LDIF

#

# LDAPv3

# base <cn=backup,cn=tasks,cn=config> with scope baseObject

# filter: (objectclass=*)

# requesting: ALL

#

 

# backup, tasks, config

dn: cn=backup,cn=tasks,cn=config

objectClass: top

objectClass: extensibleObject

cn: backup

 

# search result

search: 2

result: 0 Success

 

# numResponses: 2

# numEntries: 1

 

The only difference I can see is that the scripted version specifies a backup path with the –a option

 

2) can you confirm using the errors log (manually, not automated with a script) that your backup task was started successfully and completes successfully?  Just to be sure that it is indeed being started and finishing correctly

Error logs above.

 

        if [ ${COUNT} -ne 0 ]; then

                sleep 5s

        fi

done

tar czf ${BACKUPPATH}${BKTSTAMP}-${INSTANCE}.tgz -C / ${DSBACKUPDIR#/*}

 

However the tar file is incomplete, because (assumption) the backup has not finished writing to disk before tar starts....

 

 

  For the time being I’ve settled on two scripts, one to start backups and another to archive yesterdays (or older) backups.  Not ideal but workable

 

Thanks again,

Brett

 

From: 389-users-bounces@xxxxxxxxxxxxxxxxxxxxxxx [mailto:389-users-bounces@xxxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Noriko Hosoi
Sent: 05 April 2012 18:40
To: General discussion list for the 389 Directory server project.
Subject: Re: [389-users] How to tell when database backup has finished?

 

You could search the task.with nstaskstatus attribute.

$ ldapsearch -LLLx -h localhost -p <port> -D 'cn=directory manager' -w <pw> -b "cn=backup,cn=tasks,cn=config" "(cn=backup_*)" nstaskstatus

When the task is done, you'll see the "Backup finished." status:
dn: cn=backup_2012_4_5_9_35_35,cn=backup,cn=tasks,cn=config
nstaskstatus: Backup finished.

Another way would be checking the errors log, which logs the end of the back up.

[05/Apr/2012:09:32:09 -0700] - Backing up file 31 (/var/lib/dirsrv/slapd-totoro/bak/totoro-2012_4_5_9_32_8/DBVERSION)
[05/Apr/2012:09:32:09 -0700] - Backup finished.

Thanks,
--noriko
MATON Brett wrote:

Hi guys,

 

I want to tar up the backup set once a db2bak.pl backup job has finished.

however because it writes an entry to the LDAP and doesn’t wait for the backup to finish I was wondering what the best way to determine when the backup has actually finished?

 

I was thinking that I could

Ldapsearch at intervals for the backup cn entry, when it’s not there anymore backup has finished

Ro maybe watch the audit log for a delete action on the backup cn entry..

 

Any thoughts appreciated.

 

Cheers,

Brett

 

-------------------------------------------------------------------

GreeNRB
NRB considers its environmental responsibility and goes for green IT.
May we ask you to consider yours before printing this e-mail?  

NRB, daring to commit
This e-mail and any attachments, which may contain information that is confidential and/or protected by intellectual property rights, are intended for the exclusive use of the above-mentioned addressee(s). Any use (including reproduction, disclosure and whole or partial distribution in any form whatsoever) of their content is prohibited without prior authorization of NRB. If you have received this message by error, please contact the sender promptly by resending this e-mail back to him (her), or by calling the above number. Thank you for subsequently deleting this e-mail and any files attached thereto.

--
389 users mailing list
389-users@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-users