Re: db_verify

[Noriko Hosoi <nhosoi@xxxxxxxxxx>]

Ville Silventoinen wrote:

> Thanks Noriko, that explains a lot!
>
> I still have that other problem to solve, related to re-creating the 
> database (see the end of my first email). I will try to reproduce this 
> problem and create some test data this week.

Hi Ville,

> If I delete a database with the Console, it leaves behind couple of 
> index files:
>
> -rw-------  1 w3secure systems 16384 Mar 28 17:05 ancestorid.db4
> -rw-------  1 w3secure systems    18 Mar 28 17:03 DBVERSION
> -rw-------  1 w3secure systems 32768 Mar 28 17:05 id2entry.db4
>
> These index files don't seem to shrink when new entries are imported. 
> dbscan still shows the deleted entries in id2entry.
>
> I noticed a problem when I import a small set of entries, delete the 
> database, import large set of entries and if I query the entries, I 
> get the entries from the first set (they don't exist in the second 
> set). I can reproduce the problem. If I delete ancestorid.db4 and 
> id2entry.db4 manually when I delete the database, I don't have this 
> problem. Is there a reason why those two files are not deleted? Or can 
> this whole thing be caused by corrupted data?

I wonder this might be your case?

http://www.redhat.com/docs/manuals/dir-server/ag/7.1/entry_dist.html#22293
Deleting a Database
The following procedure describes deleting a directory database using 
the Directory Server Console. Deleting a database deletes the 
configuration information and entries for that database only, *not the 
physical database itself.*

http://www.redhat.com/docs/manuals/dir-server/ag/7.1/dbmanage.html#1117312
Importing a Database from the Console
When you perform an import operation from the Directory Server Console, 
an ldapmodify operation is executed to *append data*, as well as to 
modify and delete entries.

To overwrite the existing data, please take this step before importing 
the data.
http://www.redhat.com/docs/manuals/dir-server/ag/7.1/dbmanage.html#1117339
Initializing a Database from the Console

Or you could do the same thing from the command-line interface:
http://www.redhat.com/docs/manuals/dir-server/ag/7.1/dbmanage.html#1117378
Importing from the Command-Line
You can use three methods for importing data through the command-line:
. Using ldif2db - This import method overwrites the contents of your 
database and requires the server to be stopped.
. Using ldif2db.pl - This import method overwrites the contents of your 
database while the server is still running.

Hope it helps,
--noriko

> Thanks,
> Ville
>
>
> On Thu, 12 Apr 2007, Noriko Hosoi wrote:
>
> > Thank you, Ville, for the test data.  I could reproduce the db_verify 
> > problem.
> >
> > I have good news and bad news. :)  Good news, first...  Your db is 
> > not corrupted.  The error report from verify-db.pl is bogus.
> >
> > Bad news, next.  Please take a look at this bug.  We are going to 
> > provide a fixed utility some time soon.
> >
> > Summary: verify-db.pl (db_verify) does not work on a little endian 
> > machine
> >
> > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=236256
> >
> > Sorry about this inconvenience, and thank you for reporting the problem!
> > --noriko
> >
> > Ville Silventoinen wrote:
> >
> > > Hi Noriko,
> > >
> > > sorry it took so long to reply, I've been busy with other work.
> > >
> > > On Fri, 30 Mar 2007, Noriko Hosoi wrote:
> > >
> > > > Ville Silventoinen wrote:
> > > >
> > > > > I asked my manager but he doesn't think it's a good idea for 
> > > > > security reasons. The problem is that the data is our NIS 
> > > > > mail.aliases and passwd, and we don't want to distribute them to 
> > > > > the internet. He suggested I'll modify the data, so I can send a 
> > > > > sample to you. I'll do that next week.
> > > >
> > > > That would be great. Thanks! I'm interested in what type of 
> > > > characters your data contain. E.g., character set is UTF-8? Some of 
> > > > your DNs could contain any special characters such as '\'? etc...
> > >
> > >
> > > The character set should be plain ASCII. I created an imaginary 
> > > mail.aliases file. You can download it from here:
> > >
> > > http://www.ebi.ac.uk/systems-srv/mp/file-exchange/
> > >
> > > Type in "fedorads" to the Pass Phrase input box and click Go. You 
> > > should see three files: mail.aliases, mail.aliases.ldif and 
> > > 99user.ldif.
> > >
> > > I can reproduce my problem with the above files, for example, I've 
> > > tested like this:
> > >
> > > 1. Delete existing ebiRoot database (you could use userRoot).
> > > 2. Delete db/ebiRoot directory.
> > > 3. Create ebiRoot database.
> > > 4. Shutdown slapd.
> > > 5. Run db2index and verify-db.pl. No errors.
> > > 6. Start slapd.
> > > 7. Import mail aliases. I've tried with the Console and my own CLI, 
> > > which can import LDIF and add entries one-by-one. The method doesn't 
> > > seem to matter.
> > > 8. Shutdown slapd.
> > > 9. Run db2index and verify-db.pl, verify gives errors:
> > >
> > > Verify log files in db ... Good
> > > Verify db/ebiRoot/ancestorid.db4 ...
> > > DB ERROR: db_verify: Page 2: out-of-order key at entry 254
> > > DB ERROR: db_verify: DB->verify: db/ebiRoot/ancestorid.db4: 
> > > DB_VERIFY_BAD: Database verification failed
> > > Secondary index file ancestorid.db4 in db/ebiRoot is corrupted.
> > > Please run db2index(.pl) for reindexing.
> > > Verify db/ebiRoot/objectclass.db4 ...
> > > DB ERROR: db_verify: Page 2: out-of-order key at entry 255
> > > DB ERROR: db_verify: DB->verify: db/ebiRoot/objectclass.db4: 
> > > DB_VERIFY_BAD: Database verification failed
> > > Secondary index file objectclass.db4 in db/ebiRoot is corrupted.
> > > Please run db2index(.pl) for reindexing.
> > > Verify db/ebiRoot/nsuniqueid.db4 ... Good
> > > Verify db/ebiRoot/parentid.db4 ...
> > > DB ERROR: db_verify: Page 1: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: DB->verify: db/ebiRoot/parentid.db4: 
> > > DB_VERIFY_BAD: Database verification failed
> > > Secondary index file parentid.db4 in db/ebiRoot is corrupted.
> > > Please run db2index(.pl) for reindexing.
> > > Verify db/ebiRoot/cn.db4 ...
> > > DB ERROR: db_verify: Page 10: out-of-order key at entry 249
> > > DB ERROR: db_verify: DB->verify: db/ebiRoot/cn.db4: DB_VERIFY_BAD: 
> > > Database verification failed
> > > Secondary index file cn.db4 in db/ebiRoot is corrupted.
> > > Please run db2index(.pl) for reindexing.
> > > Verify db/ebiRoot/id2entry.db4 ... Good
> > > Verify db/ebiRoot/entrydn.db4 ... Good
> > > Verify db/ebiRoot/rfc822mailmember.db4 ...
> > > DB ERROR: db_verify: Page 2: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 3: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: DB->verify: db/ebiRoot/rfc822mailmember.db4: 
> > > DB_VERIFY_BAD: Database verification failed
> > > Secondary index file rfc822mailmember.db4 in db/ebiRoot is corrupted.
> > > Please run db2index(.pl) for reindexing.
> > >
> > > > So, in your ldif data, the mail attribute also has this type of 
> > > > value: "|/homes/majordom/wrapper 
> > > > stripmime.pl|/homes/majordom/wrapper resend -l foobar-dev 
> > > > foobar-dev-outgoing"?
> > >
> > >
> > > No, the People entries have a simpler mail value, like "foo@xxxxxxxxx".
> > >
> > > > And your mail index has the default indexing type: presence, 
> > > > equality, and substring?
> > >
> > >
> > > Yes.
> > >
> > > > What type of indexing does the rfc822MailMember attribute have?
> > >
> > >
> > > I've tried without any indexing, with presence and equality and with 
> > > presence, equality and substring. The above errors are from 
> > > verify-db.pl when I have presence and equality indeces. If I have 
> > > presence, equality and substring, I get these errors for 
> > > rfc822MailMember:
> > >
> > > Verify db/ebiRoot/rfc822mailmember.db4 ...
> > > DB ERROR: db_verify: Page 13: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 6: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 8: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 12: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 3: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 7: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 10: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 15: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 4: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 14: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 5: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 9: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: Page 11: unsorted duplicate set in sorted-dup 
> > > database
> > > DB ERROR: db_verify: DB->verify: db/ebiRoot/rfc822mailmember.db4: 
> > > DB_VERIFY_BAD: Database verification failed
> > > Secondary index file rfc822mailmember.db4 in db/ebiRoot is corrupted.
> > > Please run db2index(.pl) for reindexing.
> > >
> > > > Have we already heard what platform you are running the FDS on?
> > >
> > >
> > > CentOS release 4.4, Linux 2.6.9-42.ELsmp. Pentium III 2x1266MHz 
> > > CPUs, 2GB memory, SCSI disks. I'm using FDS 1.0.4.
> > >
> > > I'm away this week Wed-Fri, so I'll get back to you next week.
> > >
> > > Thanks for the help!
> > >
> > > Ville
> > >
> > > --
> > > Fedora-directory-users mailing list
> > > Fedora-directory-users@xxxxxxxxxx
> > > https://www.redhat.com/mailman/listinfo/fedora-directory-users
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@xxxxxxxxxx
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

--
Fedora-directory-users mailing list
Fedora-directory-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/fedora-directory-users