Re: Please review new Replication Diff Tool

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, 2017-04-12 at 17:02 -0400, Mark Reynolds wrote:
> Hello,
> 
> This is a beta version of a replication diff tool written in python. 
> 
> Design page (this needs updating - I hope to get that done tonight)
> 
> http://www.port389.org/docs/389ds/design/repl-diff-tool-design.html
> 
> 
> Current usage:
> 
>   -v, --verbose         Verbose output
>   -o FILE, --outfile=FILE
>                         The output file
>   -D BINDDN, --binddn=BINDDN
>                         The Bind DN (REQUIRED)
>   -w BINDPW, --bindpw=BINDPW
>                         The Bind password (REQUIRED)
>   -h MHOST, --master_host=MHOST
>                         The Master host (default localhost)
>   -p MPORT, --master_port=MPORT
>                         The Master port (default 389)
>   -H RHOST, --replica_host=RHOST
>                         The Replica host (REQUIRED)
>   -P RPORT, --replica_port=RPORT
>                         The Replica port (REQUIRED)
>   -b SUFFIX, --basedn=SUFFIX
>                         Replicated suffix (REQUIRED)
>   -l LAG, --lagtime=LAG
>                         The amount of time to ignore inconsistencies
> (default
>                         300 seconds)
>   -Z CERTDIR, --certdir=CERTDIR
>                         The certificate database directory for startTLS
>                         connections
>   -i IGNORE, --ignore=IGNORE
>                         Comma separated list of attributes to ignore
>   -M MLDIF, --mldif=MLDIF
>                         Master LDIF file (offline mode)
>   -R RLDIF, --rldif=RLDIF
>                         Replica LDIF file (offline mode)
> 
> |Examples: python repl-diff.py -D "cn=directory manager" -w PASSWORD -h
> localhost -p 389 -H remotehost -P 5555 -b "dc=example,dc=com" ||python repl-diff.py -D "cn=directory manager" -w PASSWORD -h localhost
> -p 389 -H remotehost -P 5555 -b "dc=example,dc=com" -Z
> /etc/dirsrv/slapd-localhost|
> |python repl-diff.py -M /tmp/master.ldif -R /tmp/replica.ldif |
> 
> 
> How long the tool takes to run depends on the number of entries per
> database.  See performance numbers below 
> 
> Entries per Replica     Time
> 
> ---------------------------------
> 100k                    40 seconds
> 500k                    3m 30secs
> 1 million               7m 30secs
> 2 million               14 minutes
> 10 million              ~70 minutes
> 
> 
> 
> I'd be very interested in feedback, RFE's, and bugs.

Hey mate, 

The tool looks great, awesome work on this. Really impressive that you
got it to 70 minutes for 10 million entries.

How responsive is the server during this process? We aren't going to
cause some odd resource exhaustion? 

import optparse

With python, optparse is deprecated. Can we use argparse instead? It's
nearly identical. Lots of examples of this in dsctl. 

With connect to replicas, some sites may only have ldaps (provided by a
load balancer). So our scripts should really be taking an LDAPurl, a
certdir, and a starttls flag. Because ldaps://localhost + certdir is a
valid option, but if we force call start_tls_s(), we break it. As well,
someone may use ldapi:// etc. It also saves on port options and more
flags to the cli because we can do ldap://localhost:30389 etc. 

Hope that helps, I'll be happy to review again later!

For now, I think our strategy with this should be to add it to
389-ds-base, and later we can move this into lib389 when we can. How
does that sound? 

-- 
Sincerely,

William Brown
Software Engineer
Red Hat, Australia/Brisbane

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
389-devel mailing list -- 389-devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-devel-leave@xxxxxxxxxxxxxxxxxxxxxxx

[Index of Archives]     [Fedora Directory Announce]     [Fedora Users]     [Older Fedora Users Mail]     [Fedora Advisory Board]     [Fedora Security]     [Fedora Devel Java]     [Fedora Desktop]     [ATA RAID]     [Fedora Marketing]     [Fedora Mentors]     [Fedora Package Review]     [Fedora Art]     [Fedora Music]     [Fedora Packaging]     [CentOS]     [Fedora SELinux]     [Big List of Linux Books]     [KDE Users]     [Fedora Art]     [Fedora Docs]

  Powered by Linux