On 05/31/2017 06:00 AM, Predrag Zečević - Technical Support Analyst wrote:
> Hi all,
>
> long time ago we have started with 389-DS and due to lack of
> experience I have installed and used admin server (which is abandoned
> later, because it is too complicated and requires someone at keyboard).
>
> As consequence of that, we have started to replicate netscapeRoot
> space... During time, we have upgraded s/w from initial
> 389-ds-1.2.1-1.el5 (started from FDS repository, moved to EPEL one
> later) to today's 389-ds-base-1.3.5.14-1.el6.x86_64 (this one is
> compiled from source and that was introduced before we have migrated
> boxes from RHEL5 to RHEL6 - actually CentOS OS).
>
> During various phases of upgrades, netscapeRoot replicas went out of
> sync (we did not spotted that, because of bug in monitoring script -
> that is another issue).
>
> Our setup includes MultiMaster ReadWrite replication (ldap1 <-->
> ldap2) and one ReadOnly (ldap3, consumes from both suppliers in MMR).
>
> Right now, this:
> $ for ldap in ldap1 ldap2; do
> ldapsearch -x -H ldaps://${ldap}.MyDomain.com -b "cn=mapping
> tree,cn=config" -D "cn=Directory Manager" -w ${DMPASS} -o ldif-wrap=no
> objectClass=nsDS5ReplicationAgreement |\
> awk -vLDAP=${ldap} '/^dn/ {printf("#===== %s =====#\n%s\n", LDAP,
> $0); next}; /^nsDS5ReplicaHost:/ {printf("%s\n", $0); next;};
> /^nsds5replicaLastUpdateStatus:/ {printf("%s\n", $0); next;}'
> done
>
> returns (I have excluded working MyDomain replicas output):
> $ #===== ldap1 =====#
> dn: cn=2eLDAPmmr,cn=replica,cn=o\3Dnetscaperoot,cn=mapping tree,cn=config
> nsDS5ReplicaHost: ldap2.MyDomain.com
> nsds5replicaLastUpdateStatus: Error (0) No replication sessions
> started since server startup
> #===== ldap1 =====#
> dn: cn=2eLDAPror,cn=replica,cn=o\3Dnetscaperoot,cn=mapping tree,cn=config
> nsDS5ReplicaHost: ldap3.MyDomain.com
> nsds5replicaLastUpdateStatus: Error (0) No replication sessions
> started since server startup
> #===== ldap2 =====#
> dn: cn=2eLDAPmmr,cn=replica,cn=o\3Dnetscaperoot,cn=mapping tree,cn=config
> nsDS5ReplicaHost: ldap1.MyDomain.com
> nsds5replicaLastUpdateStatus: Error (0) No replication sessions
> started since server startup
> #===== ldap2 =====#
> dn: cn=2eLDAPror,cn=replica,cn=o\3Dnetscaperoot,cn=mapping tree,cn=config
> nsDS5ReplicaHost: ldap3.MyDomain.com
> nsds5replicaLastUpdateStatus: Error (0) No replication sessions
> started since server startup
>
> I have tried various tricks to recover that replication, but w/o luck...
>
> When I check (for example ldap1) with this:
> $ ldapsearch -xLLLo ldif-wrap=no -H ldaps://ldap1.MyDomain.com -D
> 'cn=directory manager' -w ${DMPASS} -b o=netscapeRoot
> '(&(nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff)(objectclass=nstombstone))'
>
> I get as result:
> dn: cn=replica,cn=o\3Dnetscaperoot,cn=mapping tree,cn=config
> objectClass: nsDS5Replica
> objectClass: top
> nsDS5ReplicaRoot: o=netscaperoot
> nsDS5ReplicaType: 3
> nsDS5Flags: 1
> nsDS5ReplicaId: 11
> nsds5ReplicaPurgeDelay: 604800
> nsDS5ReplicaBindDN: cn=replication manager,cn=config
> nsDS5ReplicaReferral: ldap://ldap2.MyDomain.com:636/o%3dnetscaperoot
> cn: replica
> nsState:: CwAAAAAAAACRKiRZAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAA==
> nsDS5ReplicaName: dc964102-1dd111b2-8970c75e-63880000
> nsds50ruv: {replicageneration} 4dcb9f790000000b0000
> nsds50ruv: {replica 11 ldap://ldap1.MyDomain.com:0}
> nsds50ruv: {replica 21 ldap://ldap2.MyDomain.com:0}
> 4dda4a3a000000150000 4fd5f742000300150000
> nsds5agmtmaxcsn:
> o=netscaperoot;2eLDAPror;ldap3.MyDomain.com;636;unavailable
> nsruvReplicaLastModified: {replica 11 ldap://ldap1.MyDomain.com:0}
> 00000000
> nsruvReplicaLastModified: {replica 21 ldap://ldap2.MyDomain.com:0}
> 00000000
> nsds5ReplicaChangeCount: 1
> nsds5replicareapactive: 0
>
> Tried to CleanRUV (ldif applied with ldapmodify command to all
> suppliers and consumers):
>
> $ cat /tmp/ldap.cleanRUV-tasks-for-netscapeRoot-replica.11.ldif
> dn: cn=replica,cn=o\3Dnetscaperoot,cn=mapping tree,cn=config
> changetype: modify
> replace: nsds5task
> nsds5task: CLEANRUV11
>
> At some moment, ldap1 replied:
> "ldap_modify: Server is unwilling to perform (53)"
>
> which explains nothing, because that error means:
>
> "Indicates that the LDAP server cannot process the request because of
> server-defined restrictions. This error is returned for the following
> reasons: The add entry request violates the server's structure
> rules...OR...The modify attribute request specifies attributes that
> users cannot modify...OR...Password restrictions prevent the
> action...OR...Connection restrictions prevent the action. "
>
> Right now, CleanRUV task is stuck...
You should be using the cleanAllRUV task:
https://access.redhat.com/documentation/en-us/red_hat_directory_server/10/html/configuration_command_and_file_reference/perl_scripts#cleanallruv.pl
or read up on:
http://www.port389.org/docs/389ds/howto/howto-cleanruv.html#cleanallruv
It also looks like your replicas are not initialized - so I would also
try that after cleaning out the old replica ids(ruvs).
> and replication is still broken... Similar situation is present on
> ldap2, with RUV 21 (if not worse):
>
> dn: cn=replica,cn=o\3Dnetscaperoot,cn=mapping tree,cn=config
> objectClass: nsDS5Replica
> objectClass: top
> nsDS5ReplicaRoot: o=netscaperoot
> nsDS5ReplicaType: 3
> nsDS5Flags: 1
> nsDS5ReplicaId: 21
> nsds5ReplicaPurgeDelay: 604800
> nsDS5ReplicaBindDN: cn=replication manager,cn=config
> nsDS5ReplicaReferral: ldap://ldap1.MyDomain.com:636/o%3dnetscaperoot
> cn: replica
> nsState:: FQAAAAAAAADeiyVZAAAAAAAAAAAAAAAAAQAAAAAAAAABAAAAAAAAAA==
> nsDS5ReplicaName: cb016902-1dd111b2-821cbcea-f7780000
> nsds50ruv: {replicageneration} 4dcb9f790000000b0000
> nsds50ruv: {replica 21 ldap://ldap2.MyDomain.com:0}
> 4dda4a3a000000150000 4fd5f742000300150000
> nsds5agmtmaxcsn:
> o=netscaperoot;2eLDAPmmr;ldap1.MyDomain.com;636;unavailable
> nsds5agmtmaxcsn:
> o=netscaperoot;2eLDAPror;ldap3.MyDomain.com;636;unavailable
> nsruvReplicaLastModified: {replica 21 ldap://ldap2.MyDomain.com:0}
> 00000000
> nsds5ReplicaChangeCount: 1
> nsds5replicareapactive: 0
>
>
> # What would be proper way to get out from this situation?
> # Do I have to execute CleanAllRUV task and start replication from
> scratch or there is better way?
>
> BTW, loglevel is set to 8192, so from ldap1 logs:
> $ sudo grep cleanruv_task: /var/log/dirsrv/slapd-ldap?/errors
> [31/May/2017:09:11:39 +0200] NSMMReplicationPlugin - cleanruv_task:
> cleaning rid (11)...
>
> we see that task is "started" and never finished
>
> Any advice or documentation (which is more up-2-date) than:
> *
> http://directory.fedoraproject.org/docs/389ds/howto/howto-cleanruv.html#cleanruv
> *
> https://access.redhat.com/documentation/en-us/red_hat_directory_server/9.0/html/administration_guide/managing_replication-solving_common_replication_conflicts
> *
> http://directory.fedoraproject.org/docs/389ds/FAQ/troubleshoot-cleanallruv.html
> (CleanRUV FAQ troubleshooting is missing at all)
>
> is welcome.
>
> With best regards.
> Predrag Zečević
_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
