On 06/27/2013 08:52 PM, Rich Megginson wrote:
On 06/27/2013 12:32 PM, Michael Lang wrote:
On 06/27/2013 06:13 PM, Rich Megginson wrote:
On 06/27/2013 04:57 AM, Michael Lang wrote:
Dear all,
I would like to clarify how the procedure to "export -> LDIF"
through the 389-console GUI is done (or how if reproduce should be
done).
As far as I've understood the exports on the server itself is done
through creating a appropriate entry in cn=export,cn=task,cn=config
For "remote" machines using the GUI this isn't valid as the
nsFilename attribute requires a local write able location. What
I've seen through looking at the
LDAP protocol queries done during an export-to-ldif on console
machine task is a query to the schema listing all attributeTypes
which are then sent as attribute-retrieve-list in the ldap search.
At least that's how I was able to reproduce the same LDIF from
GUI-Console export and manually. Is this correct and should this
be done in that way ? (I've noticed also the "Warning: If you
don't have permissions" statement when using the LDIF export in
the Console which more-or-less ack's my approach)
Right. There is no way to do a database export to LDIF to a remote
file on a remote machine.
Are you trying to figure out a way to generate an LDIF file using
ldapsearch that looks exactly like a database export LDIF file? Why?
reason for that, I might not have "file based" access on the server
itself but need to do backup's of the database which are not "setup
specific" (with setup specific I mean without the right database
configurations I might not be able to import the content).
in this special case why I started to look into the ldif create, I
had database backups but ran into a situation where my directory
console was showing content but ldapsearch (and other
tools/applications) where not getting any content at all. It was a
strange situation and restoring the database trough restore didn't
help where droping the databases and importing and LDIF was working
(I still had another server running with the content). As mentioned
above for the machines I have under complete access this shouldn't
be the issue, but I also have other machines where I might be forced
to do this through the LDAP protocol.
That particular case sounds like you were missing the "aci" attribute
which is an operational attribute.
In general, if you want "everything"(1), you can do a search like this:
ldapsearch -b dc=your,dc=suffix
'(|(objectclass=*)(objectclass=ldapsubentry))' \* $OPERATIONAL_ATTRS
where $OPERATIONAL_ATTRS is a list of operational attributes you can
get from the schema. You'll definitely want "aci" in that list.
If you know python, python-ldap has a nice schema parser.
(1) This still won't get you everything - specifically, you won't get
deleted entries and other replication metadata - If you think you
need that information, add (objectclass=nsTombstone) to the search
filter above.
Rich,
thanks for you explanation, the 389-console GUI itself doesn't export
nsTombstone objects and the purpose is to be able to restore the data
not the replication setting for the database the suffice resides on
(at least that's what I would understood from the attributes in the
entry returned by the search)
again, thanks for the python-schema hint, much easier then doing it
manually, this is how I would scripted reproduce the 389-console LDIF
Database Export without the GUI)
<source lang="python">
#!/usr/bin/env python
import ldap
import ldap.schema
import logging
import sys
import ldif
logger = logging.getLogger('LDIFexport')
logger.addHandler(logging.StreamHandler(sys.stderr))
logger.setLevel(logging.INFO)
ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)
class LDIFDumper(object):
def __init__(self, uri=None):
self.uri = uri
self.dn, self.schema = ldap.schema.urlfetch(self.uri)
self.objclass = set()
self.attributes = set()
self._objects = []
self.__fetch_objectClasses__()
self.__fetch_attributes__()
def __fetch_objectClasses__(self):
for obj in self.schema.tree(ldap.schema.ObjectClass).keys():
obj = self.schema.get_obj(ldap.schema.ObjectClass, obj)
if obj == None:
continue
self.objclass.update(obj.names)
def __fetch_attributes__(self):
for attr in self.schema.tree(ldap.schema.AttributeType).keys():
attr = self.schema.get_obj(ldap.schema.AttributeType, attr)
if attr == None:
continue
self.attributes.update(attr.names)
def db2ldif(self, basedn=None, binddn=None, bindpw=None,
fileName=None):
try:
if not isinstance(fileName, ldif.LDIFWriter):
if not isinstance(fileName, file):
fileName = open(fileName, 'w')
ldifFile = ldif.LDIFWriter(fileName)
except IOError, e:
return (False, u'unable to create LDIFFile %s' %
fileName.name)
try:
self.srv = ldap.initialize(self.uri)
self.srv.start_tls_s()
except ldap.CONNECT_ERROR:
logger.info(u'unable to initialize start_tls, continue
without encryption')
except ldap.SERVER_DOWN:
return (False, u'cannot connect to %s SERVER_DOWN' %
self.uri)
try:
if not binddn == None:
if self.srv.simple_bind_s(binddn, bindpw) != (97, []):
return (False, u'couldn\'t authenticate as %s
against %s' % (binddn, self.srv._uri))
except ldap.INVALID_CREDENTIALS:
return (False, u'couldn\'t authenticate as %s against %s'
% (binddn, self.srv._uri))
try:
# for additional data change filterstr to
#
filterstr='(|(|(objectClass=*)(objectClass=ldapsubentry))(objectClass=nsTombstone))'
by Rich Megginson <rmeggins@xxxxxxxxxx>
for dn, attrs in self.srv.search_s(basedn,
ldap.SCOPE_SUBTREE, attrlist=list(self.attributes)):
self._objects.append((dn, attrs))
ldifFile.unparse(dn, attrs)
except ldap.ADMINLIMIT_EXCEEDED:
return (False, u'couldn\'t fetch all records as %s need
more privileges or increase limits on the server' % binddn)
except ldap.SIZELIMIT_EXCEEDED:
return (False, u'couldn\'t fetch all records as %s need
more privileges or increase limits on the server' % binddn)
self.srv.unbind_s()
return (True, True)
if __name__ == '__main__':
import getpass
lfd = LDIFDumper(uri='ldap://localhost/')
lfd.db2ldif(basedn='dc=ctbto,dc=org', binddn='cn=Directory
Manager', bindpw=getpass.getpass(), fileName='/tmp/dc=ctbto,dc=org.ldif')
</source>
head from /tmp/dc=ctbto,dc=org.ldif
dn: dc=ctbto,dc=org
aci: (targetattr != "userPassword") (version 3.0; acl "Anonymous
access"; al
low (read, search, compare)userdn = "ldap:///anyone";;)
aci: (targetattr != "nsroledn||aci")(version 3.0; acl "Allow self
entry modi
fication except for nsroledn and aci attributes"; allow (write)userdn
="lda
p:///self";)
aci: (targetattr = "*")(version 3.0; acl "Configuration Adminstrator";
allow
(all) userdn =
"ldap:///uid=admin,ou=Administrators,ou=TopologyManagement,
o=NetscapeRoot";)
aci: (targetattr ="*")(version 3.0;acl "Configuration Administrators
Group";
allow (all) (groupdn = "ldap:///cn=Configuration Administrators,
ou=Groups,
ou=TopologyManagement, o=NetscapeRoot");)
aci: (targetattr = "*")(version 3.0; acl "SIE Group"; allow
(all)groupdn = "
ldap:///dc=ctbto,dc=org";;)
createTimestamp: 20130626125103Z
creatorsName: cn=directory manager
domaincomponent: ctbto
entrydn: dc=ctbto,dc=org
entryid: 1
hasSubordinates: FALSE
modifiersName: cn=directory manager
modifyTimestamp: 20130626125103Z
nsUniqueId: f13af0a6-de5e11e2-ab7e96d1-4e21dbc8
numSubordinates: 0
objectClass: top
objectClass: dcobject
subschemaSubentry: cn=schema
