[389-devel] Please review & comment: [389 Project] #412: memberof performance enhancement

Noriko Hosoi <nhosoi@xxxxxxxxxx> · Thu, 19 Jul 2012 11:40:32 -0700



    https://fedorahosted.org/389/ticket/412

      
https://fedorahosted.org/389/attachment/ticket/412/0001-Trac-Ticket-412-memberof-performance-enhancement.patch

      
        Fix description:

        memberof.c: replaced DN strings with Slapi_DN and set the

        normalized info to Slapi_Value flags. It reduces the number

        of slaip_dn_normalize_ext call by ~25%.

      
        attr.c, slapi-plugin.h: introduced a new API slapi_attr_

        value_cmp_ext which takes Slapi_Value instead of struct

        berval. By replacing with Slapi_Value, the value flag

        (e.g., normalized info) can be passed to the syntax plugin.

      
      value.c: changed slapi_value_compare to call slapi_attr_

      value_cmp_ext instead of slapi_attr_value_cmp.

      
        Replying to nkinder:

      
          What sort of performance increase do these changes give you in
          your testing?

        
      Unfortunately, there was no observable performance gain. Let me
      explain it with the test env next...

      As replied to the Nathan's question, I
      could not see any performance gain in the elapsed time, although
      it reduced the count of slapi_dn_normalize_ext by ~25%.

      
      But the server's behaviour observed in the test was interesting. 
      The dominant time of the elapsed time spent in the memberof
      operation is likely in libdb.  The time difference 14 sec vs. 27
      sec between "Straight ldapmodify ...; sh memof_script.sh" and
      "ldapmodify ...; restart the server; [ldapsearch <all>; ]sh
      memof_script.sh" cannot be explained in the server level.  
      Callgrind outputs almost the same graph except the libdb
      internal.  And once we put all memberof operations in one
      transaction, the response time is extremely short.  We have not
      enabled betxn by default yet.  Can we make a plan to do so for
      brushing up the code especially on Fedora?

      
Result:

      
        Note: With/Without? the attached patch:
        0001-Trac-Ticket-412-memberof-performance-enhancement.patch,
        there was no difference in the elapsed time.

      
      This test adds 1000 user entries first (ldapmodify...).

        Then memof_script.sh adds a group entry which contains the 1000
        members; the script waits for all the user entries have memberof
        attribute value.

        (The test files are attached to the trac #412 with the
        instructions.)

      
        Straight ldapmodify ...; sh memof_script.sh: 14 seconds

        ldapmodify ...; restart the server; sh memof_script.sh: 27
        seconds

        ldapmodify ...; restart the server; ldapsearch <all>; sh
        memof_script.sh: 27 seconds

        (I.e, the difference is not due to the entries in the entry
        cache or not)

      
      Enable betxn of the Memberof Plugin

      dn: cn=MemberOf?
      Plugin,cn=plugins,cn=config

      nsslapd-pluginType: betxnpostoperation

      Straight ldapmodify ...; sh memof_script.sh: < 1 second

      ldapmodify ...; restart the server; ldapsearch <all>; sh
      memof_script.sh: < 1 second 

      
Thanks,

      --noriko

    
--
389-devel mailing list
389-devel@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/389-devel