Re: LDBM recommended Setting

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, 2018-08-14 at 10:36 -0600, David Boreham wrote:
> 
> While it is true that you want to have the highest possible hit ratio
> on the two kinds of cache slapd maintains in order to achieve optimal
> read performance, you _can_ simply configure quite small caches for
> slapd (e.g. 1 few thousand entry cache and a few 100 MB DB cache) and
> rely on the OS's filesystem cache and the relatively high speed of
> I/O these days (SSDs...). This has the big advantage of removing the
> cognitive load associated with setting the "correct" cache size.
> 
> On 8/14/2018 9:48 AM, Mark Reynolds wrote:
> > 
> > On 08/14/2018 11:32 AM, Paul Whitney wrote:
> > > Hi guys,
> > > 
> > > Am looking to improve performance in my 389 DS deployment.  In
> > > reviewing the documentation, the recommended size for the LDBM
> > > cache is the sum of the backend database + 15% of the backend
> > > database.  For me that comes out to almost 27GB.  Seems high
> > > considering the database cache is set very high as well. Is that
> > > a recommended setting or is there a better practice?
> >  Well the more of the database you can keep in the cache the better
> > the performance.   However, you are talking about the same cache:
> > LDBM cache is the same thing as the database cache. 
> > 
> > But you should also include the entry cache for your database.  So
> > you almost want to double that to 50GB total ;-)    27 GB for DB
> > cache, and 20 GB for entry cache (this varies of course for the
> > entry cache and it will probably be less than 20GB, but you won't
> > know until you start priming the entry cache and checking the
> > database monitor - trying to achieve 99% cache hit ratio).

There are other things to look at too. You have to look at the
lifecycle of IO. There are two main parts:.

Entries: Entry Cache <-> DB cache <-> Disk

Indexes: DB cache <-> Disk


The entry cache will keep copies of Slapi_Entry in the memory -
generally this is the biggest affect on performance. It prevents re-
accessing id2entry and performing the deserialisation process.
Deserialising entries is slow!!!

The db cache is caching writes and reads from the indexes and between
all the .db files. It can help speed up searches and more, but it's
affects are more limited compared to the entry cache. 

Finally, if you are on a 1.3.7 release, you can consider *raising* your
idlscanlimit. 1.3.7 contains a filter optimisation step that means
idlscanlimit is now possibly slowing you down on certain searches,
rather than helping. I saw good results in benchmarks raising it to
99999999, but we did not yet make this a default as our team wanted
much more evidence and testing.

In same cases if your idlscanlimit is "low", this causes needless
access to the slapi-entries in the entry cache (and possible
eviction/inclusion), when accessing an index would be faster. If you
want to know more, I can write it up.

Finally, look at your thread count (newer 1.3.x should set this
automatically), to make sure you are getting good parallel throughput. 

As with all changes, test and review to make sure it helps you in your
situation. 

Today with 1.3.x we pushed to make many defaults "correct" out of box,
including autotuning backends and more. Perhaps consider using the
autotuning subsystem to help guide your decisions. 

Hope that helps! 

>  
> _______________________________________________
> 389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to 
> 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: 
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx/message/F7KUNPB7DYLYE6EXKXCANRIBONAABBOP/
-- 
Sincerely,

William
_______________________________________________
389-users mailing list -- 389-users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-users@xxxxxxxxxxxxxxxxxxxxxxx/message/4KRF6HZQXLUJ5EU7FVMB2RBCISBI7HLH/




[Index of Archives]     [Fedora User Discussion]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [Fedora News]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Maintainers]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Legacy]     [Fedora Desktop]     [Fedora Fonts]     [ATA RAID]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Centos]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora QA]     [Fedora Triage]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Tux]     [Yosemite News]     [Yosemite Photos]     [Linux Apps]     [Maemo Users]     [Gnome Users]     [KDE Users]     [Fedora Tools]     [Fedora Art]     [Fedora Docs]     [Maemo Users]     [Asterisk PBX]     [Fedora Sparc]     [Fedora Universal Network Connector]     [Fedora ARM]

  Powered by Linux