On Thu, 2016-06-30 at 20:53 -0600, Rich Megginson wrote: > On 06/30/2016 08:14 PM, William Brown wrote: > > On Thu, 2016-06-30 at 20:01 -0600, Rich Megginson wrote: > >> On 06/30/2016 07:52 PM, William Brown wrote: > >>> Hi, > >>> > >>> I've been thinking about this for a while, so I decided to dump my > >>> thoughts to a document. I think I won't get to implementing this for a > >>> while, but it would really help our server performance. > >>> > >>> http://www.port389.org/docs/389ds/design/logging-performance-improvement.html > >> Looks good. Can we quantify the current log overhead? > > Sure, I could probably sit down and work out a way to bench mark > > this ..... > > > > But without the alternative being written, hard to say. I could always > > patch out logging and drop the lock in a hacked build so we can show > > what "without logging contention" looks like? > > That's only one part of it - you'd have to figure out some way to get > rid of the overhead of the formatting and flushing in the operation > threads too. > This change would move the formatting into the thread that does the write + flush. Currently, we format the message in the operation thread, and send it to the log buffer, and every now and again, an "unlucky" operation thread has to stall to write the buffer .... However, I'm proposing we send unformatted time struct + unformatted message to the queue, then the actual thread which is separate does the dequeue and format. > I suppose you could just write it and see what happens. Well, a hacked up test that has slapi_log_access return 0 immediately (IE does not take the lock) showed a 6% performance improvement on my laptop. On a higher cpu count system, this would exceed that number easily, as more cpus == more cost to acquire the lock. I've put the numbers on the design doc page, but I think with this, the proof would be in the implementation. We also likely wouldn't see the benefit of this as much until we remove other serialisation points in the code IE ldbm vs lmdb, plugins that take locks, etc. So it's only a 6% possible gain now, but it may be 10% or more in the future in combination with other efforts such as nunc-stans. > > > > >>> > >>> -- > >>> 389-devel mailing list > >>> 389-devel@xxxxxxxxxxxxxxxxxxxxxxx > >>> https://lists.fedoraproject.org/admin/lists/389-devel@xxxxxxxxxxxxxxxxxxxxxxx > >> > >> -- > >> 389-devel mailing list > >> 389-devel@xxxxxxxxxxxxxxxxxxxxxxx > >> https://lists.fedoraproject.org/admin/lists/389-devel@xxxxxxxxxxxxxxxxxxxxxxx > > > > > > -- > > 389-devel mailing list > > 389-devel@xxxxxxxxxxxxxxxxxxxxxxx > > https://lists.fedoraproject.org/admin/lists/389-devel@xxxxxxxxxxxxxxxxxxxxxxx > > > -- > 389-devel mailing list > 389-devel@xxxxxxxxxxxxxxxxxxxxxxx > https://lists.fedoraproject.org/admin/lists/389-devel@xxxxxxxxxxxxxxxxxxxxxxx -- Sincerely, William Brown Software Engineer Red Hat, Brisbane
Attachment:
signature.asc
Description: This is a digitally signed message part
-- 389-devel mailing list 389-devel@xxxxxxxxxxxxxxxxxxxxxxx https://lists.fedoraproject.org/admin/lists/389-devel@xxxxxxxxxxxxxxxxxxxxxxx
