On Tue, Feb 12, 2019 at 1:30 AM Vijay Bellur <vbellur@xxxxxxxxxx> wrote:
On Tue, Feb 5, 2019 at 10:57 PM Xavi Hernandez <xhernandez@xxxxxxxxxx> wrote:On Wed, Feb 6, 2019 at 7:00 AM Poornima Gurusiddaiah <pgurusid@xxxxxxxxxx> wrote:On Tue, Feb 5, 2019, 10:53 PM Xavi Hernandez <xhernandez@xxxxxxxxxx wrote:On Fri, Feb 1, 2019 at 1:51 PM Xavi Hernandez <xhernandez@xxxxxxxxxx> wrote:On Fri, Feb 1, 2019 at 1:25 PM Poornima Gurusiddaiah <pgurusid@xxxxxxxxxx> wrote:Can the threads be categorised to do certain kinds of fops?Could be, but creating multiple thread groups for different tasks is generally bad because many times you end up with lots of idle threads which waste resources and could increase contention. I think we should only differentiate threads if it's absolutely necessary.Read/write affinitise to certain set of threads, the other metadata fops to other set of threads. So we limit the read/write threads and not the metadata threads? Also if aio is enabled in the backend the threads will not be blocked on disk IO right?If we don't block the thread but we don't prevent more requests to go to the disk, then we'll probably have the same problem. Anyway, I'll try to run some tests with AIO to see if anything changes.I've run some simple tests with AIO enabled and results are not good. A simple dd takes >25% more time. Multiple parallel dd take 35% more time to complete.Thank you. That is strange! Had few questions, what tests are you running for measuring the io-threads performance(not particularly aoi)? is it dd from multiple clients?Yes, it's a bit strange. What I see is that many threads from the thread pool are active but using very little CPU. I also see an AIO thread for each brick, but its CPU usage is not big either. Wait time is always 0 (I think this is a side effect of AIO activity). However system load grows very high. I've seen around 50, while on the normal test without AIO it's stays around 20-25.Right now I'm running the tests on a single machine (no real network communication) using an NVMe disk as storage. I use a single mount point. The tests I'm running are these:
- Single dd, 128 GiB, blocks of 1MiB
- 16 parallel dd, 8 GiB per dd, blocks of 1MiB
- fio in sequential write mode, direct I/O, blocks of 128k, 16 threads, 8GiB per file
- fio in sequential read mode, direct I/O, blocks of 128k, 16 threads, 8GiB per file
- fio in random write mode, direct I/O, blocks of 128k, 16 threads, 8GiB per file
- fio in random read mode, direct I/O, blocks of 128k, 16 threads, 8GiB per file
- smallfile create, 16 threads, 256 files per thread, 32 MiB per file (with one brick down, for the following test)
- self-heal of an entire brick (from the previous smallfile test)
- pgbench init phase with scale 100
I run all these tests for a replica 3 volume and a disperse 4+2 volume.Are these performance results available somewhere? I am quite curious to understand the performance gains on NVMe!
I'm updating test results with the latest build. I'll report it here once it's complete.
Xavi
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-devel
