Hi Mark, Am 05.03.19 um 23:12 schrieb Mark Nelson: > Hi Stefan, > > > Could you try running your random write workload against bluestore and > then take a wallclock profile of an OSD using gdbpmp? It's available here: > > > https://github.com/markhpc/gdbpmp sure but it does not work: # ./gdbpmp.py -p 3760442 -n 100 -o gdbpmp.data Attaching to process 3760442...0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 Thread 1 "ceph-osd" received signal SIGCONT, Continued. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 Done. Gathering Samples Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 2 "log" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 2 "log" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 2 "log" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 2 "log" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 2 "log" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 2 "log" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 . Thread 1 "ceph-osd" received signal SIGINT, Interrupt. 0x00007f917b6a615f in pthread_cond_wait@@GLIBC_2.3.2 () from target:/lib/x86_64-linux-gnu/libpthread.so.0 Greets, Stefan > Thanks, > > Mark > > > On 3/5/19 2:29 AM, Stefan Priebe - Profihost AG wrote: >> Hello list, >> >> while the performance of sequential writes 4k on bluestore is very high >> and even higher than filestore i was wondering what i can do to optimize >> random pattern as well. >> >> While using: >> fio --rw=write --iodepth=32 --ioengine=libaio --bs=4k --numjobs=4 >> --filename=/tmp/test --size=10G --runtime=60 --group_reporting >> --name=test --direct=1 >> >> I get 36000 iop/s on bluestore while having 11500 on filestore. >> >> Using randwrite gives me 17000 on filestore and only 9500 on bluestore. >> >> This is on all flash / ssd running luminous 12.2.10. >> >> Greets, >> Stefan >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |