|
On Monday 01 May 2017 10:24 AM, David
Turner wrote:
You don't have results that include the added network
latency of having replica 3 replicating across multiple hosts.
The reads would be very similar as the primary is the only thing
that is read from, but writes will not return until after all 3
copies are written.
I started this as an experiment to see why table creation takes too
much time on CephFS. That was my prime focus, David. So haven't
tried it on pools with size > 1.
On Sat, Apr 29, 2017, 9:46 PM Babu Shanmugam < babu@xxxxxxxx>
wrote:
H i,
I did some basic experiments with mysql and measured the
time taken by a set of operations on CephFS and RBD. The
RBD measurements are taken on a 1GB RBD disk with ext4
filesystem. Following are my observation. The time listed
below are in seconds.
|
|
Plain file system |
CephFS |
RBD |
| Mysql install db |
7.9 |
38.3 |
36.4 |
| Create table |
0.43 |
4.2 |
2.5 |
| Drop table |
0.14 |
0.21 |
0.40 |
| Create table + 1000 recs |
2.76 |
4.69 |
5.07 |
| Create table + 10000 recs |
|
7.69 |
11.96 |
| Create table + 100K recs |
|
12.06 |
29.65 |
From the above numbers, CephFS seems to fare
very well while creating records whereas RBD does well
while creating a table. I tried measuring the syscalls of
ceph-osd, ceph-mds and the mysqld while creating a table
on CephFS and RBD. Following is how the key syscalls of
mysqld performed while creating a table (time includes
wait time as well).
| Syscalls of MYSQLD |
CephFS |
RBD |
| fsync |
338.237 ms |
183.697 ms |
| fdatasync |
75.635 ms |
96.359 ms |
| io_submit |
50 us |
151 us |
| open |
2266 us |
61 us |
| close |
1186 us |
33 us |
| write |
115 us |
51 us |
From the above numbers, open, close and
fsync syscalls take too much time on CephFs as compared to
RBD.
Sysbench results are below;
| Sysbence 100K
records in 60 secs |
CephFS |
RBD |
| Read Queries performed |
631876 |
501690 |
| Other Queries performed |
90268 |
71670 |
| No. of transactions |
45134 |
35835 |
| No. of transactions per
sec |
752.04 |
597.17 |
| R/W requests per sec |
10528.55 |
8360.37 |
| Other operations per
sec |
1504.08 |
1194.34 |
Above numbers seems to indicate the CephFS does very well
with MYSQL transactions, better than RBD.
Following is my setup;
Num MONs : 1
Num OSDs : 1
Num MDSs : 1
Disk : 10 GB
Qemu disk file (Both journal and data in the same disk)
Ceph version : 10.2.5 (Built
from source)
Build config : ./configure
--without-debug --without-fuse --with-libaio \
--without-libatomic-ops --without-hadoop
--with-nss --without-cryptopp \
--without-gtk2 --disable-static --with-jemalloc
\
--without-libzfs --without-lttng
--without-babeltrace \
--with-eventfd --with-python -without-kinetic
--without-librocksdb \
--without-openldap \
CFLAGS="-g -O2 -fPIC" CXXFLAGS="-g -O2
-std=c++11 -fPIC
Ceph conf : Apart from host
and network settings nothing else is configured
CephFS mount options:
rw,relatime,name=cephfs,secret=<hidden>,acl
RBD mount options:
rw,relatime,stripe=1024,data="">
All the processes were run in a Qemu virtual machine with
Linux 4.4.18 kernel
Searching for "Mysql on CephFS" in google does not give
any useful results. If this kind of experiments had been
done previously and shared publicly, kindly share a link
to it.
If you are aware of anything that I can do to optimise
this, kindly let me know. I am willing to continue this
experiment to see how well we can optimise CephFs for
mysql.
Thank you,
Babu Shanmugam
www.aalam.io
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
|
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
