On Wed, Jun 1, 2016 at 6:52 PM, qisy <qisy@xxxxxxxxxxxx> wrote: > my test fio > > fio -ioengine=libaio -bs=4k -direct=1 -thread -rw=randwrite -size=1G > -filename=test.iso -name="CEPH 4KB randwrite test" -iodepth=32 -runtime=60 > You were testing direct-IO performance. Hammer does not handle direct-IO correctly, data are cached in ceph-fuse. Regards Yan, Zheng > > 在 16/6/1 15:22, Yan, Zheng 写道: > >> On Mon, May 30, 2016 at 10:22 PM, qisy <qisy@xxxxxxxxxxxx> wrote: >>> >>> Hi, >>> After jewel released fs product ready version, I upgrade the old >>> hammer >>> cluster, but iops droped a lot >>> >>> I made a test, with 3 nodes, each one have 8c 16G 1osd, the osd >>> device >>> got 15000 iops >>> >>> I found ceph-fuse client has better performance on hammer than >>> jewel. >>> >>> fio randwrite 4K >>> | | jewel server | hammer server | >>> |jewel client | 480+ iops | no test | >>> |hammer client | 6000+ iops | 6000+ iops | >> >> please post the fio config file. >> >> Regards >> Yan, Zheng >> >>> ceph-fuse(jewel) mount with jewel server got pity iops, is there any >>> special options need to set? >>> If I continue use ceph-fuse(hammer) with jewel server, any problems >>> will >>> cause? >>> >>> thanks >>> >>> my ceph.conf below: >>> >>> [global] >>> fsid = xxxxxxx >>> mon_initial_members = xxx, xxx, xxx >>> mon_host = 10.0.0.1,10.0.0.2,10.0.0.3 >>> auth_cluster_required = cephx >>> auth_service_required = cephx >>> auth_client_required = cephx >>> >>> filestore_xattr_use_omap = true >>> osd_pool_default_size = 2 >>> osd_pool_default_min_size = 1 >>> mon_data_avail_warn = 15 >>> mon_data_avail_crit = 5 >>> mon_clock_drift_allowed = 0.6 >>> >>> [osd] >>> osd_disk_threads = 8 >>> osd_op_threads = 8 >>> journal_block_align = true >>> journal_dio = true >>> journal_aio = true >>> journal_force_aio = true >>> filestore_journal_writeahead = true >>> filestore_max_sync_interval = 15 >>> filestore_min_sync_interval = 10 >>> filestore_queue_max_ops = 25000 >>> filestore_queue_committing_max_ops = 5000 >>> filestore_op_threads = 32 >>> osd_journal_size = 20000 >>> osd_map_cache_size = 1024 >>> osd_max_write_size = 512 >>> osd_scrub_load_threshold = 1 >>> osd_heartbeat_grace = 30 >>> >>> [mds] >>> mds_session_timeout = 120 >>> mds_session_autoclose = 600 >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@xxxxxxxxxxxxxx >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |