Re: ceph 9.2.0 SAMSUNG ssd performance issue?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



thanks nick,

filestore-> journal_latency: ~1.1ms

214.0/180611

0.0011848669239415096


seems ssd write is ok, any other idea is highly appreciated! 

  

 "filestore": {

        "journal_queue_max_ops": 300,

        "journal_queue_ops": 0,

        "journal_ops": 180611,

        "journal_queue_max_bytes": 33554432,

        "journal_queue_bytes": 0,

        "journal_bytes": 32637888155,

        "journal_latency": {

            "avgcount": 180611,

            "sum": 214.095788552

        },

        "journal_wr": 176801,

        "journal_wr_bytes": {

            "avgcount": 176801,

            "sum": 33122885632

        },

        "journal_full": 0,

        "committing": 0,

        "commitcycle": 14648,

        "commitcycle_interval": {

            "avgcount": 14648,

            "sum": 73299.187956076

        },



2016-02-12 18:04 GMT+08:00 Nick Fisk <nick@xxxxxxxxxx>:


> -----Original Message-----
> From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of
> Huan Zhang
> Sent: 12 February 2016 10:00
> To: Irek Fasikhov <malmyzh@xxxxxxxxx>
> Cc: ceph-users <ceph-users@xxxxxxxx>
> Subject: Re: ceph 9.2.0 SAMSUNG ssd performance issue?
>
> "op_w_latency":
>      "avgcount": 42991,
>       "sum": 402.804741329
>
> 402.0/42991
> 0.009350794352306296
>
> ~9ms latency, that means this ssd not suitable for journal device?

I believe that counter includes lots of other operations in the OSD including the journal write. If you want pure journal stats, I would under the Filestore->journal_latency counter

>
>
>  "osd": {
>         "op_wip": 0,
>         "op": 58683,
>         "op_in_bytes": 7309042294,
>         "op_out_bytes": 507137488,
>         "op_latency": {
>             "avgcount": 58683,
>             "sum": 484.302231121
>         },
>         "op_process_latency": {
>             "avgcount": 58683,
>             "sum": 323.332046552
>         },
>         "op_r": 902,
>         "op_r_out_bytes": 507137488,
>         "op_r_latency": {
>             "avgcount": 902,
>             "sum": 0.793759596
>         },
>         "op_r_process_latency": {
>             "avgcount": 902,
>             "sum": 0.619918138
>         },
>         "op_w": 42991,
>         "op_w_in_bytes": 7092142080,
>         "op_w_rlat": {
>             "avgcount": 38738,
>             "sum": 334.643717526
>         },
>         "op_w_latency": {
>             "avgcount": 42991,
>             "sum": 402.804741329
>         },
>         "op_w_process_latency": {
>             "avgcount": 42991,
>             "sum": 260.489972416
>         },
> ...
>
>
> 2016-02-12 15:56 GMT+08:00 Irek Fasikhov <malmyzh@xxxxxxxxx>:
> Hi.
> You need to read : https://www.sebastien-han.fr/blog/2014/10/10/ceph-
> how-to-test-if-your-ssd-is-suitable-as-a-journal-device/
>
>
> С уважением, Фасихов Ирек Нургаязович
> Моб.: +79229045757
>
> 2016-02-12 10:41 GMT+03:00 Huan Zhang <huan.zhang.jn@xxxxxxxxx>:
> Hi,
>
> ceph VERY SLOW with 24 osd(SAMSUNG ssd).
> fio /dev/rbd0 iodepth=1 direct=1   IOPS only ~200
> fio /dev/rbd0 iodepth=32 direct=1 IOPS only ~3000
>
> But test single ssd deive with fio:
> fio iodepth=1 direct=1   IOPS  ~15000
> fio iodepth=32 direct=1 IOPS  ~30000
>
> Why ceph SO SLOW? Could you give me some help?
> Appreciated!
>
>
> My Enviroment:
> [root@szcrh-controller ~]# ceph -s
>     cluster eb26a8b9-e937-4e56-a273-7166ffaa832e
>      health HEALTH_WARN
>             1 mons down, quorum 0,1,2,3,4 ceph01,ceph02,ceph03,ceph04,ceph05
>      monmap e1: 6 mons at {ceph01=
> 10.10.204.144:6789/0,ceph02=10.10.204.145:6789/0,ceph03=10.10.204.146:67
> 89/0,ceph04=10.10.204.147:6789/0,ceph05=10.10.204.148:6789/0,ceph06=0.0
> .0.0:0/5
> }
>             election epoch 6, quorum 0,1,2,3,4
> ceph01,ceph02,ceph03,ceph04,ceph05
>      osdmap e114: 24 osds: 24 up, 24 in
>             flags sortbitwise
>       pgmap v2213: 1864 pgs, 3 pools, 49181 MB data, 4485 objects
>             144 GB used, 42638 GB / 42782 GB avail
>                 1864 active+clean
>
> [root@ceph03 ~]# lsscsi
> [0:0:6:0]    disk    ATA      SAMSUNG MZ7KM1T9 003Q  /dev/sda
> [0:0:7:0]    disk    ATA      SAMSUNG MZ7KM1T9 003Q  /dev/sdb
> [0:0:8:0]    disk    ATA      SAMSUNG MZ7KM1T9 003Q  /dev/sdc
> [0:0:9:0]    disk    ATA      SAMSUNG MZ7KM1T9 003Q  /dev/sdd
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>



_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux