Re: Problems with tgt with ceph support

["Toni F. [ackstorm]" <toni.fuentes@xxxxxxxxxxx>]

Here's my results.

The performance test was seq write

On 15/07/13 10:12, Toni F. [ackstorm] wrote:

I'm going to do a performance test with fio to see the difference.

Regards

On 12/07/13 18:15, Dan Mick wrote:

Ceph performance is a very very complicated subject. How does that compare to other access methods?  Say, rbd import/export for an easy test?

On Jul 12, 2013 8:22 AM, "Toni F. [ackstorm]" <toni.fuentes@xxxxxxxxxxx> wrote:

It works, but the performance is very poor. 100MB/s or less

Which are your performance experience?

Regards

On 12/07/13 13:56, Toni F. [ackstorm] wrote:

It works!

Thanks for all

On 12/07/13 11:23, Toni F. [ackstorm] wrote:

Yes! it seems that i wasn't compiled the rbd support.

System:
    State: ready
    debug: off
LLDs:
    iscsi: ready
Backing stores:
    bsg
    sg
    null
    ssc
    aio
    rdwr (bsoflags sync:direct)
Device types:
    disk
    cd/dvd
    osd
    controller
    changer
    tape
    passthrough
iSNS:
    iSNS=Off
    iSNSServerIP=
    iSNSServerPort=3205
    iSNSAccessControl=Off

I'm going to recompile it
Thanks a lot!

On 11/07/13 07:45, Dan Mick wrote:

On 07/10/2013 04:12 AM, Toni F. [ackstorm] wrote:

Hi all,

I have installed the v0.37 of tgt.

To test this feature i follow the
http://ceph.com/dev-notes/adding-support-for-rbd-to-stgt/ guide

When i launch the command:

tgtadm --lld iscsi --mode logicalunit --op new --tid 1 --lun 0
--backing-store iscsi-image --bstype rbd

fails

First i think that lun cannot be 0 because lun 0 is used by the
controller (previous command)

This worked when I first wrote the backend, but tgt may have changed; I'll investigate and change the blog entry if so.  Thanks.

If i launch the correct command with lun 1 i have this error:

tgtadm: invalid request

In syslog:

Jul 10 12:54:03 datastore-lnx001 tgtd: device_mgmt(245) sz:28
params:path=iscsi-image,bstype=rbd
Jul 10 12:54:03 datastore-lnx001 tgtd: tgt_device_create(532) failed to
find bstype, rbd

What's wrong? not supported?

Where did you get your tgtd?  Was it built with rbd support (CEPH_RBD defined
in the environment for make)?

sudo ./tgtadm --lld iscsi --op show --mode system

should tell you.

How did you set up access to ceph.conf?

--

Toni Fuentes Rico
toni.fuentes@xxxxxxxxxxx
Administración de Sistemas

Oficina central: 902 888 345

ACK STORM, S.L.
ISO 9001:2008 (Cert.nº. 536932)
http://ackstorm.es

Este mensaje electrónico contiene información de ACK STORM, S.L. que es privada y confidencial, siendo para el uso exclusivo de la persona(s) o entidades arriba  mencionadas. Si usted no es el destinatario señalado, le informamos que cualquier divulgación, copia, distribución o uso de los contenidos está prohibida. Si usted ha recibido este mensaje por error, por favor borre su contenido y comuníquenoslo en la dirección ackstorm@xxxxxxxxxxx

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  

-- 

Toni Fuentes Rico
toni.fuentes@xxxxxxxxxxx
Administración de Sistemas

Oficina central: 902 888 345

ACK STORM, S.L.
ISO 9001:2008 (Cert.nº. 536932)
http://ackstorm.es

Este mensaje electrónico contiene información de ACK STORM, S.L. que es privada y confidencial, siendo para el uso exclusivo de la persona(s) o entidades arriba  mencionadas. Si usted no es el destinatario señalado, le informamos que cualquier divulgación, copia, distribución o uso de los contenidos está prohibida. Si usted ha recibido este mensaje por error, por favor borre su contenido y comuníquenoslo en la dirección ackstorm@xxxxxxxxxxx
  

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
  

-- 

Toni Fuentes Rico
toni.fuentes@xxxxxxxxxxx
Administración de Sistemas

Oficina central: 902 888 345

ACK STORM, S.L.
ISO 9001:2008 (Cert.nº. 536932)
http://ackstorm.es

Este mensaje electrónico contiene información de ACK STORM, S.L. que es privada y confidencial, siendo para el uso exclusivo de la persona(s) o entidades arriba  mencionadas. Si usted no es el destinatario señalado, le informamos que cualquier divulgación, copia, distribución o uso de los contenidos está prohibida. Si usted ha recibido este mensaje por error, por favor borre su contenido y comuníquenoslo en la dirección ackstorm@xxxxxxxxxxx

PERFORMANCE 

RBD Device /dev/rbd/rbd/iscsi-image-part1 exported with tgt

seq-write-1: (g=0): rw=write, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 1.59
Starting 1 process
seq-write-1: Laying out IO file(s) (1 file(s) / 10240MB)
Jobs: 1 (f=1): [W] [100.0% done] [0K/210.7M /s] [0 /52.7K iops] [eta 00m:00s]
seq-write-1: (groupid=0, jobs=1): err= 0: pid=12600
  write: io=10240MB, bw=258334KB/s, iops=64583 , runt= 40590msec
    clat (usec): min=4 , max=185785 , avg=14.34, stdev=285.90
     lat (usec): min=4 , max=185785 , avg=14.60, stdev=285.95
    bw (KB/s) : min=140000, max=680912, per=100.51%, avg=259650.01, stdev=137146.60
  cpu          : usr=10.13%, sys=40.54%, ctx=6729, majf=1, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=0/2621440/0, short=0/0/0
     lat (usec): 10=90.82%, 20=8.15%, 50=0.81%, 100=0.07%, 250=0.03%
     lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
     lat (msec): 2=0.01%, 4=0.01%, 10=0.07%, 20=0.02%, 50=0.01%
     lat (msec): 250=0.01%

Run status group 0 (all jobs):
  WRITE: io=10240MB, aggrb=258333KB/s, minb=264533KB/s, maxb=264533KB/s, mint=40590msec, maxt=40590msec

Disk stats (read/write):
  sdk: ios=0/17848, merge=0/159, ticks=0/5393060, in_queue=5461852, util=94.16%


TGT-RBD connector

seq-write-1: (g=0): rw=write, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 1.59
Starting 1 process
seq-write-1: Laying out IO file(s) (1 file(s) / 10240MB)
Jobs: 1 (f=1): [W] [100.0% done] [0K/132.7M /s] [0 /33.2K iops] [eta 00m:00s]
seq-write-1: (groupid=0, jobs=1): err= 0: pid=13158
  write: io=10240MB, bw=198316KB/s, iops=49578 , runt= 52874msec
    clat (usec): min=4 , max=593897 , avg=19.11, stdev=964.17
     lat (usec): min=4 , max=593898 , avg=19.35, stdev=964.17
    bw (KB/s) : min=   19, max=662384, per=102.35%, avg=202980.17, stdev=138701.97
  cpu          : usr=7.31%, sys=29.81%, ctx=3997, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=0/2621440/0, short=0/0/0
     lat (usec): 10=93.87%, 20=5.18%, 50=0.76%, 100=0.07%, 250=0.03%
     lat (usec): 500=0.01%, 750=0.01%, 1000=0.01%
     lat (msec): 2=0.01%, 4=0.01%, 10=0.01%, 20=0.03%, 50=0.03%
     lat (msec): 100=0.01%, 500=0.01%, 750=0.01%

Run status group 0 (all jobs):
  WRITE: io=10240MB, aggrb=198315KB/s, minb=203075KB/s, maxb=203075KB/s, mint=52874msec, maxt=52874msec

Disk stats (read/write):
  sdk: ios=0/15935, merge=0/182, ticks=0/6846740, in_queue=6881512, util=95.43%

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com