Re: iSCSI over RBD

Joshua Chen <cschen@xxxxxxxxxxxxxxxxxxx> · Thu, 4 Jan 2018 17:50:10 +0800

Dear all,
  Although I managed to run gwcli and created some iqns, or luns,
but I do need some working config example so that my initiator could connect and get the lun.

  I am familiar with targetcli and I used to do the following ACL style connection rather than password, 
the targetcli setting tree is here:

(or see this page)

#targetcli ls
o- / ......................................................................................................................... [...]
  o- backstores .............................................................................................................. [...]
  | o- block .................................................................................................. [Storage Objects: 1]
  | | o- vmware_5t .......................................................... [/dev/rbd/rbd/vmware_5t (5.0TiB) write-thru activated]
  | |   o- alua ................................................................................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
  | o- fileio ................................................................................................. [Storage Objects: 0]
  | o- pscsi .................................................................................................. [Storage Objects: 0]
  | o- ramdisk ................................................................................................ [Storage Objects: 0]
  | o- user:rbd ............................................................................................... [Storage Objects: 0]
  o- iscsi ............................................................................................................ [Targets: 1]
  | o- iqn.2017-12.asiaa.cephosd1:vmware5t ............................................................................... [TPGs: 1]
  |   o- tpg1 .................................................................................................. [gen-acls, no-auth]
  |     o- acls ......................................................................................................... [ACLs: 12]
  |     | o- iqn.1994-05.com.redhat:15dbed23be9e .................................................................. [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1994-05.com.redhat:15dbed23be9e-ovirt1 ........................................................... [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1994-05.com.redhat:2af344ba6ae5-ceph-admin-test .................................................. [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1994-05.com.redhat:67669afedddf .................................................................. [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1994-05.com.redhat:67669afedddf-ovirt3 ........................................................... [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1994-05.com.redhat:a7c1ec3c43f7 .................................................................. [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1994-05.com.redhat:a7c1ec3c43f7-ovirt2 ........................................................... [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1994-05.com.redhat:b01662ec2129-ceph-node2 ....................................................... [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1994-05.com.redhat:d46b42a1915b-ceph-node3 ....................................................... [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1994-05.com.redhat:e7692a10f661-ceph-node1 ....................................................... [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1998-01.com.vmware:localhost-0f904dfd ............................................................ [Mapped LUNs: 1]
  |     | | o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     | o- iqn.1998-01.com.vmware:localhost-6af62e4c ............................................................ [Mapped LUNs: 1]
  |     |   o- mapped_lun0 ............................................................................. [lun0 block/vmware_5t (rw)]
  |     o- luns .......................................................................................................... [LUNs: 1]
  |     | o- lun0 .................................................... [block/vmware_5t (/dev/rbd/rbd/vmware_5t) (default_tg_pt_gp)]
  |     o- portals .................................................................................................... [Portals: 1]
  |       o- 172.20.0.12:3260 ................................................................................................. [OK]
  o- loopback ......................................................................................................... [Targets: 0]
  o- xen_pvscsi ....................................................................................................... [Targets: 0]

My targetcli setup procedure is like this, could someone translate it to gwcli equivalent procedure?
sorry for asking for this due to lack of documentation and examples.
thanks in adavance 

Cheers
Joshua

targetcli /backstores/block create name=vmware_5t dev=/dev/rbd/rbd/vmware_5t
targetcli /iscsi/ create iqn.2017-12.asiaa.cephosd1:vmware5t
targetcli /iscsi/iqn.2017-12.asiaa.cephosd1:vmware5t/tpg1/portals delete ip_address=0.0.0.0 ip_port=3260

targetcli
  cd /iscsi/iqn.2017-12.asiaa.cephosd1:vmware5t/tpg1
        portals/  create 172.20.0.12
        acls/
                create iqn.1994-05.com.redhat:e7692a10f661-ceph-node1
                create iqn.1994-05.com.redhat:b01662ec2129-ceph-node2
                create iqn.1994-05.com.redhat:d46b42a1915b-ceph-node3
                create iqn.1994-05.com.redhat:15dbed23be9e
                create iqn.1994-05.com.redhat:a7c1ec3c43f7
                create iqn.1994-05.com.redhat:67669afedddf
                create iqn.1994-05.com.redhat:15dbed23be9e-ovirt1
                create iqn.1994-05.com.redhat:a7c1ec3c43f7-ovirt2
                create iqn.1994-05.com.redhat:67669afedddf-ovirt3
                create iqn.1994-05.com.redhat:2af344ba6ae5-ceph-admin-test
                create iqn.1998-01.com.vmware:localhost-6af62e4c
                create iqn.1998-01.com.vmware:localhost-0f904dfd
                cd ..
                set attribute generate_node_acls=1
                cd luns
                create /backstores/block/vmware_5t

On Thu, Jan 4, 2018 at 10:55 AM, Joshua Chen <cschen@xxxxxxxxxxxxxxxxxxx> wrote:
I had the same problem before, mine is CentOS, and when I created
/iscsi/create iqn_bla-bla
it goes
ocal LIO instance already has LIO configured with a target - unable to  continue

then finally the solution happened to be, turn off target service

systemctl stop target
systemctl disable target

somehow they are doing the same thing, you need to disable 'target' service (targetcli) in order to allow gwcli (rbd-target-api) do it's job.

Cheers
Joshua

On Thu, Jan 4, 2018 at 2:39 AM, Mike Christie <mchristi@xxxxxxxxxx> wrote:
On 12/25/2017 03:13 PM, Joshua Chen wrote:

> Hello folks,

>   I am trying to share my ceph rbd images through iscsi protocol.

>

> I am trying iscsi-gateway

> http://docs.ceph.com/docs/master/rbd/iscsi-overview/

>

>

> now

>

> systemctl start rbd-target-api

> is working and I could run gwcli

> (at a CentOS 7.4 osd node)

>

> gwcli

> /> ls

> o- /

> .........................................................................................................................

> [...]

>   o- clusters

> ........................................................................................................

> [Clusters: 1]

>   | o- ceph

> ............................................................................................................

> [HEALTH_OK]

>   |   o- pools

> ..........................................................................................................

> [Pools: 1]

>   |   | o- rbd

> ...........................................................................

> [(x3), Commit: 0b/25.9T (0%), Used: 395M]

>   |   o- topology

> ................................................................................................

> [OSDs: 9,MONs: 3]

>   o- disks

> ..........................................................................................................

> [0b, Disks: 0]

>   o- iscsi-target

> .....................................................................................................

> [Targets: 0]

>

>

> but when I created iscsi-target, I got

>

> Local LIO instance already has LIO configured with a target - unable to

> continue

>

>

> /> /iscsi-target create

> iqn.2003-01.org.linux-iscsi.ceph-node1.x8664:sn.571e1ab51af2

> Local LIO instance already has LIO configured with a target - unable to

> continue

> />

>

Could you send the output of

targetcli ls

?

What distro are you using?

You might just have a target setup from a non gwcli source. Maybe from

the distro targetcli systemd tools.

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com