Re: iscsi resize -vmware datastore cannot increase size - FIXED

[Steven Vacaroaia <stef97@xxxxxxxxx>]

OK .. fixed

Just for posterity, this was not a CEPH / gwcli issue but a VMWare quark 

There were 2 issues 

1. authentication issues ( not sure why)

These were addressed by running 

 auth chap=cephuser/paasword for all gwcli hosts

then 

esxcli iscsi adapter auth chap set --direction=uni --authname=cephuser --secret=password --level=discouraged -A vmhba64

on all esxi hosts then

enable "inherit settings from parent" on the target authentication

then rescan 

The above HAS to be done for all ESXi servers that are mounting the datastore 

2. even after there are no more authentication errors, datastore cannot be extent from vCenter 

IT HAS TO BE DONE DIRECTLY on one of the ESXi servers which can be accomplished by simply logging in as root at https://esxiserver/ui

Many thanks for your suggestions 

Steven

On Fri, 25 Oct 2019 at 10:43, Steven Vacaroaia <stef97@xxxxxxxxx> wrote:

the error seems to indicate mismatched passwords 

on the gwcli host , /var/log/messages contains the following

osd02 kernel: CHAP user or password not set for Initiator ACL
Oct 25 10:37:22 osd02 kernel: Security negotiation failed.
Oct 25 10:37:22 osd02 kernel: iSCSI Login negotiation failed.
Oct 25 10:37:22 osd02 kernel: CHAP user or password not set for Initiator ACL
Oct 25 10:37:22 osd02 kernel: Security negotiation failed.
Oct 25 10:37:22 osd02 kernel: iSCSI Login negotiation failed.

However, I (re)entered auth details in gwcli which seems fine 

iscsi-target...san1-7bb6d7ac> info
Client Iqn .. iqn.1998-01.com.vmware:vsan1-7bb6d7ac
Ip Address .. 10.10.35.111
Alias      ..
Logged In  .. LOGGED_IN
Auth
- chap .. cephuser/CORRECT_PASSWORD
Group Name ..
Luns
- rbd.rep01    .. lun_id=0
- rbd.vmware01 .. lun_id=1

What am I missing ???

On Fri, 25 Oct 2019 at 10:30, Steven Vacaroaia <stef97@xxxxxxxxx> wrote:

spoke to soon

still same issue event after re entering credentials 

here is an excerpt from ESXi server

 [esx.problem.storage.iscsi.discovery.login.error] iSCSI discovery to 10.10.35.202 on vmhba64 failed. The Discovery target returned a login error of: 0201.

On Fri, 25 Oct 2019 at 10:08, Steven Vacaroaia <stef97@xxxxxxxxx> wrote:

I can confirm that, after reentering credentials for the target on each ESXi server and rescanning storage, the device appear and datastore can be increased 

Thanks for your help and patience 

Steven

On Fri, 25 Oct 2019 at 09:59, Steven Vacaroaia <stef97@xxxxxxxxx> wrote:

I noticed this 

[vob.iscsi.discovery.login.error] discovery failure on vmhba64 to 10.10.35.202 because the target returned a login status of 0201.

A restart of rbd services will require reentering chap credentials on targets ?

Steven 

On Fri, 25 Oct 2019 at 09:57, Steven Vacaroaia <stef97@xxxxxxxxx> wrote:

Yes, I did. 

I event restarted rbd-target services 

uname -a
Linux osd01.chi.medavail.net 4.18.11-1.el7.elrepo.x86_64 #1 SMP Sat Sep 29 09:42:38 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@osd01 ~]# rpm -qa | grep tcmu

tcmu-runner-1.4.0-1.el7.x86_64 

On Fri, 25 Oct 2019 at 09:51, Jason Dillaman <jdillama@xxxxxxxxxx> wrote:

On Fri, Oct 25, 2019 at 9:49 AM Steven Vacaroaia <stef97@xxxxxxxxx> wrote:
>
> Thanks for your prompt response
> Unfortunately , still no luck
> Device shows with correct size under "Device backing" but not showing at all  under "increase datastore capacity)
>
> resize rbd.rep01 7T
> ok
> /disks> ls
> o- disks ......................................................................................................... [13.0T, Disks: 2]
>   o- rbd.rep01 ........................................................................................................ [rep01 (7T)]

Did you rescan the LUNs in VMware after this latest resize attempt?
What kernel and tcmu-runner version are you using?

> On Fri, 25 Oct 2019 at 09:24, Jason Dillaman <jdillama@xxxxxxxxxx> wrote:
>>
>> On Fri, Oct 25, 2019 at 9:13 AM Steven Vacaroaia <stef97@xxxxxxxxx> wrote:
>> >
>> > Hi,
>> > I am trying to increase size of a datastore made available through ceph iscsi rbd
>> > The steps I followed are depicted below
>> > Basically gwcli report correct data and even VMware device capacity is correct but when tried to increase it there is no device listed
>> >
>> > I am using ceph-iscsi-config-2.6-42.gccca57d.el7 and ceph 13.2.2
>> >
>> > Any guidance/help will be appreciated
>> >
>> > 1. increase rbd size
>> > rbd -p rbd resize --size 6T rep01
>> > Resizing image: 100% complete...done.
>>
>> Never resize the RBD images backing a LUN via the "rbd" CLI -- use
>> "gwcli" to resize the images and it will handle resizing the LUNs.
>>
>> > 2. restart gwcli
>> >  systemctl restart rbd-target-gw &&  systemctl restart rbd-target-api
>> >
>> > 3 check size
>> >  gwcli
>> > /iscsi-target...go-ceph/hosts> ls
>> > o- hosts .................................................................................................... [Hosts: 8: Auth: CHAP]
>> >   o- iqn.1998-01.com.vmware:vsan5-66c18541 ................................................ [LOGGED-IN, Auth: CHAP, Disks: 2(12.0T)]
>> >   | o- lun 0 .................................................................................... [rbd.vmware01(6.0T), Owner: osd01]
>> >   | o- lun 1 ....................................................................................... [rbd.rep01(6.0T), Owner: osd02]
>> >
>> > 4. VMware rescan devices
>> >
>> >
>> > _______________________________________________
>> > ceph-users mailing list
>> > ceph-users@xxxxxxxxxxxxxx
>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>>
>>
>> --
>> Jason
>>


--
Jason

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com