Have you disabled the vaai functions in ESXi? I can't remember off the top of my head, but one of them makes everything slow to a crawl. > -----Original Message----- > From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Oliver Dzombic > Sent: 02 September 2016 09:50 > To: ceph-users@xxxxxxxxxxxxxx > Subject: Re: vmware + iscsi + tgt + reservations > > Hi, > > VMFS-5.61 file system spanning 1 partitions. > Mode: public > > The Filesystem is working fine ( on the 1st node where multiple instances are started ). And it continues to work fine, also after > mounting the same LUN to the 2nd node and trying write operations there. > > So i have no reason to think that anything is corrupted. > > The problem seems to be, that tgt simply does not support reservations. > > And this way, the reservation of the lun of the 1st node got never released. > > I could release it now by force, and maybe it will work, but i cant manually releasing locks every time vmotion or what ever > administrativ operation is done. > > So if there is no solution with tgt, i would have to try out to map the rbd locally to the iSCSI servers and provide the targets via LIO ( > which is slower and more complex from the solution, because i have to map the rbd locally ). I try to avoid that. > > In that case, maybe it would better to switch to NFS, but that can not be used in high availbility style without a lot of complexity of the > configuration ( loadbalancer in front of everything ). > > Changing the whole cluster to SuSE because of that is also no option. > > -- > Mit freundlichen Gruessen / Best regards > > Oliver Dzombic > IP-Interactive > > mailto:info@xxxxxxxxxxxxxxxxx > > Anschrift: > > IP Interactive UG ( haftungsbeschraenkt ) Zum Sonnenberg 1-3 > 63571 Gelnhausen > > HRB 93402 beim Amtsgericht Hanau > Geschäftsführung: Oliver Dzombic > > Steuer Nr.: 35 236 3622 1 > UST ID: DE274086107 > > > Am 02.09.2016 um 01:28 schrieb Brad Hubbard: > > On Fri, Sep 2, 2016 at 7:41 AM, Oliver Dzombic <info@xxxxxxxxxxxxxxxxx> wrote: > >> Hi, > >> > >> i know, this is not really ceph related anymore. But i guess it could > >> be helpful for others too. > >> > >> I was using: > >> > >> https://ceph.com/dev-notes/adding-support-for-rbd-to-stgt/ > >> > >> and i am currently running in a problem, where > >> > >> ONE LUN > >> > >> is connected to > >> > >> TWO Nodes ( esxi 6.0 ) > > > > What filesystem are you using on the LUN? > > > >> > >> And the 2nd node is unable to do any kind of write operations on the > >> (successful mounted, and readable ) lun. > > > > Depending on the filesystem you just corrupted it by mounting it > > concurrently on two hosts. > > > >> > >> As it seems, it has to do with reservations. > >> > >> So the question is now, how to solve that. > >> > >> The vmware log says: > >> > >> 2016-09-01T21:09:54.281Z cpu18:33538)NMP: nmp_PathDetermineFailure:3002: > >> SCSI cmd RESERVE failed on path vmhba37:C0:T0:L1, reservation state > >> on device naa.60000000000000000e00000000010001 is unknown. > >> > >> > >> tgtd --version > >> 1.0.55 > >> > >> Any help / idea is appriciated ! > >> > >> Thank you ! > >> > >> -- > >> Mit freundlichen Gruessen / Best regards > >> > >> Oliver Dzombic > >> IP-Interactive > >> > >> mailto:info@xxxxxxxxxxxxxxxxx > >> > >> Anschrift: > >> > >> IP Interactive UG ( haftungsbeschraenkt ) Zum Sonnenberg 1-3 > >> 63571 Gelnhausen > >> > >> HRB 93402 beim Amtsgericht Hanau > >> Geschäftsführung: Oliver Dzombic > >> > >> Steuer Nr.: 35 236 3622 1 > >> UST ID: DE274086107 > >> > >> _______________________________________________ > >> ceph-users mailing list > >> ceph-users@xxxxxxxxxxxxxx > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |