On Thu, Sep 22, 2022 at 11:26:29AM -0500, michael.christie@xxxxxxxxxx wrote: > > On 9/15/22 1:08 AM, Dmitry Bogdanov wrote: > > On Wed, Sep 14, 2022 at 02:18:40PM -0500, Mike Christie wrote: > >> > >> On 9/12/22 4:45 PM, Dmitry Bogdanov wrote: > >>>> Remember how ESX used to send a RTPG to one port and expect that it got > >>>> every group and that the state info was all in sync (basically opposite > >>>> if what's in the spec now)? > >>>> > >>>> The spec and ESX were updated, but I don't know if other OSs did this and > >>>> if/when everyone was updated. Do you know this info? Are the old ESX versions > >>>> that worked like that end of life? > >>> ESXi is kinda a pain. But fortunately it has nothing to do with that > >>> patch 😄 > >> > >> I didn't get why that is. How do you set up a distributed/cluster/HA target? I'm > >> probably missing that part. > >> > >> Software drivers like iscsi I get, but for HW drivers I didn't see how you do it. > >> > >> For example, if you have 2 systems/nodes running LIO which each export the same > >> device via 1 port each where one is active/optimized and the other is standby and you > >> are using qla2xxx, then on the local node would you create 2 groups: > >> > >> [root@ol8n4 alua]# pwd > >> /sys/kernel/config/target/core/iblock_0/device0/alua > >> > >> [root@ol8n4 alua]# ls > >> default_tg_pt_gp local remote > >> > >> Then under the mapped lun: > >> > >> [root@ol8n4 lun_0]# pwd > >> /sys/kernel/config/target/..../tpgt_1/lun/lun_0 > >> > >> in the alua_tg_pt_gp file you set that to local. That would then have tg_pt_gp_members > >> set, but remote would not. > >> > >> Before your patch, windows and ESX could do a RTPG to just one port/path and we would > >> return the default, local and remote groups. We don't want the default group, but we > >> wanted the local and remote one returned. With your patch we only return the the local > >> one now. I wasn't sure how that works for drivers like qla2xxx. > >> > >> For iscsi, you could just mirror the remote node locally, so you would have a second > >> tpg: > >> > >> [root@ol8n4 lun_0]# pwd > >> /sys/kernel/config/target/..../tpgt_2/lun/lun_0 > >> > > >> and in there set alua_tg_pt_gp to remote. Your patch works fine for that because both > >> groups then have tg_pt_gp_members set so if the OS just does a RTPG to one path/port > >> you get all the groups. > >> > > I use a virtual remote fabric driver to configure wwn/iqn-tpg-lun of > > remote peers at each local node. In that way 'remote' alua port group > > will have ports(RTPI) too. That allows RTPG (and other discovery-like > > commands) report all ports in all port groups in the cluster. > > I sent it within the RFC patchset: > > Ok shoot. I think this type of setup is not common, so the patch should be ok for most > users. However, I know people did do some complex setups and I'm worried those might > break. I am not sure that any kind of cluster configuration is available now. Because RTPI (without my patchset) is not configurable and normally it is the same on the ports(LUNs now) on different nodes. That prevents a correct reporting of ports in RTPG. > > I think the remote target patch is fine. Did that require any additional patches? Remote target patch is self-sufficient but it is not useful alone I think. Together with RTPI patchset it allows to report in RTPG all ports in a cluster. > Maybe we could add that patch and your patch in this email at the same time and we > could migrate users. > > What's your take? I will send the remote target patch soon, but as I said, it is useful only together with RTPI patchset (published but not yet merged to scsi-queue). BR, Dmitry
