Hi Xiubo, Thx for pointing me into the right direction. All involved esx host seem to use the correct policy. I am going to detach the LUN on each host one by one until i found the host causing the problem. Regards Felix --------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Volker Rieke Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Dr. Astrid Lambrecht, Prof. Dr. Frauke Melchior --------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------- Am 12.12.2022 um 13:03 schrieb Xiubo Li <xiubli@xxxxxxxxxx>: Hi Stolte, For the VMware config could you refer to : https://docs.ceph.com/en/latest/rbd/iscsi-initiator-esx/ ? What's the "Path Selection Policy with ALUA" you are using ? The ceph-iscsi couldn't implement the real AA, so if you use the RR I think it will be like this. - Xiubo On 12/12/2022 17:45, Stolte, Felix wrote: Hi guys, we are using ceph-iscsi to provide block storage for Microsoft Exchange and vmware vsphere. Ceph docs state that you need to configure Windows iSCSI Initatior for fail-over-only but there is no such point for vmware. In my tcmu-runner logs on both ceph-iscsi gateways I see the following: 2022-12-12 10:36:06.978 33789 [WARN] tcmu_notify_lock_lost:222 rbd/mailbox.vmdk_junet_sata: Async lock drop. Old state 1 2022-12-12 10:36:06.993 33789 [INFO] alua_implicit_transition:570 rbd/mailbox.vmdk_junet_sata: Starting lock acquisition operation. 2022-12-12 10:36:08.064 33789 [WARN] tcmu_rbd_lock:762 rbd/mailbox.vmdk_junet_sata: Acquired exclusive lock. 2022-12-12 10:36:09.067 33789 [WARN] tcmu_notify_lock_lost:222 rbd/mailbox.vmdk_junet_sata: Async lock drop. Old state 1 2022-12-12 10:36:09.071 33789 [INFO] alua_implicit_transition:570 rbd/mailbox.vmdk_junet_sata: Starting lock acquisition operation. 2022-12-12 10:36:10.109 33789 [WARN] tcmu_rbd_lock:762 rbd/mailbox.vmdk_junet_sata: Acquired exclusive lock. 2022-12-12 10:36:11.104 33789 [WARN] tcmu_notify_lock_lost:222 rbd/mailbox.vmdk_junet_sata: Async lock drop. Old state 1 2022-12-12 10:36:11.106 33789 [INFO] alua_implicit_transition:570 rbd/mailbox.vmdk_junet_sata: Starting lock acquisition operation. At the same time there are these log entries in ceph.audit.logs: 2022-12-12T10:36:06.731621+0100 mon.mon-k2-1 (mon.1) 3407851 : audit [INF] from='client.? 10.100.8.55:0/2392201639' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "10 .100.8.56:0/1598475844"}]: dispatch 2022-12-12T10:36:06.731913+0100 mon.mon-e2-1 (mon.0) 783726 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "10.100.8.56:0/1598475844"}] : dispatch 2022-12-12T10:36:06.905082+0100 mon.mon-e2-1 (mon.0) 783727 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "10.100.8.56:0/1598475844"} ]': finished Can someone explaint to me, what is happening? Why are the gateways blacklisting each other? All involved daemons are running Version 16.2.10. ceph-iscsi gateways are running on Ubuntu 20.04 with ceph-isci package from the Ubuntu repo (all other packers came directly from ceph.com <http://ceph.com/><http://ceph.com/>) regards Felix --------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Volker Rieke Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Dr. Astrid Lambrecht, Prof. Dr. Frauke Melchior --------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------- _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> To unsubscribe send an email to ceph-users-leave@xxxxxxx<mailto:ceph-users-leave@xxxxxxx> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |