On Tuesday, September 26, 2023 6:00:23 AM EDT Ackermann, Christoph wrote: > Dear list members,, > > after upgrading to reef (18.2.0) I spent some time with CephFS, NFS & > HA(Ingress). I can confirm that Ingress (count either 1 or 2) works well > IF only ONE backend server is configured. But this is, of course, no HA. ;-) > Two or more backend servers won't work because there isn't ANY "*check*" > directive used in HAproxy backend config. This means any client will > stay/stick on the last NFS backend server used. IMHO in haproxy.cfg this > should look more like the following: > > [..] > backend backend > mode tcp > option tcp-check > tcp-check connect port 2049 > balance source > hash-type consistent > server nfs.nfstest-st.0 10.100.1.111:2049 check > server nfs.nfstest-st.1 10.100.1.112:2049 check > > > BTW: There are plenty of configuration directives to get a proper working > NFS backend. Please refer to: > > https://cbonte.github.io/haproxy-dconv/2.4/configuration.html#5.2-check > > https://cbonte.github.io/haproxy-dconv/2.4/configuration.html#stick-table > > https://infohub.delltechnologies.com/l/ecs-with-haproxy-load-balancer-2/nfs-> configuration-definitions/ > > > Did (NFS) HA ever work in previous versions? Unfortunately, I had no reason > to test this in advance. Currently, only a manual installation of HAproxy > based ingress controller(s) should help. > > And, can we hope for a correction in future versions? FWIW there is a tracker for this specific issue https://tracker.ceph.com/ issues/62638 I don't think that alone will make a full HA solution but it would be an improvement. :-) > > Thanks, > Christoph > > > > Am Mo., 4. Sept. 2023 um 07:43 Uhr schrieb Thorne Lawler <thorne@xxxxxxxxxxx > > One more question, for John or anyone: > > > > It looks like Ganesha NFS wants to use dbus to talk to other services > > (although I'm not sure how well this works across containers). > > > > I just realised that my build (on AlmaLinux 8 'minimal') did not include > > dbus, and Ceph has not installed it. > > > > I have manually installed the default 'dbus' package and its deps, but > > this doesn't seem to have had any effect. > > > > * Does Ganesha use dbus to talk to other services, even in a cephadm > > > > containerised build? > > > > * Is there a non-destructive way to make Ganesha retry gsh_dbus_pkginit > > ? > > * Is the default dbus package sufficient, or does Ceph require > > > > specific dbus plugins? > > > > Thank you. > > > > On 4/09/2023 9:55 am, Thorne Lawler wrote: > > > John, > > > > > > Thanks for getting back to me. I am indeed using cephadm, and I will > > > dig up those configurations. > > > > > > Even if Ceph Quincy is current completely incapable of configuring its > > > own HA failover, I would really like to know what the /imagined/ > > > process would be for detecting a node failure and failing over. > > > > > > Can you elaborate about those changes that need to happen, or point me > > > to the forums (ideally the posts_ where this work is broken down in > > > more detail? > > > > > > It's too late for me to change this; I am 100% committed to using Ceph > > > for HA NFS, no matter what that involves. > > > > > > Thanks. > > > > > > On 31/08/2023 11:18 pm, John Mulligan wrote: > > >> On Wednesday, August 30, 2023 8:38:21 PM EDT Thorne Lawler wrote: > > >>> If there isn't any documentation for this yet, can anyone tell me: > > >>> * How do I inspect/change my NFS/haproxy/keepalived configuration? > > >>> * What is it supposed to look like? Does someone have a working > > >>> > > >>> example? > > >> > > >> The configuration for haproxy, keepalive, and ganesha are generated. > > >> I'm assuming you are using cephadm orchestration. If you want to see > > >> what it > > >> generated configs contain look in /var/lib/ceph/<FSID>/<svc-name>/ > > >> under those dirs may be additional subdirectories like etc/ or > > >> config/ (it > > >> varies from service to service) > > >> > > >> It's not simple to customize those files directly. It's not > > >> impossible but it's > > >> probably not worth it (IMO). > > >> > > >> Also, speaking only for my personal opinion, the current NFS > > >> situation is one > > >> of OK scale-out but is HA in name only. Fail over is not mature. To > > >> make it > > >> so, changes need to happen throughout the stack including in > > >> nfs-ganesha. I > > >> know there are some conversations happening around this topic but I > > >> don't know > > >> the best place to get involved upstream. > > >> > > >> I know this probably isn't very satisfactory, but I hope the > > >> information > > >> helps. > > >> > > >> _______________________________________________ > > >> ceph-users mailing list --ceph-users@xxxxxxx > > >> To unsubscribe send an email toceph-users-leave@xxxxxxx > > > > -- > > > > Regards, > > > > Thorne Lawler - Senior System Administrator > > *DDNS* | ABN 76 088 607 265 > > First registrar certified ISO 27001-2013 Data Security Standard ITGOV40172 > > P +61 499 449 170 > > > > _DDNS > > > > /_*Please note:* The information contained in this email message and any > > attached files may be confidential information, and may also be the > > subject of legal professional privilege. _If you are not the intended > > recipient any use, disclosure or copying of this email is unauthorised. > > _If you received this email in error, please notify Discount Domain Name > > Services Pty Ltd on 03 9815 6868 to report this matter and delete all > > copies of this transmission together with any attachments. / > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |