Thanks Christoph!
I had just been assuming that the connectivity check was elsewhere, or
was implicit in some way.
I have certainly not seen any evidence of Quincy trying to move the IP
address when the node fails.
On 26/09/2023 8:00 pm, Ackermann, Christoph wrote:
Dear list members,,
after upgrading to reef (18.2.0) I spent some time with CephFS, NFS &
HA(Ingress). I can confirm that Ingress (count either 1 or 2) works
well IF only ONE backend server is configured. But this is, of course,
no HA. ;-)
Two or more backend servers won't work because there isn't ANY
"*check*" directive used in HAproxy backend config. This means any
client will stay/stick on the last NFS backend server used. IMHO in
haproxy.cfg this should look more like the following:
[..]
backend backend
mode tcp
option tcp-check
tcp-check connect port 2049
balance source
hash-type consistent
server nfs.nfstest-st.0 10.100.1.111:2049
<http://10.100.1.111:2049> check
server nfs.nfstest-st.1 10.100.1.112:2049
<http://10.100.1.112:2049> check
BTW: There are plenty of configuration directives to get a proper
working NFS backend. Please refer to:
https://cbonte.github.io/haproxy-dconv/2.4/configuration.html#5.2-check
https://cbonte.github.io/haproxy-dconv/2.4/configuration.html#stick-table
https://infohub.delltechnologies.com/l/ecs-with-haproxy-load-balancer-2/nfs-configuration-definitions/
Did (NFS) HA ever work in previous versions? Unfortunately, I had no
reason to test this in advance. Currently, only a manual installation
of HAproxybased ingress controller(s) should help.
And, can we hope for a correction in future versions?
Thanks,
Christoph
Am Mo., 4. Sept. 2023 um 07:43 Uhr schrieb Thorne Lawler
<thorne@xxxxxxxxxxx>:
One more question, for John or anyone:
It looks like Ganesha NFS wants to use dbus to talk to other services
(although I'm not sure how well this works across containers).
I just realised that my build (on AlmaLinux 8 'minimal') did not
include
dbus, and Ceph has not installed it.
I have manually installed the default 'dbus' package and its deps,
but
this doesn't seem to have had any effect.
* Does Ganesha use dbus to talk to other services, even in a cephadm
containerised build?
* Is there a non-destructive way to make Ganesha retry
gsh_dbus_pkginit ?
* Is the default dbus package sufficient, or does Ceph require
specific dbus plugins?
Thank you.
On 4/09/2023 9:55 am, Thorne Lawler wrote:
> John,
>
> Thanks for getting back to me. I am indeed using cephadm, and I
will
> dig up those configurations.
>
> Even if Ceph Quincy is current completely incapable of
configuring its
> own HA failover, I would really like to know what the /imagined/
> process would be for detecting a node failure and failing over.
>
> Can you elaborate about those changes that need to happen, or
point me
> to the forums (ideally the posts_ where this work is broken down in
> more detail?
>
> It's too late for me to change this; I am 100% committed to
using Ceph
> for HA NFS, no matter what that involves.
>
> Thanks.
>
> On 31/08/2023 11:18 pm, John Mulligan wrote:
>> On Wednesday, August 30, 2023 8:38:21 PM EDT Thorne Lawler wrote:
>>> If there isn't any documentation for this yet, can anyone tell me:
>>>
>>> * How do I inspect/change my NFS/haproxy/keepalived
configuration?
>>> * What is it supposed to look like? Does someone have a
working
>>> example?
>> The configuration for haproxy, keepalive, and ganesha are
generated.
>> I'm assuming you are using cephadm orchestration. If you want
to see
>> what it
>> generated configs contain look in /var/lib/ceph/<FSID>/<svc-name>/
>> under those dirs may be additional subdirectories like etc/ or
>> config/ (it
>> varies from service to service)
>>
>> It's not simple to customize those files directly. It's not
>> impossible but it's
>> probably not worth it (IMO).
>>
>> Also, speaking only for my personal opinion, the current NFS
>> situation is one
>> of OK scale-out but is HA in name only. Fail over is not
mature. To
>> make it
>> so, changes need to happen throughout the stack including in
>> nfs-ganesha. I
>> know there are some conversations happening around this topic
but I
>> don't know
>> the best place to get involved upstream.
>>
>> I know this probably isn't very satisfactory, but I hope the
information
>> helps.
>>
>> _______________________________________________
>> ceph-users mailing list --ceph-users@xxxxxxx
>> To unsubscribe send an email toceph-users-leave@xxxxxxx
--
Regards,
Thorne Lawler - Senior System Administrator
*DDNS* | ABN 76 088 607 265
First registrar certified ISO 27001-2013 Data Security Standard
ITGOV40172
P +61 499 449 170
_DDNS
/_*Please note:* The information contained in this email message
and any
attached files may be confidential information, and may also be the
subject of legal professional privilege. _If you are not the intended
recipient any use, disclosure or copying of this email is
unauthorised.
_If you received this email in error, please notify Discount
Domain Name
Services Pty Ltd on 03 9815 6868 to report this matter and delete all
copies of this transmission together with any attachments. /
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
--
Regards,
Thorne Lawler - Senior System Administrator
*DDNS* | ABN 76 088 607 265
First registrar certified ISO 27001-2013 Data Security Standard ITGOV40172
P +61 499 449 170
_DDNS
/_*Please note:* The information contained in this email message and any
attached files may be confidential information, and may also be the
subject of legal professional privilege. _If you are not the intended
recipient any use, disclosure or copying of this email is unauthorised.
_If you received this email in error, please notify Discount Domain Name
Services Pty Ltd on 03 9815 6868 to report this matter and delete all
copies of this transmission together with any attachments. /
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
