Hello
found this old thread after troubleshooting missing metrics on-and-off
for a week!
is there a way to provide these targets to prometheus using some
temporary workaround ?
Having metrics would really be nice.
best regards
Ronny Aasen
On 25.01.2024 16:45, David C. wrote:
It would be cool, actually, to have the metrics working in 18.2.2, for IPv6
only
Otherwise, everything works fine on my side.
________________________________________________________
Cordialement,
*David CASIER*
________________________________________________________
Le jeu. 25 janv. 2024 à 16:12, Nicolas FOURNIL <nicolas.fournil@xxxxxxxxx>
a écrit :
Gotcha !
I've got the point, after restarting the CA certificate creation with :
ceph restful create-self-signed-cert
I get this error :
Module 'cephadm' has failed: Expected 4 octets in
'fd30:xxxx:xxxx:0:1101:2:0:501'
*Ouch 4 octets = IP4 address expected... some nice code in perspective.*
I go through podman to get more traces :
File "/usr/share/ceph/mgr/cephadm/ssl_cert_utils.py", line 49, in
generate_root_cert
[x509.IPAddress(ipaddress.IPv4Address(addr))]
File "/lib64/python3.6/ipaddress.py", line 1284, in __init__
self._ip = self._ip_int_from_string(addr_str)
File "/lib64/python3.6/ipaddress.py", line 1118, in _ip_int_from_string
raise AddressValueError("Expected 4 octets in %r" % ip_str)
ipaddress.AddressValueError: Expected 4 octets in
'fd30:xxxx:xxxx:0:1101:2:0:501'
So I github this and find this fix in 19.0.0 (with backport not yet
released) :
https://github.com/ceph/ceph/commit/647b5d67a8a800091acea68d20e87354373b0fac
This example shows that it's impossible to get any metrics in an IPv6 only
network (Discovery is impossible) and it's visible at install so there's no
test for IPv6 only environnement before release ?
Now I'm seriously asking myself to put a crappy IPv4 subnet only for my
ceph cluster, because it's always a headache to get it working in an IPv6
environment.
Le mar. 23 janv. 2024 à 17:58, David C. <david.casier@xxxxxxxx> a écrit :
According to sources, the certificates are generated automatically at
startup. Hence my question if the service started correctly.
I also had problems with IPv6 only, but I don't immediately have more info
________________________________________________________
Cordialement,
*David CASIER*
________________________________________________________
Le mar. 23 janv. 2024 à 17:46, Nicolas FOURNIL <nicolas.fournil@xxxxxxxxx>
a écrit :
IPv6 only : Yes, the -ms_bind_ipv6=true is already set-
I had tried a rotation of the keys for node-exporter and I get this :
2024-01-23T16:43:56.098796+0000 mgr.srv06-r2b-fl1.foxykh (mgr.342408)
87074 : cephadm [INF] Rotating authentication key for
node-exporter.srv06-r2b-fl1
2024-01-23T16:43:56.099224+0000 mgr.srv06-r2b-fl1.foxykh (mgr.342408)
87075 : cephadm [ERR] unknown daemon type node-exporter
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1039, in
_check_daemons
self.mgr._daemon_action(daemon_spec, action=action)
File "/usr/share/ceph/mgr/cephadm/module.py", line 2203, in
_daemon_action
return self._rotate_daemon_key(daemon_spec)
File "/usr/share/ceph/mgr/cephadm/module.py", line 2147, in
_rotate_daemon_key
'entity': daemon_spec.entity_name(),
File "/usr/share/ceph/mgr/cephadm/services/cephadmservice.py", line
108, in entity_name
return get_auth_entity(self.daemon_type, self.daemon_id,
host=self.host)
File "/usr/share/ceph/mgr/cephadm/services/cephadmservice.py", line
47, in get_auth_entity
raise OrchestratorError(f"unknown daemon type {daemon_type}")
orchestrator._interface.OrchestratorError: unknown daemon type
node-exporter
Tried to remove & recreate service : it's the same ... how to stop the
rotation now :-/
Le mar. 23 janv. 2024 à 17:18, David C. <david.casier@xxxxxxxx> a
écrit :
Is the cephadm http server service starting correctly (in the mgr logs)?
IPv6 ?
________________________________________________________
Cordialement,
*David CASIER*
________________________________________________________
Le mar. 23 janv. 2024 à 16:29, Nicolas FOURNIL <
nicolas.fournil@xxxxxxxxx> a écrit :
Hello,
Thanks for advice but Prometheus cert is ok, (Self signed) and tested
with curl and web navigator.
it seems to be the "Service discovery" certificate from cephadm who
is missing but I cannot figure out how to set it.
There's in the code a function to create this certificate inside the
Key store but how ... that's the point :-(
Regards.
Le mar. 23 janv. 2024 à 15:52, David C. <david.casier@xxxxxxxx> a
écrit :
Hello Nicolas,
I don't know if it's an update issue.
If this is not a problem for you, you can consider redeploying
grafana/prometheus.
It is also possible to inject your own certificates :
https://docs.ceph.com/en/latest/cephadm/services/monitoring/#example
https://github.com/ceph/ceph/blob/main/src/pybind/mgr/cephadm/templates/services/prometheus/prometheus.yml.j2
________________________________________________________
Cordialement,
*David CASIER*
________________________________________________________
Le mar. 23 janv. 2024 à 10:56, Nicolas FOURNIL <
nicolas.fournil@xxxxxxxxx> a écrit :
Hello,
I've just fresh upgrade from Quincy to Reef and my graphs are now
blank...
after investigations, it seems that discovery service is not working
because of no certificate :
# ceph orch sd dump cert
Error EINVAL: No certificate found for service discovery
Maybe an upgrade issue ?
Is there a way to generate or replace the certificate properly ?
Regards
Nicolas F.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
