Re: Grafana dashboards is missing data

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Those two daemons are handled differently by cephadm, they're different classes (grafana is "class Monitoring(ContainerDaemonForm)" while ceph-exporter is "class CephExporter(ContainerDaemonForm)"), therefore they have different metadata etc., for example:

soc9-ceph:~ # jq '.ports' /var/lib/ceph/{FSID}/ceph-exporter.soc9-ceph/unit.meta
[]

soc9-ceph:~ # jq '.ports' /var/lib/ceph/{FSID}/grafana.soc9-ceph/unit.meta
[
  3000
]

But that's about all I can provide here. Maybe the host OS plays some role here as well, not sure.

Zitat von Sake Ceph <ceph@xxxxxxxxxxx>:

We're using default :) I'm talking about the deployment configuration which is shown in the log files when deploying grafana/ceph-exporter.

I got the same configuration as you for ceph-exporter (the default) when exporting the service.

Kind regards,
Sake

Op 09-09-2024 12:04 CEST schreef Eugen Block <eblock@xxxxxx>:


Can you be more specific about "deploy configuration"? Do you have
your own spec files for grafana and ceph-exporter?
I just ran 'ceph orch apply ceph-exporter' and the resulting config is
this one:

# ceph orch ls ceph-exporter --export
service_type: ceph-exporter
service_name: ceph-exporter
placement:
   host_pattern: '*'
spec:
   prio_limit: 5
   stats_period: 5

Zitat von Sake Ceph <ceph@xxxxxxxxxxx>:

> Hello Eugen,
>
> Well nothing about enabling port 9926.
>
> For example I see the following when deploying Grafanan:
> 2024-09-05 14:27:30,969 7fd2e6583740 INFO firewalld ready
> 2024-09-05 14:27:31,334 7fd2e6583740 DEBUG /bin/firewall-cmd: stdout success
> 2024-09-05 14:27:31,350 7fd2e6583740 INFO firewalld ready
> 2024-09-05 14:27:31,593 7fd2e6583740 DEBUG Non-zero exit code 1 from
> /bin/firewall-cmd --permanent --query-port 3000/tcp
> 2024-09-05 14:27:31,594 7fd2e6583740 DEBUG /bin/firewall-cmd: stdout no
> 2024-09-05 14:27:31,594 7fd2e6583740 INFO Enabling firewalld port
> 3000/tcp in current zone...
> 2024-09-05 14:27:31,832 7fd2e6583740 DEBUG /bin/firewall-cmd: stdout success > 2024-09-05 14:27:32,212 7fd2e6583740 DEBUG /bin/firewall-cmd: stdout success
>
> But only the following when deploying ceph-exporter:
> 2024-09-05 12:17:48,897 7f3d7cc0e740 INFO firewalld ready
> 2024-09-05 12:17:49,269 7f3d7cc0e740 DEBUG /bin/firewall-cmd: stdout success
>
> When looking in the deploy configuration, Grafana shows 'ports':
> [3000], but ceph-exporter shows 'ports': []
>
> Kind regards,
> Sake
>
>> Op 09-09-2024 10:50 CEST schreef Eugen Block <eblock@xxxxxx>:
>>
>>
>> Sorry, clicked "send" too soon. In a test cluster, cephadm.log shows
>> that it would try to open a port if a firewall was enabled:
>>
>> 2024-09-09 10:48:59,686 7f3142b11740 DEBUG firewalld.service is not enabled
>> 2024-09-09 10:48:59,686 7f3142b11740 DEBUG Not possible to enable
>> service <ceph-exporter>. firewalld.service is not available
>>
>> Zitat von Eugen Block <eblock@xxxxxx>:
>>
>> > Do you see anything in the cephadm.log related to the firewall?
>> >
>> > Zitat von Sake Ceph <ceph@xxxxxxxxxxx>:
>> >
>> >> After opening port 9926 manually, the Grafana dashboards show the data.
>> >> So is this a bug?
>> >>
>> >> Kind regards,
>> >> Sake
>> >>> Op 06-09-2024 17:39 CEST schreef Sake Ceph <ceph@xxxxxxxxxxx>:
>> >>>
>> >>>
>> >>> That is working, but I noticed the firewall isn't opened for that
>> >>> port. Shouldn't cephadm manage this, like it does for all the
>> >>> other ports?
>> >>>
>> >>> Kind regards,
>> >>> Sake
>> >>>
>> >>>> Op 06-09-2024 16:14 CEST schreef Björn Lässig
>> <b.laessig@xxxxxxxxxxxxxx>:
>> >>>>
>> >>>>
>> >>>> Am Mittwoch, dem 04.09.2024 um 20:01 +0200 schrieb Sake Ceph:
>> >>>> > After the upgrade from 17.2.7 to 18.2.4 a lot of graphs are
>> empty. For
>> >>>> > example the Osd latency under OSD device details or the Osd Overview
>> >>>> > has a lot of No data messages.
>> >>>> >
>> >>>>
>> >>>> is the ceph-exporter listening on port 9926 (on every host)?
>> >>>>
>> >>>>   ss -tlpn sport 9926
>> >>>>
>> >>>> Can you connect via browser?
>> >>>>
>> >>>>   curl localhost:9926/metrics
>> >>>>
>> >>>> > I deployed ceph-exporter on all hosts, am I missing something? Did
>> >>>> > even a redeploy of prometheus.
>> >>>>
>> >>>> there is a bug, that this exporter does not listens for IPv6.
>> >>>>
>> >>>> greetings
>> >>>> Björn
>> >>>> _______________________________________________
>> >>>> ceph-users mailing list -- ceph-users@xxxxxxx
>> >>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>> >>> _______________________________________________
>> >>> ceph-users mailing list -- ceph-users@xxxxxxx
>> >>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>> >> _______________________________________________
>> >> ceph-users mailing list -- ceph-users@xxxxxxx
>> >> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>>
>>
>> _______________________________________________
>> ceph-users mailing list -- ceph-users@xxxxxxx
>> To unsubscribe send an email to ceph-users-leave@xxxxxxx


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux