In my case I'm adding a label that is unique to each ceph cluster and
then can filter on that. In my ceph dashboard in grafana I've added a
pull-down list to check each different ceph cluster.
You need a way for me to configure what labels to filter on so I can
match it up with how I configured the prometheus. Alternately publish
all metrics to prometheus with fsid label then you can auto-filter
based on the fsid of the ceph cluster since fsid is unique.
On 2023-11-02 01:03, Nizamudeen A wrote:
We have 4 ceph clusters going into the same prometheus instance.
Just curious, In the prometheus, if you want to see the details for
a single cluster, how's it done through query?
For reference, these are the queries that we are currently using now.
USEDCAPACITY = 'ceph_cluster_total_used_bytes',
WRITEIOPS = 'sum(rate(ceph_pool_wr[1m]))',
READIOPS = 'sum(rate(ceph_pool_rd[1m]))',
READLATENCY = 'avg_over_time(ceph_osd_apply_latency_ms[1m])',
WRITELATENCY = 'avg_over_time(ceph_osd_commit_latency_ms[1m])',
READCLIENTTHROUGHPUT = 'sum(rate(ceph_pool_rd_bytes[1m]))',
WRITECLIENTTHROUGHPUT = 'sum(rate(ceph_pool_wr_bytes[1m]))',
RECOVERYBYTES = 'sum(rate(ceph_osd_recovery_bytes[1m]))'
We might not have considered the possibility of multiple
ceph-clusters pointing to a single prometheus instance.
In that case there should be some filtering done with cluster id or
something to properly identify it.
FYI @Pedro Gonzalez Gomez <mailto:pegonzal@xxxxxxxxxx> @Ankush Behl
<mailto:anbehl@xxxxxxxxxx> @Aashish Sharma <mailto:aasharma@xxxxxxxxxx>
Regards,
Nizam
On Mon, Oct 30, 2023 at 11:05 PM Matthew Darwin <bugs@xxxxxxxxxx> wrote:
Ok, so I tried the new ceph dashboard by "set-prometheus-api-host"
(note "host" and not "url") and it returns the wrong data. We
have 4
ceph clusters going into the same prometheus instance. How does it
know which data to pull? Do I need to pass a promql query?
The capacity widget at the top right (not using prometheus)
shows 35%
of 51 TiB used (test cluster data)... This is correct. The chart
shows
use capacity is 1.7 PiB, which is coming from the production
cluster
(incorrect).
Ideas?
On 2023-10-30 11:30, Nizamudeen A wrote:
> Ah yeah, probably that's why the utilization charts are empty
> because it relies on
> the prometheus info.
>
> And I raised a PR to disable the new dashboard in quincy.
> https://github.com/ceph/ceph/pull/54250
>
> Regards,
> Nizam
>
> On Mon, Oct 30, 2023 at 6:09 PM Matthew Darwin
<bugs@xxxxxxxxxx> wrote:
>
> Hello,
>
> We're not using prometheus within ceph (ceph dashboards
show in our
> grafana which is hosted elsewhere). The old dashboard
showed the
> metrics fine, so not sure why in a patch release we would need
> to make
> configuration changes to get the same metrics.... Agree it
> should be
> off by default.
>
> "ceph dashboard feature disable dashboard" works to put
the old
> dashboard back. Thanks.
>
> On 2023-10-30 00:09, Nizamudeen A wrote:
> > Hi Matthew,
> >
> > Is the prometheus configured in the cluster? And also the
> > PROMETHUEUS_API_URL is set? You can set it manually by ceph
> dashboard
> > set-prometheus-api-url <url-of-prom>.
> >
> > You can switch to the old Dashboard by switching the feature
> toggle in the
> > dashboard. `ceph dashboard feature disable dashboard` and
> reloading the
> > page. Probably this should have been disabled by default.
> >
> > Regards,
> > Nizam
> >
> > On Sun, Oct 29, 2023, 23:04 Matthew
Darwin<bugs@xxxxxxxxxx> wrote:
> >
> >> Hi all,
> >>
> >> I see17.2.7 quincy is published as debian-bullseye
packages.
> So I
> >> tried it on a test cluster.
> >>
> >> I must say I was not expecting the big dashboard change
in a
> patch
> >> release. Also all the "cluster utilization" numbers
are all
> blank now
> >> (any way to fix it?), so the dashboard is much less
usable now.
> >>
> >> Thoughts?
> >> _______________________________________________
> >> ceph-users mailing list --ceph-users@xxxxxxx
> >> To unsubscribe send an email toceph-users-leave@xxxxxxx
> >>
> > _______________________________________________
> > ceph-users mailing list --ceph-users@xxxxxxx
> > To unsubscribe send an email toceph-users-leave@xxxxxxx
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
