https://tracker.ceph.com/issues/67583 didn't reproduce across 10 reruns https://pulpito.ceph.com/lflores-2024-08-16_00:04:51-upgrade:quincy-x-squid-release-distro-default-smithi/. Given the original failure was just "Unable to find image ' quay.io/ceph/grafana:9.4.12' locally" which doesn't look very serious anyway, I don't think there's any reason for the failure to hold up the release On Thu, Aug 15, 2024 at 6:53 PM Laura Flores <lflores@xxxxxxxxxx> wrote: > The upgrade suites look mostly good to me, except for one tracker I think > would be in @Adam King <adking@xxxxxxxxxx>'s realm to look at. If the new > grafana issue below is deemed okay, then we can proceed with approving the > upgrade suite. > > *This issue stood out to me, where the cluster had trouble pulling the > grafana image locally to redeploy it. *@Adam King <adking@xxxxxxxxxx>* can > you take a look?* > > - *https://tracker.ceph.com/issues/67583 > <https://tracker.ceph.com/issues/67583> - upgrade:quincy-x/stress-split: > Cluster fails to redeploy grafana daemon after image is unable to be found > locally* > > > Otherwise, tests failed from cluster log warnings that are expected during > upgrade tests. Many of these warnings have already been fixed and are in > the stages of getting backported. > I checked for each test that the cluster had upgraded all daemons to > 19.1.1, and that was the case. > > - https://tracker.ceph.com/issues/66602 - rados/upgrade: Health check > failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) > - https://tracker.ceph.com/issues/65422 - upgrade/quincy-x: "1 pg > degraded (PG_DEGRADED)" in cluster log > - https://tracker.ceph.com/issues/67584 - upgrade:quincy-x: cluster > [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log > - https://tracker.ceph.com/issues/64460 - rados/upgrade: "[WRN] > MON_DOWN: 1/3 mons down, quorum a,b" in cluster log > - https://tracker.ceph.com/issues/66809 - upgrade/quincy-x; > upgrade/reef-x: Health check failed: Reduced data availability: 1 pg > peering (PG_AVAILABILITY)" in cluster log > > > > On Thu, Aug 15, 2024 at 11:55 AM Laura Flores <lflores@xxxxxxxxxx> wrote: > >> Rados approved. Failures tracked here: >> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1911-httpstrackercephcomissues67340 >> >> On Thu, Aug 15, 2024 at 11:30 AM Yuri Weinstein <yweinste@xxxxxxxxxx> >> wrote: >> >>> Laura, >>> >>> The PR was cherry-picked and the `squid-release` branch was built. >>> Please review the run results in the tracker. >>> >>> On Wed, Aug 14, 2024 at 2:18 PM Laura Flores <lflores@xxxxxxxxxx> wrote: >>> > >>> > Hey @Yuri Weinstein <yweinste@xxxxxxxxxx>, >>> > >>> > We've fixed a couple of issues and now need a few things rerun. >>> > >>> > >>> > 1. *Can you please rerun upgrade/reef-x and upgrade/quincy-x? * >>> > - Reasoning: Many jobs in those suites died due to >>> > https://tracker.ceph.com/issues/66883, which we deduced was a >>> recent >>> > merge to teuthology. Now that the affecting commit was reverted, >>> we are >>> > ready to have those rerun. >>> > 2. *Can you please cherry-pick >>> https://github.com/ceph/ceph/pull/58607 >>> > <https://github.com/ceph/ceph/pull/58607> to squid-release and >>> reschedule >>> > rados:thrash-old-clients?* >>> > - Reasoning: Since we stopped building focal for squid, we can no >>> > longer test squid against pacific clients. >>> > - For this second RC, we had to make the decision to drop pacific >>> > from the *rados:thrash-old-clients* tests, which will now use >>> centos >>> > 9 stream packages to test against only reef and quincy clients ( >>> > https://github.com/ceph/ceph/pull/58607). >>> > - We have raised https://tracker.ceph.com/issues/67469 to track >>> the >>> > implementation of a containerized solution for older clients >>> that don't >>> > have centos 9 stream packages, so that we can reincorporate >>> > pacific in the >>> > future. >>> > >>> > After these two things are rescheduled, we can proceed with a rados >>> suite >>> > approval and an upgrade suite approval. >>> > >>> > Thanks, >>> > Laura >>> > >>> > On Wed, Aug 14, 2024 at 12:49 PM Adam Emerson <aemerson@xxxxxxxxxx> >>> wrote: >>> > >>> > > On 14/08/2024, Yuri Weinstein wrote: >>> > > > Still waiting to hear back: >>> > > > >>> > > > rgw - Eric, Adam E >>> > > >>> > > Approved. >>> > > >>> > > (Sorry, I thought we were supposed to reply on the tracker.) >>> > > _______________________________________________ >>> > > Dev mailing list -- dev@xxxxxxx >>> > > To unsubscribe send an email to dev-leave@xxxxxxx >>> > > >>> > > >>> > >>> > -- >>> > >>> > Laura Flores >>> > >>> > She/Her/Hers >>> > >>> > Software Engineer, Ceph Storage <https://ceph.io> >>> > >>> > Chicago, IL >>> > >>> > lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>> > M: +17087388804 >>> > _______________________________________________ >>> > ceph-users mailing list -- ceph-users@xxxxxxx >>> > To unsubscribe send an email to ceph-users-leave@xxxxxxx >>> >>> >> >> -- >> >> Laura Flores >> >> She/Her/Hers >> >> Software Engineer, Ceph Storage <https://ceph.io> >> >> Chicago, IL >> >> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >> M: +17087388804 >> >> >> > > -- > > Laura Flores > > She/Her/Hers > > Software Engineer, Ceph Storage <https://ceph.io> > > Chicago, IL > > lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> > M: +17087388804 > > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |