The gibba cluster upgrade is complete! All looks good. On Thu, Sep 12, 2024 at 9:54 AM Laura Flores <lflores@xxxxxxxxxx> wrote: > Will do, @Rachana Patel <racpatel@xxxxxxxxxx>. > > On Thu, Sep 12, 2024 at 3:44 AM Rachana Patel <racpatel@xxxxxxxxxx> wrote: > >> Thanks Venky ! >> We can now focus on next tasks - >> >> - Release Notes >> https://github.com/ceph/ceph/pull/59539 . Requesting all TLs >> to review Release notes. >> - Gibba upgrade >> - LRC upgrade >> >> @Laura Flores <lflores@xxxxxxx> , Kindly upgrade cluster and share >> results. >> >> On Thu, Sep 12, 2024 at 1:03 AM Venky Shankar <vshankar@xxxxxxxxxx> >> wrote: >> >>> >>> >>> On Wed, Sep 11, 2024 at 10:00 PM Laura Flores <lflores@xxxxxxxxxx> >>> wrote: >>> >>>> Thanks @Venky Shankar <vshankar@xxxxxxxxxx> for looking into >>>> https://tracker.ceph.com/issues/68002. >>>> >>>> As for https://tracker.ceph.com/issues/67999, you are right- it is not >>>> cephfs related. Mistake on my part. After looking into this one, it seems >>>> to be a test issue, where we check for a "Filestore has been deprecated" >>>> warning in all the OSD logs once they've upgraded to Squid. On one of the >>>> OSDs, the warning was missing, but that OSD hadn't yet been upgraded to >>>> Squid. I believe the test needs to be modified to wait until all OSDs have >>>> been upgraded. >>>> >>>> Thus, it should not be a blocker. >>>> >>>> @Rachana Patel <racpatel@xxxxxxxxxx> if Venky + Xiubo confirm that >>>> https://tracker.ceph.com/issues/68002 is not a blocker, we are good to >>>> proceed with next steps. >>>> >>> >>> https://tracker.ceph.com/issues/68002 is NOT a blocker either and is a >>> duplicate of https://tracker.ceph.com/issues/64471. >>> >>> >>>> >>>> On Wed, Sep 11, 2024 at 4:46 AM Venky Shankar <vshankar@xxxxxxxxxx> >>>> wrote: >>>> >>>>> Hey Laura, >>>>> >>>>> On Wed, Sep 11, 2024 at 12:13 AM Laura Flores <lflores@xxxxxxxxxx> >>>>> wrote: >>>>> >>>>>> I have finished reviewing the upgrade and smoke suites. Most failures >>>>>> are already known/tracked: >>>>>> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstrackercephcomissues67779 >>>>>> >>>>>> *Upgrade: Pending check from FS team* >>>>>> Two fs issues stood out. @Gregory Farnum <gfarnum@xxxxxxxxxx> @Venky >>>>>> Shankar <vshankar@xxxxxxxxxx> can you confirm these are not blockers? >>>>>> >>>>>> - https://tracker.ceph.com/issues/67999 - >>>>>> upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has >>>>>> been deprecated and is no longer supported" is absent from osd log >>>>>> >>>>>> >>>>> How is this cephfs related? >>>>> >>>>> >>>>>> >>>>>> - >>>>>> - https://tracker.ceph.com/issues/68002 - >>>>>> upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps >>>>>> >>>>>> >>>>> I left my findings in the tracker. See >>>>> >>>>> https://tracker.ceph.com/issues/68002#note-2 >>>>> >>>>> The fsstress crash is due to SIGBUS and there are a flurry of OSD >>>>> related heartbeat_check messages from a node where there is a cephfs kernel >>>>> driver traceback in the kernel ring buffer. I am running the tracker >>>>> through Xiubo. >>>>> >>>>> >>>>>> >>>>>> - >>>>>> >>>>>> When this is confirmed, the upgrade suite can be approved. >>>>>> >>>>>> *Smoke: APPROVED* >>>>>> >>>>>> *Rados: Pending check from core team* >>>>>> >>>>>> >>>>>> On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@xxxxxxxxxx> >>>>>> wrote: >>>>>> >>>>>>> Rados reviewed here: >>>>>>> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstrackercephcomissues67779 >>>>>>> >>>>>>> I have asked @Radoslaw Zarzynski <rzarzyns@xxxxxxxxxx> to take a >>>>>>> look at the summary and confirm there are no blockers. >>>>>>> >>>>>>> In progress reviewing upgrade and smoke; will provide another update >>>>>>> for those soon as well. >>>>>>> >>>>>>> On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@xxxxxxxxxx> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey Rachana, >>>>>>>> >>>>>>>> Thanks for these links. Currently reviewing them and will update >>>>>>>> the thread soon. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Laura >>>>>>>> >>>>>>>> On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@xxxxxxxxxx> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi All ! >>>>>>>>> PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked >>>>>>>>> on top of the last built. RADOS, upgrade and smoke test suits were >>>>>>>>> executed. >>>>>>>>> >>>>>>>>> Seeking approvals/reviews for following reruns >>>>>>>>> >>>>>>>>> RADOS Test Result >>>>>>>>> <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- >>>>>>>>> Neha >>>>>>>>> Upgrade Test result >>>>>>>>> <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- >>>>>>>>> Neha, Laura >>>>>>>>> Smoke Test Result >>>>>>>>> <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@xxxxxxxxxx> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> We have added one more core PR to the release, which was deemed a >>>>>>>>>> blocker ( >>>>>>>>>> https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and >>>>>>>>>> smoke will >>>>>>>>>> need to be reapproved. >>>>>>>>>> >>>>>>>>>> On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@xxxxxxxxxx> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> > Upgrade suites approved: >>>>>>>>>> > >>>>>>>>>> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcomissues67779 >>>>>>>>>> > >>>>>>>>>> > On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@xxxxxxxxxx> >>>>>>>>>> wrote: >>>>>>>>>> > >>>>>>>>>> >> orch approved, failures are known issues and not release >>>>>>>>>> blockers >>>>>>>>>> >> >>>>>>>>>> >> On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein < >>>>>>>>>> yweinste@xxxxxxxxxx> >>>>>>>>>> >> wrote: >>>>>>>>>> >> >>>>>>>>>> >> > Details of this release are summarized here: >>>>>>>>>> >> > >>>>>>>>>> >> > https://tracker.ceph.com/issues/67779#note-1 >>>>>>>>>> >> > >>>>>>>>>> >> > Release Notes - TBD >>>>>>>>>> >> > Gibba upgrade -TBD >>>>>>>>>> >> > LRC upgrade - TBD >>>>>>>>>> >> > >>>>>>>>>> >> > It was decided and agreed upon that there would be limited >>>>>>>>>> testing for >>>>>>>>>> >> > this release, given it is based on 19.1.1 rather than a full >>>>>>>>>> rebase. >>>>>>>>>> >> > >>>>>>>>>> >> > Seeking approvals/reviews for: >>>>>>>>>> >> > (some reruns are still in progress) >>>>>>>>>> >> > >>>>>>>>>> >> > rgw - Eric, Adam E >>>>>>>>>> >> > fs - Venky >>>>>>>>>> >> > orch - Adam King >>>>>>>>>> >> > >>>>>>>>>> >> > quincy-x - Laura, Neha >>>>>>>>>> >> > reef-x - Laura, Neha >>>>>>>>>> >> > >>>>>>>>>> >> > Pls let me know if any tests were missed from this list. >>>>>>>>>> >> > _______________________________________________ >>>>>>>>>> >> > ceph-users mailing list -- ceph-users@xxxxxxx >>>>>>>>>> >> > To unsubscribe send an email to ceph-users-leave@xxxxxxx >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> _______________________________________________ >>>>>>>>>> >> ceph-users mailing list -- ceph-users@xxxxxxx >>>>>>>>>> >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >>>>>>>>>> >> >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > -- >>>>>>>>>> > >>>>>>>>>> > Laura Flores >>>>>>>>>> > >>>>>>>>>> > She/Her/Hers >>>>>>>>>> > >>>>>>>>>> > Software Engineer, Ceph Storage <https://ceph.io> >>>>>>>>>> > >>>>>>>>>> > Chicago, IL >>>>>>>>>> > >>>>>>>>>> > lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>>>>>>>>> > M: +17087388804 >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Laura Flores >>>>>>>>>> >>>>>>>>>> She/Her/Hers >>>>>>>>>> >>>>>>>>>> Software Engineer, Ceph Storage <https://ceph.io> >>>>>>>>>> >>>>>>>>>> Chicago, IL >>>>>>>>>> >>>>>>>>>> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>>>>>>>>> M: +17087388804 >>>>>>>>>> _______________________________________________ >>>>>>>>>> ceph-users mailing list -- ceph-users@xxxxxxx >>>>>>>>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Rachana >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Laura Flores >>>>>>>> >>>>>>>> She/Her/Hers >>>>>>>> >>>>>>>> Software Engineer, Ceph Storage <https://ceph.io> >>>>>>>> >>>>>>>> Chicago, IL >>>>>>>> >>>>>>>> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>>>>>>> M: +17087388804 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Laura Flores >>>>>>> >>>>>>> She/Her/Hers >>>>>>> >>>>>>> Software Engineer, Ceph Storage <https://ceph.io> >>>>>>> >>>>>>> Chicago, IL >>>>>>> >>>>>>> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>>>>>> M: +17087388804 >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Laura Flores >>>>>> >>>>>> She/Her/Hers >>>>>> >>>>>> Software Engineer, Ceph Storage <https://ceph.io> >>>>>> >>>>>> Chicago, IL >>>>>> >>>>>> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>>>>> M: +17087388804 >>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Cheers, >>>>> Venky >>>>> >>>> >>>> >>>> -- >>>> >>>> Laura Flores >>>> >>>> She/Her/Hers >>>> >>>> Software Engineer, Ceph Storage <https://ceph.io> >>>> >>>> Chicago, IL >>>> >>>> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>>> M: +17087388804 >>>> >>>> >>>> >>> >>> -- >>> Cheers, >>> Venky >>> >> >> >> -- >> >> Thanks, >> Rachana >> > > > -- > > Laura Flores > > She/Her/Hers > > Software Engineer, Ceph Storage <https://ceph.io> > > Chicago, IL > > lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> > M: +17087388804 > > > -- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |