Thanks @Venky Shankar <vshankar@xxxxxxxxxx> for looking into https://tracker.ceph.com/issues/68002. As for https://tracker.ceph.com/issues/67999, you are right- it is not cephfs related. Mistake on my part. After looking into this one, it seems to be a test issue, where we check for a "Filestore has been deprecated" warning in all the OSD logs once they've upgraded to Squid. On one of the OSDs, the warning was missing, but that OSD hadn't yet been upgraded to Squid. I believe the test needs to be modified to wait until all OSDs have been upgraded. Thus, it should not be a blocker. @Rachana Patel <racpatel@xxxxxxxxxx> if Venky + Xiubo confirm that https://tracker.ceph.com/issues/68002 is not a blocker, we are good to proceed with next steps. On Wed, Sep 11, 2024 at 4:46 AM Venky Shankar <vshankar@xxxxxxxxxx> wrote: > Hey Laura, > > On Wed, Sep 11, 2024 at 12:13 AM Laura Flores <lflores@xxxxxxxxxx> wrote: > >> I have finished reviewing the upgrade and smoke suites. Most failures are >> already known/tracked: >> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstrackercephcomissues67779 >> >> *Upgrade: Pending check from FS team* >> Two fs issues stood out. @Gregory Farnum <gfarnum@xxxxxxxxxx> @Venky >> Shankar <vshankar@xxxxxxxxxx> can you confirm these are not blockers? >> >> - https://tracker.ceph.com/issues/67999 - >> upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has >> been deprecated and is no longer supported" is absent from osd log >> >> > How is this cephfs related? > > >> >> - >> - https://tracker.ceph.com/issues/68002 - >> upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps >> >> > I left my findings in the tracker. See > > https://tracker.ceph.com/issues/68002#note-2 > > The fsstress crash is due to SIGBUS and there are a flurry of OSD related > heartbeat_check messages from a node where there is a cephfs kernel driver > traceback in the kernel ring buffer. I am running the tracker through Xiubo. > > >> >> - >> >> When this is confirmed, the upgrade suite can be approved. >> >> *Smoke: APPROVED* >> >> *Rados: Pending check from core team* >> >> >> On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@xxxxxxxxxx> wrote: >> >>> Rados reviewed here: >>> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstrackercephcomissues67779 >>> >>> I have asked @Radoslaw Zarzynski <rzarzyns@xxxxxxxxxx> to take a look >>> at the summary and confirm there are no blockers. >>> >>> In progress reviewing upgrade and smoke; will provide another update for >>> those soon as well. >>> >>> On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@xxxxxxxxxx> wrote: >>> >>>> Hey Rachana, >>>> >>>> Thanks for these links. Currently reviewing them and will update the >>>> thread soon. >>>> >>>> Thanks, >>>> Laura >>>> >>>> On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@xxxxxxxxxx> >>>> wrote: >>>> >>>>> Hi All ! >>>>> PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked >>>>> on top of the last built. RADOS, upgrade and smoke test suits were >>>>> executed. >>>>> >>>>> Seeking approvals/reviews for following reruns >>>>> >>>>> RADOS Test Result >>>>> <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- >>>>> Neha >>>>> Upgrade Test result >>>>> <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- >>>>> Neha, Laura >>>>> Smoke Test Result >>>>> <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/> >>>>> >>>>> >>>>> >>>>> On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@xxxxxxxxxx> >>>>> wrote: >>>>> >>>>>> We have added one more core PR to the release, which was deemed a >>>>>> blocker ( >>>>>> https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke >>>>>> will >>>>>> need to be reapproved. >>>>>> >>>>>> On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@xxxxxxxxxx> >>>>>> wrote: >>>>>> >>>>>> > Upgrade suites approved: >>>>>> > >>>>>> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcomissues67779 >>>>>> > >>>>>> > On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@xxxxxxxxxx> >>>>>> wrote: >>>>>> > >>>>>> >> orch approved, failures are known issues and not release blockers >>>>>> >> >>>>>> >> On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein < >>>>>> yweinste@xxxxxxxxxx> >>>>>> >> wrote: >>>>>> >> >>>>>> >> > Details of this release are summarized here: >>>>>> >> > >>>>>> >> > https://tracker.ceph.com/issues/67779#note-1 >>>>>> >> > >>>>>> >> > Release Notes - TBD >>>>>> >> > Gibba upgrade -TBD >>>>>> >> > LRC upgrade - TBD >>>>>> >> > >>>>>> >> > It was decided and agreed upon that there would be limited >>>>>> testing for >>>>>> >> > this release, given it is based on 19.1.1 rather than a full >>>>>> rebase. >>>>>> >> > >>>>>> >> > Seeking approvals/reviews for: >>>>>> >> > (some reruns are still in progress) >>>>>> >> > >>>>>> >> > rgw - Eric, Adam E >>>>>> >> > fs - Venky >>>>>> >> > orch - Adam King >>>>>> >> > >>>>>> >> > quincy-x - Laura, Neha >>>>>> >> > reef-x - Laura, Neha >>>>>> >> > >>>>>> >> > Pls let me know if any tests were missed from this list. >>>>>> >> > _______________________________________________ >>>>>> >> > ceph-users mailing list -- ceph-users@xxxxxxx >>>>>> >> > To unsubscribe send an email to ceph-users-leave@xxxxxxx >>>>>> >> > >>>>>> >> > >>>>>> >> _______________________________________________ >>>>>> >> ceph-users mailing list -- ceph-users@xxxxxxx >>>>>> >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >>>>>> >> >>>>>> > >>>>>> > >>>>>> > -- >>>>>> > >>>>>> > Laura Flores >>>>>> > >>>>>> > She/Her/Hers >>>>>> > >>>>>> > Software Engineer, Ceph Storage <https://ceph.io> >>>>>> > >>>>>> > Chicago, IL >>>>>> > >>>>>> > lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>>>>> > M: +17087388804 >>>>>> > >>>>>> > >>>>>> > >>>>>> >>>>>> -- >>>>>> >>>>>> Laura Flores >>>>>> >>>>>> She/Her/Hers >>>>>> >>>>>> Software Engineer, Ceph Storage <https://ceph.io> >>>>>> >>>>>> Chicago, IL >>>>>> >>>>>> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>>>>> M: +17087388804 >>>>>> _______________________________________________ >>>>>> ceph-users mailing list -- ceph-users@xxxxxxx >>>>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> Rachana >>>>> >>>> >>>> >>>> -- >>>> >>>> Laura Flores >>>> >>>> She/Her/Hers >>>> >>>> Software Engineer, Ceph Storage <https://ceph.io> >>>> >>>> Chicago, IL >>>> >>>> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>>> M: +17087388804 >>>> >>>> >>>> >>> >>> -- >>> >>> Laura Flores >>> >>> She/Her/Hers >>> >>> Software Engineer, Ceph Storage <https://ceph.io> >>> >>> Chicago, IL >>> >>> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >>> M: +17087388804 >>> >>> >>> >> >> -- >> >> Laura Flores >> >> She/Her/Hers >> >> Software Engineer, Ceph Storage <https://ceph.io> >> >> Chicago, IL >> >> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> >> M: +17087388804 >> >> >> > > -- > Cheers, > Venky > -- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |