Re: [ceph-users] Re: 16.2.11 pacific QE validation status

[Casey Bodley <cbodley@xxxxxxxxxx>]

thanks Yuri, rgw approved based on today's results from https://pulpito.ceph.com/yuriw-2022-12-20_15:27:49-rgw-pacific_16.2.11_RC2-distro-default-smithi/

On Mon, Dec 19, 2022 at 12:08 PM Yuri Weinstein <yweinste@xxxxxxxxxx> wrote:

If you look at the pacific 16.2.8 QE validation history (https://tracker.ceph.com/issues/55356), we had pacific-x, nautilus-x, and pacific-p2p all green with one exception (https://tracker.ceph.com/issues/51652)

Now we see so many failures in this point release with references to old issues.

Is there anything we can fix to make them less "red"?

Thx

YuriW

On Thu, Dec 15, 2022 at 2:56 PM Laura Flores <lflores@xxxxxxxxxx> wrote:

I reviewed the upgrade runs:

https://pulpito.ceph.com/yuriw-2022-12-13_15:57:57-upgrade:nautilus-x-pacific_16.2.11_RC-distro-default-smithi/
https://pulpito.ceph.com/yuriw-2022-12-13_21:47:46-upgrade:nautilus-x-pacific_16.2.11_RC-distro-default-smithi/
https://pulpito.ceph.com/yuriw-2022-12-13_15:58:18-upgrade:octopus-x-pacific_16.2.11_RC-distro-default-smithi/
https://pulpito.ceph.com/yuriw-2022-12-14_15:41:10-upgrade:octopus-x-pacific_16.2.11_RC-distro-default-smithi/

Failures:
  1. https://tracker.ceph.com/issues/50618 -- known bug assigned to Ilya; assuming it's not a big deal since it's been around for over a year
 
Details:
  1. qemu_xfstests_luks1 failed on xfstest 168 - Ceph - RBD


https://pulpito.ceph.com/yuriw-2022-12-13_15:58:24-upgrade:pacific-p2p-pacific_16.2.11_RC-distro-default-smithi/
https://pulpito.ceph.com/yuriw-2022-12-14_15:40:37-upgrade:pacific-p2p-pacific_16.2.11_RC-distro-default-smithi/

Failures, unrelated:
  1. https://tracker.ceph.com/issues/58223 -- new failure reported by me 7 days ago; seems infrastructure related and not regression-related
  2. https://tracker.ceph.com/issues/52590 -- closed by Casey; must not be of importance
  3. https://tracker.ceph.com/issues/58289 -- new failure raised by me today; seems related to other "wait_for_recovery" failures, which are generally not cause for concern since they're so infrequent.
  4. https://tracker.ceph.com/issues/51652 -- known bug from over a year ago

Details;
  1. failure on `sudo fuser -v /var/lib/dpkg/lock-frontend` - Infrastructure
  2. "[ FAILED ] CmpOmap.cmp_vals_u64_invalid_default" in upgrade:pacific-p2p-pacific - Ceph - RGW
  3. "AssertionError: wait_for_recovery: failed before timeout expired" from down pg in pacific-p2p-pacific - Ceph - RADOS
  4. heartbeat timeouts on filestore OSDs while deleting objects in upgrade:pacific-p2p-pacific - Ceph - RADOS

On Thu, Dec 15, 2022 at 4:34 PM Brad Hubbard <bhubbard@xxxxxxxxxx> wrote:

On Fri, Dec 16, 2022 at 3:15 AM Yuri Weinstein <yweinste@xxxxxxxxxx> wrote:
>
> Details of this release are summarized here:
>
> https://tracker.ceph.com/issues/58257#note-1
> Release Notes - TBD
>
> Seeking approvals for:
>
> rados - Neha (https://github.com/ceph/ceph/pull/49431 is still being
> tested and will be merged soon)
> rook - Sébastien Han
> cephadm - Adam
> dashboard - Ernesto
> rgw - Casey (rwg will be rerun on the latest SHA1)
> rbd - Ilya, Deepika
> krbd - Ilya, Deepika
> fs - Venky, Patrick
> upgrade/nautilus-x (pacific) - Neha, Laura
> upgrade/octopus-x (pacific) - Neha, Laura
> upgrade/pacific-p2p - Neha - Neha, Laura
> powercycle - Brad

The failure here is due to fallout from the recent lab issues and was
fixed in main by https://github.com/ceph/ceph/pull/49021 I'm waiting
to see if there are plans to backport this to pacific and quincy since
that will be needed.

> ceph-volume - Guillaume, Adam K
>
> Thx
> YuriW
>
> _______________________________________________
> Dev mailing list -- dev@xxxxxxx
> To unsubscribe send an email to dev-leave@xxxxxxx



--
Cheers,
Brad

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

--

Laura Flores

She/Her/Hers

Software Engineer, Ceph Storage

Red Hat Inc.

Chicago, IL

lflores@xxxxxxxxxx   
M: +17087388804    

@RedHat   Red Hat   Red Hat

_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx

_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx