Re: reef 18.2.3 QE validation status

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Jul 3, 2024 at 11:17 AM Casey Bodley <cbodley@xxxxxxxxxx> wrote:

> (cc Thomas Goirand)
>
> in April, an 18.2.3 tarball was uploaded to
> https://download.ceph.com/tarballs/ceph_18.2.3.orig.tar.gz. that's been
> picked up and packaged by the Debian project under the assumption that it
> was a supported release
>
> when we do finally release 18.2.3, we will presumably overwrite that
> existing tarball with the latest contents. Thomas has requested that we
> re-number this release to 18.2.4 to prevent further confusion. Thomas,
> could you explain the consequences of overwriting that tarball in place?
>

Debian is not alone in this regard. :-/

Putting tar files on download.ceph.com/tarballs ahead of the *real* GA is
misleading at best. Replacing the tar file is problematic too, if only
because it's a potential source of confusion for people who aren't paying
attention.

AIUI, after the tar file lands, if the testing is successful, a
corresponding src.rpm and a set of debian packaging files land in
download.ceph.com/rpm-$ver/... and download.ceph.com/debian-$ver/...
respectively.

I gather that most of the time testing goes well, and the sources
download.ceph.com/tarballs/... match whats in
download.ceph.com/{rpm,debian}-$ver and match what's tagged in github.

But IIRC the 18.2.2 testing did not go well what was ultimately tagged as
18.2.2 does not match the tar file in .../tarballs/ceph-18.2.2.tar.gz.  (Or
did the tar file get replaced?)

And the same thing is about to happen again with 18.2.3.

I'm not sure I believe that making this next release 18.2.4 really solves
anything, especially if we're not solving the underlying problem. Can we at
least fix the mistake of uploading a tar file to .../tarballs if it's not
the final, official tar file that matches what is tagged in github.

I'm sort of okay with replacing the 18.2.3 tar file. I'll just respin
Fedora and CentOS Storage SIG packages, either with the new tarball, or
with the sources extracted from the GA .../rpm-$ver or .../debian-$ver.
IOW with a higher Release than the previous spins) (Fedora and CentOS track
the source through their look-aside cache and the sources file in distgit,
so there's actually a trail of cookie crumbs.)

Otherwise I suppose I will be forced to wait for files to land in
.../rpm-$ver and .../debian-$ver if the tar file in .../tarballs can't be
relied on.


>
> On Tue, Jul 2, 2024 at 10:06 AM Yuri Weinstein <yweinste@xxxxxxxxxx>
> wrote:
>
>> After fixing the issues identified below we cherry-picked all PRs from
>> this list for 18.2.3
>> https://pad.ceph.com/p/release-cherry-pick-coordination.
>>
>> The question to the dev leads: do you think we can proceed with the
>> release without rerunning suites, as they were already approved?
>>
>> Please reply with your recommendations.
>>
>>
>> On Thu, Jun 6, 2024 at 4:59 PM Yuri Weinstein <yweinste@xxxxxxxxxx>
>> wrote:
>>
>>> Please see the update from Laura below for the status of this release.
>>>
>>> Dev Leads help is appreciated to expedite fixes necessary to publish it
>>> soon.
>>>
>>> "Hi all, we have hit another blocker with this release.
>>>
>>> Due to centos 8 stream going end of life, we only have the option of
>>> releasing centos 9 stream containers.
>>>
>>> However, we did not test the efficacy of centos 9 stream containers
>>> against the orch and upgrade suites during the initial 18.2.3 release cycle.
>>>
>>> This problem is tracked here: https://tracker.ceph.com/issues/66334
>>>
>>> What needs to happen now is:
>>>
>>> 1.  The orch team needs to fix all references to centos 8 stream in the
>>> orch suite
>>> 2.  fs, rados, etc. need to fix their relative jobs the same way in the
>>> upgrade suite
>>>
>>> The easiest way to tackle that is to raise a PR against main and
>>> backport to stable releases since this problem actually affects main and
>>> all other releases.
>>>
>>> Then, we will:
>>>
>>> 1.  Rerun orch and upgrade with these fixes
>>> 2.  Re-approve orch and upgrade
>>> 3.  Re-upgrade gibba and LRC
>>>
>>> Then the release will be unblocked."
>>>
>>> On Tue, Jun 4, 2024 at 3:26 PM Laura Flores <lflores@xxxxxxxxxx> wrote:
>>>
>>>> Rados results were approved, and we successfully upgraded the gibba
>>>> cluster.
>>>>
>>>> Now waiting on @Dan Mick <dmick@xxxxxxxxxx> to upgrade the LRC.
>>>>
>>>> On Thu, May 30, 2024 at 8:32 PM Yuri Weinstein <yweinste@xxxxxxxxxx>
>>>> wrote:
>>>>
>>>>> I reran rados on the fix
>>>>> https://github.com/ceph/ceph/pull/57794/commits
>>>>> and seeking approvals from Radek and Laure
>>>>>
>>>>> https://tracker.ceph.com/issues/65393#note-1
>>>>>
>>>>> On Tue, May 28, 2024 at 2:12 PM Yuri Weinstein <yweinste@xxxxxxxxxx>
>>>>> wrote:
>>>>> >
>>>>> > We have discovered some issues (#1 and #2) during the final stages of
>>>>> > testing that require considering a delay in this point release until
>>>>> > all options and risks are assessed and resolved.
>>>>> >
>>>>> > We will keep you all updated on the progress.
>>>>> >
>>>>> > Thank you for your patience!
>>>>> >
>>>>> > #1 https://tracker.ceph.com/issues/66260
>>>>> > #2 https://tracker.ceph.com/issues/61948#note-21
>>>>> >
>>>>> > On Wed, May 1, 2024 at 3:41 PM Yuri Weinstein <yweinste@xxxxxxxxxx>
>>>>> wrote:
>>>>> > >
>>>>> > > We've run into a problem during the last verification steps before
>>>>> > > publishing this release after upgrading the LRC to it  =>
>>>>> > > https://tracker.ceph.com/issues/65733
>>>>> > >
>>>>> > > After this issue is resolved, we will continue testing and
>>>>> publishing
>>>>> > > this point release.
>>>>> > >
>>>>> > > Thanks for your patience!
>>>>> > >
>>>>> > > On Thu, Apr 18, 2024 at 11:29 PM Christian Rohmann
>>>>> > > <christian.rohmann@xxxxxxxxx> wrote:
>>>>> > > >
>>>>> > > > On 18.04.24 8:13 PM, Laura Flores wrote:
>>>>> > > > > Thanks for bringing this to our attention. The leads have
>>>>> decided that
>>>>> > > > > since this PR hasn't been merged to main yet and isn't
>>>>> approved, it
>>>>> > > > > will not go in v18.2.3, but it will be prioritized for v18.2.4.
>>>>> > > > > I've already added the PR to the v18.2.4 milestone so it's
>>>>> sure to be
>>>>> > > > > picked up.
>>>>> > > >
>>>>> > > > Thanks a bunch. If you miss the train, you miss the train - fair
>>>>> enough.
>>>>> > > > Nice to know there is another one going soon and that bug is
>>>>> going to be
>>>>> > > > on it !
>>>>> > > >
>>>>> > > >
>>>>> > > > Regards
>>>>> > > >
>>>>> > > > Christian
>>>>> > > > _______________________________________________
>>>>> > > > ceph-users mailing list -- ceph-users@xxxxxxx
>>>>> > > > To unsubscribe send an email to ceph-users-leave@xxxxxxx
>>>>> > > >
>>>>> _______________________________________________
>>>>> ceph-users mailing list -- ceph-users@xxxxxxx
>>>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Laura Flores
>>>>
>>>> She/Her/Hers
>>>>
>>>> Software Engineer, Ceph Storage <https://ceph.io>
>>>>
>>>> Chicago, IL
>>>>
>>>> lflores@xxxxxxx | lflores@xxxxxxxxxx <lflores@xxxxxxxxxx>
>>>> M: +17087388804
>>>>
>>>>
>>>> _______________________________________________
>> Dev mailing list -- dev@xxxxxxx
>> To unsubscribe send an email to dev-leave@xxxxxxx
>>
> _______________________________________________
> Dev mailing list -- dev@xxxxxxx
> To unsubscribe send an email to dev-leave@xxxxxxx
>


-- 

Kaleb
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux