On Thu, Jun 18, 2020 at 9:26 AM Neal Gompa <ngompa13@xxxxxxxxx> wrote: > > On Thu, Jun 18, 2020 at 8:45 AM Josh Boyer <jwboyer@xxxxxxxxxx> wrote: > > > > Hello Fedora Community! > > > > I am a long-time Fedora Community member, and may be familiar to many > > through previous FESCo or devel list discussions and passionate > > debates. However I write to you today with a different community hat > > on, as a lead Architect for Red Hat Enterprise Linux. The RHEL > > organization has been following the modularity discussions within > > Fedora, particularly around ELN, and often the question of what plans > > we have for modularity in RHEL 9 has come up. Our Fedora Project Lead > > and a number of FESCo members have reached out and asked if we can > > provide some perspective here, and I am both happy and excited to have > > that opportunity. > > > > Thank you for taking the opportunity to talk to us from the Red Hat > Enterprise Linux perspective. I greatly appreciate that and I hope > others respond kindly to this outreach with constructive feedback. > > > As the Fedora Council has pointed out [1], we certainly acknowledge > > there are improvements to be made and have a team already working on > > them. They recently outlined their plans in conjunction with our > > Product Management team in a Fedora Council call as well [2]. We’re > > continuing to invest time and effort in this packaging solution and > > are confident that the team can deliver against their plan. It is > > somewhat of a new experience for all of us when Red Hat is direct with > > our product intentions, but we discussed the larger gaps we see with > > usage in RHEL and are putting our efforts towards solving those gaps > > with this plan. > > > > Modularity is important to RHEL and those efforts are already > > underway. We will be leveraging modularity in RHEL 9 where it most > > makes sense. This is primarily centered around our Application > > Streams concept, which has been well received by our customer base. > > Providing a consistent but improved experience is the base > > requirement, which allows us to have continuity from RHEL 8 to RHEL 9 > > and lowers the hurdle for our customers when upgrading from one major > > version to another. > > > > Personally, as a user of the Application Streams stuff in my > custom-built EL containers, it's very nice, and it works well for > providing the flexibility I've needed while having a fully supportable > stack of software. It's a bit less fun on regular servers and VM > environments, but I think this can improve. > > > It is always good to push the boundaries and search for better ideas > > and improvements, and that is part of what makes Fedora great. We are > > doing this in the context of the RHEL 9 release as well, so our near > > term timeline and requirements mean we are working on evolving > > modularity, not a revolution or a replacement. We are excited by ELN, > > as it presents a possible space to allow those that want to continue > > to iterate on modules a place to do so without necessarily impacting > > the broader Fedora distribution in its entirety. It is my personal > > hope that we can use that opportunity to improve modules and > > modularity in the open source, Fedora-first way we’d prefer. Our near > > term effort to improve the existing modularity implementation ahead of > > RHEL 9 needs to occur, and we’d like to do that work in Fedora, rather > > than in closed product development. Longer term, we are open to > > contributing to a better replacement that meets many of the same > > goals. This is what makes our distribution ecosystem work well, and > > not having that upstream lessens the value we all get from such > > experimentation in the open. > > > > Something that has been bothering me a bit is that there's a lot of > mixed messaging around ELN, even from Red Hatters. Indeed. It's new :) > Don't get me wrong: I *absolutely* want ELN to exist, and I like that > we're doing it. I *want* RHEL development happening in Fedora. > > But I'm confused about the purpose of ELN. Is it intended to be the > development playground for Red Hatters? Or is it a community > initiative to support Fedora and Red Hat to come together on > developing RHEL? Or is it just a fake-RHEL built on Fedora to minimize > the burden of forking Fedora for making RHEL later? I had personally > hoped that ELN would be an opportunity to allow the Fedora community > and Red Hat to work together on building the future RHEL more > directly, but I am unsure of what it does or what it is for now. I can tell you my opinion, but because ELN is a new concept/idea it is only my interpretation. I see ELN as an opportunity born from the Fedora community to experiment with the distribution aimed at a different userbase than what we'd traditionally see with typical Fedora usage. It is not that different from Editions in many ways, except Editions were crafted as a focusing mechanism leveraging the same Fedora packages and defaults. ELN allows experimentation with different defaults or baselines. Matthew has talked about a Playground concept before, and ELN incorporates some of those ideas as well. Clear as mud, yes? New ideas often make things murky, particularly as they are being worked through. I see this as a natural occurrence in open source. With my official Red Hat hat on, I do *not* see ELN as a fake-RHEL to minimize forking Fedora. If we wanted to do that, we could. ELN has the potential to become a development space for major RHEL releases, but that potential only exists as much as we let it. There are many times where Enterprise and non-Enterprise requirements are in direct conflict, and ELN could be the place to iterate on them. If people aren't comfortable developing and experimenting with that deviation in ELN, then it makes developing RHEL in Fedora harder but it doesn't make ELN useless or remove options for open RHEL development entirely. It's an opportunity. It's up to both sides to use it. > As for iterating on modularity in Fedora through ELN, I think this is > a good idea. I want to see the implementation of modularity fleshed > out and the packager experience improved, and I think the only way to > do that is to actually use it and inflict all the pain on the people > who need it by not permitting weird hacky workarounds to make module > builds work. If something is broken, the standard code path has to be > fixed, and that's how I expect this will work. I'm already aware that > CentOS rebuilds of RHEL are not necessarily straightforward because > many hacky shortcuts are taken to build RHEL content and CentOS does > not have those in place. I would say that we shouldn't conflate building a distribution with modules included and *rebuilding* a distribution with modules included. CentOS rebuilds are probably not something I would consider in the context of Fedora, ELN, and modules. Probably best to discuss that on the CentOS lists. Put another way, what you phrase as hacky shortcuts and struggles are not the same issues we'd necessarily see when trying to build a distribution that way for the first time. There is certainly an overlap of issues, but we should constrain ourselves a bit. > However, I am concerned that as ELN develops further, we are likely to > be even more starved for build resources than we have been previously. > Modules are huge build chains that essentially fill up the builders. > Outside of the improved AArch64 hardware, I'm personally unaware of > any improvements in our build capacity to help support the higher > demands for the build system. To note, we'd have this problem without > modules if we had Koschei configured to auto-rebuild and submit > rebuilds on dependency drift so that packagers didn't have to do that > grunt work manually, so it's a matter of we literally do not have > enough resources to support more automation. I've mentioned this > before in other threads, but to reiterate: it is my belief that Fedora > does not have enough build capacity to support building a modularized > distribution. Even when we were doing modularized builds in the Fedora > Rust SIG, it was common for module build jobs to stall out waiting for > resources, and thus get stuck midway through. This also starved > regular builds of resources to get things done. My understanding is that this was discussed with Fedora infrastructure. I have no insight into that further. We probably aren't going to get additional builders for ELN or Fedora in general anytime soon, so allocation of existing resources is probably something to consider. I'll offer that there are actually more Fedora build hosts than there are RHEL 8 build hosts, by comparison. Personally, I have long wanted burst-to-cloud or the ability for others to donate hosts to the Fedora build system without having to physically ship hardware. Koji is somewhat limited in that regard. Maybe developing a shim layer and some security best practices to allow that would help. > > Hopefully that provides some context and helps FESCo and the wider > > community understand where Red Hat is headed with modularity on the > > Enterprise side. > > > > It absolutely does, and I hope you continue to engage with us on this! > Let's make everything better together! That's the idea :) josh _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
