On Thu, 16 Oct 2014 11:14:01 +0200 Mikolaj Izdebski <mizdebsk@xxxxxxxxxx> wrote: > On 10/15/2014 09:31 PM, Kevin Fenzi wrote: > > ok, some general questions, please excuse me if they are dumb. ;) > > > > high level: > > > > * How well does it keep up currently? I know you are careful not to > > overload koji, but I wonder if that means things like perl builds > > are often behind because there are so many of them? > > Koji has more than enough resources to sustain current Koschei load > (~3000 packages). Storage might become problematic if more packages > are added (scratch build results are kept for some period of time), > but we have a solution for that (see [2] or [3]). If more Koji > builders are ever need then I think it sould be quite easy to add > them, as long as there is budget for that. Great. I wasn't sure if it was keeping up or not. ;) Interesting idea on the immediate purge. > > * right now the service is opt-in right? Someone adds a group and > > packages in that group and then when one of them changes it > > scratch rebuilds the rest. Do you see a time/case when we could > > just make it operate on all builds? that is, build foo is made, and > > it just does all the things that buildrequire foo? > > For now only some subset of all packages is tracked by Koschei, but > the ultimate goal is to track all packages - they would be added > automatically after first build appears on Koji and removed when they > are blocked. What would be up to individuals is maintainig package > groups. (One package can be in any number of groups.) ok. Groups are only managed manually by maintainers? And deps are then checked when something updates in a group and the rest of the group rebuilt? or can you explain when a scratch build is fired? > > * The notifications of failed builds currently are via fedmsg? We > > should investigate adding this to FMN if it's not already there, > > so anyone interested could be notified via that. > > fedmsg publishing is already operational as can be seen on [1]. FMN > rule has been recently added. The new FMN is not yet in production, > but in (hopefully near) future users will be able to enable email or > IRC notifications for buildability status of packages they are > interested in. Great! > > todo's/ideas: > > > > * Could this ever be a koji plugin? Or does it do too much on top of > > that to ever be a plugin? > > Koschei has its own architecture and converting it to Koji plugin > would require substantial amount of work. In other words, it should > be possible, but I don't see any good reason to do so. Fair enough. > > * Might it be possible to run on all the broken deps packages in > > rawhide/branched? This would depend I guess on the compose process > > generating fedmsgs with those package names, but if so it could > > tell maintainers "hey, your package is broken in rawhide, but a > > simple rebuild will fix it" (or any other group that just wants to > > go fix them). > > This is an interesting idea. > > A simillar feature was planned for future. The idea was that Koschei > could be resolving runtime dependencies of all packages besides just > build dependencies. Users would be able to see whether package is > installable and if yes, see its installation size with dependencies > (in terms of MB to download, MB installed size and package count). > There would be graphs showing how this changes in time. (We had a > simillar POC service runnig for a few months, see [4].) > > We could extend this and make Koschei resolve runtime dependencies of > successful scratch builds it runs. In case scratch build would fix > broken package in offcial repo, a fedmsg would be emited. Yeah, something to consider... > > * boost is another group of packages I could see this being useful > > for. Perhaps it woul<d be worth reaching out to the boost > > maintainers? > > I don't know specifics of boost packages, but we'll cosider any > feature request. ok. Boost often updates once a cycle or so, and lots of dependent packages need rebuilding. If we could see which of those fail it could be helpfull. But this is up to boost maintainers I suppose. > > * Could this be used to scratch build packages that are > > ExcludeArch/ExclusiveArch with that removed? ie, to tell > > maintainers, "hey, you exclude arm, but it builds ok, are you sure > > thats fine?" > > This would require generating a new SRPM with > ExcludeArch/ExclusiveArch removed, which requires installing all > build dependencies, so it should be done by Koji as buildSRPMfromSCM > task. This in turn requires Koschei having ability to push to some > branch in SCM or maintaining separate git repo and changing Koji > policy to allow scratch builds from it. And of course this would have > to be implemented in Koschei. Not impossible, but looks like a lot of > work for something that could be done manually by running some script > from time to time. Yeah, agreed. > > technical: > > > > * Can this application be load balanced any? Ie, if we have two of > > them could they operate against the same db at the same time? > > To answer this question I need to elaborate more about Koschei > architecture. tl;dr yes, it can be load balanced well. ...snip... > To sum up, all components either don't need load balancing or can be > load-balanced. ok. And I suspect this service would be ok just being one instance as well, since it's not critical. ie, if we had to update/reboot it we could just do so and it would go on, no need to keep everything up 100%? > > > * Are there any common sysadmin tasks we need to know about with the > > instance? Is there any special process to start/stop/reinstall > > it? > > Installing Koschei is done by installing RPM package from Fedora or > EPEL repositories and coping a single config file. > Managing all services (starting, stopping, viewing logs etc.) is done > using standard system tools (systemctl, journalctl and so on). > There is nothing special to be done besides standard sysadmin stuff > (updating packages, viewing logs, backing up database and so on). Excellent. > > * When there is koji maint, should we stop this service? How do we > > gracefully do that and start it again? > > In this case you can stop Koschei services that communicate with Koji > (using systemctl stop koschei-<servicename>) and start them when > maintenance is over. Web UI will remain functional during that time, > but there will be no new build scheduled. > > > Thats all I can think of right now. :) > > I hope this pretty long email answers your questions. It does indeed. Thanks for taking the time to do so. :) kevin
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
