On Fri, 6 May 2016 17:30:18 -0500 Adam Miller <maxamillion@xxxxxxxxxxxxxxxxx> wrote: ...snip... > > Proposal: > > Pulp[1] + Crane[2] + MirrorManager[3] + Docker Distribution[4] Are all of these packaged up? For EPEL? (aside mirrormanager). ...snip... > Workflow: > OSBS will perform Builds, as these builds complete they will be > pushed to the docker-distribution (v2) registry, these will be > considered "candidate images". Pulp will sync and publish the > candidate repository. > > Testing will occur using the "candidate images" (details of how > we want to handle that are outside the scope of this proposal). So, at this point the 'candidate image' is just in pulp? Or it's been published to a directory and mirrored out? I'm guessing it would be published and mirrored so people could test it? ...snip... > Mirror Manager will Pulp distribute to the mirrors the image > layers and their metadata. This should work for mirrors to just rsync the directories right? ...snip... > Some more in depth technical items around this solution that I think > the Fedora Infrastructure Team are likely interested in: > > Pulp Requirements: > - An AMPQ message queue, currently qpid and rabbitmq are > supported upstream. However, the requirement appears to stem from the > use of Celery[6] and Celery upstream supports redis[7] as a broker > backend so I have requested that it be made available as > supported option Pulp[8]. This will obviously take some amount of dev > time, but we can plan for that if adding a message queue to Fedora > Infra is a show stopper. Well, what needs to listen on/publish to this queue? We have tried to avoid celery serveral times in the past and always managed to, but perhaps we can't this time. Is there any alternative to the celery use? > - MongoDB, this is currently a hard requirement but > postgresql is planned replace MongoDB in the future[9] (probably a > year-ish timeline on that). The question is, can we wait that long > from a Fedora Project standpoint for the new feature before having a > solution in place? I imagine some of this will need to be > planned/scoped as time goes on and we learn more but it's > worth keeping in mind well, OSBS already uses mongo (as does openstack), so I don't think this is a blocker, it would be nice to reuse the roles/mongodb for it tho > - Storage. I've been told Pulp likes a lot of storage, I > don't know hard numbers for what we'd need since we're getting into > uncharted territory but I've heard that a few hundred GB is not > uncommon in pulp deployments when combining the MongoDB storage needs > with all the artifacts in the repos. ok. Can this storage be NFS? Or is there some fs requirement? > Crane Requirements: > - Crane is just a small python wsgi app written in flask Hurray! > A couple of things to note about maintenance and uptime > considerations: > > The Intermediate docker-distribution registry is needed for > builds in koji+OSBS > > Pulp will be required for "promotion" of builds from candidate to > testing or stable > > Crane will be required for end users out in the world to access > in order to actually pull down Docker images from us. > > The only service here that needs to be public end-user facing > (i.e. wide open to the internet and not have access locked to a FAS > group) is Crane. All other components should be able to be locked > down similar to the "Fedora internal" components koji (builders, > etc), bodhi (signing, etc) and similar. What port(s) does crane need open? Is this something we could proxy and cache via varnish? Can we/should we look at any HA with any of these parts? For example, if we wanted to apply a kernel update and reboot everything, how could we avoid any downtime that users would see? Would it be as easy as having 2 crane frontends or would downtime on the other internal components affect crane? As far as backups of this, we would only need the pulp storage and the mongodb? Or are there other parts that need backups to restore the entire stack in case of doom? I'm sure I will think of more, but thats all at the moment... kevin
Attachment:
pgpqtETluHuxa.pgp
Description: OpenPGP digital signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
