On Fri, Feb 19, 2016 at 11:09:25AM -0500, Ralph Bean wrote: > On Fri, Feb 19, 2016 at 12:14:30PM +0100, Pierre-Yves Chibon wrote: > > On Wed, Feb 17, 2016 at 11:51:51AM -0500, Ralph Bean wrote: > > > On Wed, Feb 17, 2016 at 12:11:09PM +0100, Pierre-Yves Chibon wrote: > > > > We could have the async service directly linked to pkgdb's DB. This means: > > > > - Changes to pkgdb are directly propagated to pkgs.fp.o > > > > - We can rely on the collections information directly retrieved from the DB > > > > - We can use our current set-up (one shell account / packager) and not tweak > > > > gitolite until it behaves as we want (which is only supported by gitolite to > > > > please us). > > > > - No need for the alias warning for namespacing, we can check if a namespace was > > > > specified and use ``rpms`` if not > > > > - May be easier to hack/maintain in the long term (may be not, hard to say in a > > > > way :)) > > FWIW, the namespace warning is a bit of a red herring. > - We can configure gitolite to not issue the warning now, as things > stand. > - Once we patch fedpkg to adjust the git urls to use the correct > namespace, the warning would go away anyways. (Just need some round > 'tuits there.) > > The other points are right on. I agree that this is a little bit of a red herring since gitolite supports it fine, but then again, gitolite does work for our need and we almost never have to touch it. The last two times we did touch it was: - migration to gitolite 3 - add support for namespacing. I can't see in the future but I also don't think this will change much from now on. > > > I'm not sure about querying pkgdb DB directly from the git hook -- or > > > even querying pkgdb's JSON API directly. > > > > > > - If we connect directly to the DB, it exposes the internals of pkgdb > > > in a way that could make it much harder to upgrade that schema in > > > the future. Better to connect over the REST API. > > > > +1 there > > > > > - If we connect over the REST API, we could run into system > > > interdependence problems in the future. If pkgdb goes down > > > accidentally, or if it needs to go down for maintenance, then the > > > dist-git repos will be dead in the water. No one will be able to > > > push anything. > > > > So my idea here was to not rely on pkgdb itself (also because of the number of > > requests we may have to deal with), but rather to a small service (much like > > mdapi) that would connect directly to pkgdb's DB. > > The service could also run on pkgs01 directly (but would likely be py3, meaning > > some porting work required on pkgdb, which is a good thing anyway). > > I like the idea of a microservice that provides a read-only interface > to pkgdb ACLs. Currently, querying for ACLs from pkgdb can be quite > slow. > > > This would mean: > > - git remains independent from pkgdb (good) > > - git is blocked if the DB server goes down (bad), but if our DB server goes > > down, most of our infra will have troubles anyway. > > Heh, yeah. But let's not put more problems on that pile if we can > avoid it, I guess. > > Could the pkgdb read-only API service use an on-disk cache of the ACLs > just like mdapi does? A sqlite stash? So thinking more about this, if we are to cache the ACLs on disk we loose the biggest win which was: direct propagation of the changes in pkgdb to git. So if we cache on disk, what is left as advantages: - python - not tweak gitolite for our special case (but is supported upstream for us) - may be easier to maintain/hack, or maybe not This makes me wondering if the gain is worth the work then :) (Note that I don't want to do it, async service programming is pretty fun, but `it's fun` might be a little weak as a justification for replacing something that works :-] ) Pierre
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
