============================================ #fedora-meeting: Infrastructure (2014-09-04) ============================================ Meeting started by nirik at 18:00:18 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2014-09-04/infrastructure.2014-09-04-18.00.log.html . Meeting summary --------------- * welcome (nirik, 18:00:18) * New folks introductions and Apprentice tasks (nirik, 18:03:02) * Freeze reminder (nirik, 18:06:32) * Applications status / discussion (nirik, 18:07:05) * LINK: http://209.132.184.188/spechub/ (pingou, 18:07:20) * http://209.132.184.188/spechub/ under development, feedback welcome (dist-git frontend) (nirik, 18:16:08) * taskotron to hit prod after alpha hopefully and replace autoqa once and for all (nirik, 18:16:25) * bodhi1 updates this week to handle epel7 (nirik, 18:16:34) * Sysadmin status / discussion (nirik, 18:19:17) * Nagios recap (nirik, 18:26:13) * Upcoming Tasks/Items (nirik, 18:27:07) * LINK: https://apps.fedoraproject.org/calendar/list/infrastructure/ (nirik, 18:27:07) * Open Floor (nirik, 18:32:49) * LINK: https://groups.google.com/forum/#!msg/gitolite/iSfd4b-UbhM/MDU2aP0DvCgJ (pingou, 18:41:45) * LINK: ssh://pingou@xxxxxxxxxxxxxxxxxxxxxx/fedocal -> ssh://gitolite@xxxxxxxxxxxxxxxxxxxxxx/fedocal (pingou, 18:45:11) * LINK: https://groups.google.com/forum/#!msg/gitolite/iSfd4b-UbhM/MDU2aP0DvCgJ (pingou, 18:47:41) Meeting ended at 18:56:34 UTC. Action Items ------------ Action Items, by person ----------------------- * **UNASSIGNED** * (none) People Present (lines said) --------------------------- * nirik (75) * pingou (64) * bochecha (25) * mirek-hm (15) * tflink (13) * dgilmore (13) * threebean (12) * zodbot (5) * smooge (2) * akshays (2) * lmacken (1) * michel_slm (1) * oddshocks (1) * danofsatx (1) * danielbruno (1) * relrod (1) * janeznemanic (1) * roshi (1) * puiterwijk (0) * abadger1999 (0) * mdomsch (0) -- 18:00:18 <nirik> #startmeeting Infrastructure (2014-09-04) 18:00:18 <zodbot> Meeting started Thu Sep 4 18:00:18 2014 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:18 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:18 <nirik> #meetingname infrastructure 18:00:18 <nirik> #topic welcome 18:00:18 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk 18:00:18 <zodbot> The meeting name has been set to 'infrastructure' 18:00:18 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean 18:00:51 * threebean is here 18:01:06 * pingou 18:01:14 * relrod here 18:01:16 <janeznemanic> hi 18:01:30 <bochecha> hi there 18:01:41 <dgilmore> hola 18:02:16 <akshays> hello 18:02:21 * mirek-hm is here 18:02:50 <nirik> hello everyone. ;) 18:03:02 <nirik> #topic New folks introductions and Apprentice tasks 18:03:16 <nirik> any new folks like to introduce themselves? 18:03:26 <nirik> or apprentices with questions or comments? 18:04:21 * michel_slm here 18:04:26 <akshays> Hi, i am new here. I am a sys admin.' 18:04:58 <nirik> akshays: welcome. 18:05:32 * oddshocks pops in 18:05:45 * danielbruno here 18:06:29 * roshi listens 18:06:32 <nirik> #topic Freeze reminder 18:06:43 <nirik> Just a reminder we are in freeze until f21 alpha goes out... 18:06:57 <nirik> see the note on the list for more information. 18:07:05 <nirik> #topic Applications status / discussion 18:07:15 <nirik> any application news/status? 18:07:20 <pingou> http://209.132.184.188/spechub/ 18:07:24 <threebean> oo 18:07:29 <pingou> based on progit, made for pkgs.fp.o 18:07:46 <pingou> would integrate quite easily with gitolite2 18:08:02 <pingou> we'll have to see when we figure out how to handle gitolite3 :) 18:08:04 <nirik> yeah, I still need to poke around on it. 18:08:45 <pingou> basically, for the integration w/ gitolite2, we can just call http://209.132.184.188/spechub/admin/gitolite/conf and embed the output in the conf file with our cron task 18:08:47 <nirik> this (or something like it) might meet the needs mirek-hm was looking for for copr dist-git perhaps. 18:08:48 <mirek-hm> we (as copr team) started thinking about dist-git for copr, I sent email to fedora-devel today, so please respond and discuss there 18:08:56 <pingou> oh, and it works fine on the 19,000+ git repos :) 18:09:15 <nirik> mirek-hm: yeah, I wonder if something like this might help that case. ;) 18:09:18 <threebean> pingou: looks neat 18:09:20 <nirik> but we will have to discuss. 18:09:31 <pingou> threebean: thanks ;-) 18:09:38 <tflink> we've been finding small issues/annoyances with taskotron in stg and fixing them as we go - no more pto for a while so taskotron production should be do-able before beta 18:09:44 <mirek-hm> pingou: but that is "just" fronted to dist-git, isn't it? 18:10:01 <pingou> mirek-hm: it's a front-end to our git repos, allows forking and merging 18:10:28 <pingou> well, I need to work with threebean for the auth part that requires caching the acls from pkgdb until they change 18:10:52 <mirek-hm> ok, that would be nice one we choose how to do our dist-git, will check that. 18:10:55 <nirik> pingou: forks and commits to those folks are stored where? in git with the project forked? or ? 18:11:20 <nirik> tflink: cool. ;) so, after alpha we can nuke autoqa and enable prod? 18:11:24 <pingou> nirik: in a new git repo under the forks/ folder 18:11:34 <mirek-hm> pingou: https://fedorahosted.org/spechub/ does not exist (taken from footer) 18:11:43 <pingou> mirek-hm: request pending in trac ;-) 18:11:48 <mirek-hm> ok 18:11:50 <nirik> mirek-hm: I haven't processed his request to add it yet. ;) 18:11:51 <pingou> mirek-hm: code is in github ftm 18:12:00 <threebean> tflink: any chance there will be a new release of resultsdb before you go to prod? 18:12:09 <tflink> nirik: that's the plan 18:12:17 <tflink> threebean: are there unreleased features? 18:12:20 * threebean nods 18:12:23 <threebean> unreleased bugfix 18:12:24 <nirik> pingou: so, you could fork say kernel, then commit whatever to it and others can clone that,e tc? 18:12:31 <tflink> I thought that the jsonp stuff was already part of stg 18:12:59 <pingou> nirik: we don't allow forking a fork, but you can clone locally a fork sure 18:13:02 <tflink> threebean: I'll look into that after the meeting, if we have unreleased fixes, I'll do a release 18:13:12 <threebean> tflink: yeah, this one in particular -> https://bitbucket.org/fedoraqa/resultsdb/commits/72ce429c1c68073780178566163927f75cdaec94 18:13:38 <nirik> pingou: ok. 18:14:47 <nirik> alright. Any other application news? 18:14:54 <pingou> oh 18:14:59 * danofsatx is here, but muy busy 18:15:02 <tflink> threebean: I think that's already been released - should be in dev and stg 18:15:04 <pingou> I'm loading datagrepper's data into mongodb 18:15:21 <lmacken> I upgraded bodhi in production this week with changes for EPEL-7 18:15:23 <pingou> on 209.132.184.235 18:15:42 <pingou> db.messages.count() : 92541 -- it's slow to load 18:15:59 <threebean> clarification -> slow to populate, right? 18:16:08 <nirik> #info http://209.132.184.188/spechub/ under development, feedback welcome (dist-git frontend) 18:16:25 <nirik> #info taskotron to hit prod after alpha hopefully and replace autoqa once and for all 18:16:28 <pingou> threebean: yes 18:16:33 <pingou> arg, it failed :( 18:16:34 <nirik> #info bodhi1 updates this week to handle epel7 18:17:10 <threebean> tflink: not sure. I'll send a link in #fedora-apps 18:18:41 <threebean> pingou: yeah, it will be interesting to see if there's any significant difference in query speed when mongo and postgres have the same number of records/documents. 18:19:17 <nirik> #topic Sysadmin status / discussion 18:19:37 <nirik> we have had several mirrormanager blowups this week sadly. 18:19:38 <pingou> threebean: looking forward that :) 18:19:56 <mirek-hm> I would like to urge https://fedorahosted.org/fedora-infrastructure/ticket/4488 (/me is looking at puiterwijk) 18:20:02 <nirik> One seems to have been due to bad data in the db causing the update directory list to fail. 18:20:26 <pingou> a foreign key not enforced 18:20:29 <nirik> mirek-hm: yeah, not sure where puiterwijk is. ;( I will do that myself after the meeting. 18:20:32 <pingou> and a delete not propagate 18:21:13 <nirik> anyhow, we will keep things going along on it until we replace it. ;) 18:21:39 <nirik> we have also been having some vpn saturation issues... 18:22:06 <nirik> It's possible these are due to high fedmsg busgateway traffic. So, we moved that to not go over the vpn... 18:22:23 <nirik> but that won't entirely take effect until clients restart 18:22:54 <mirek-hm> nirik: and regards https://fedorahosted.org/fedora-infrastructure/ticket/4514 - nirik do you think 256 IPs per tenant is enough? I would rather go with 172.N.x.x Class B networks for every tenant. E.g. 172.1.x.x for copr, 172.2.x.x for permanent, 172.3.x.x for fooo.... 18:23:07 <threebean> I'm preparing a freeze break request to push some of that traffic back to the proxies which should help. 18:23:35 <nirik> mirek-hm: well, if you like, sure. I really don't think we are going to have more than 256 per client, but I guess I could be wrong. Use Class B if you like... 18:24:23 <mirek-hm> that give us 16k networks and 65k IP per each network, that is more then enough in both directions 18:25:24 <nirik> sure, works for me 18:26:13 <nirik> #topic Nagios recap 18:26:26 <nirik> .tiny https://admin.fedoraproject.org/nagios/cgi-bin//summary.cgi?report=1&displaytype=3&timeperiod=last7days&smon=9&sday=1&syear=2014&shour=0&smin=0&ssec=0&emon=9&eday=4&eyear=2014&ehour=24&emin=0&esec=0&hostgroup=all&servicegroup=all&host=all&alerttypes=3&statetypes=3&hoststates=7&servicestates=120&limit=25 18:26:27 <zodbot> nirik: http://tinyurl.com/l3vjae8 18:26:34 <nirik> pretty much all of those are due to the vpn thing. 18:27:07 <nirik> #topic Upcoming Tasks/Items 18:27:07 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/ 18:27:20 <nirik> anyone have upcoming items they would like to note or schedule? 18:28:13 <pingou> threebean and I prepare an announce about FMN 18:28:21 <nirik> cool. 18:28:30 <nirik> just more widespread? or ? 18:28:35 <pingou> of the 1566 packagers we have in FAS, 1453 are not using FMN yet 18:28:50 <pingou> so we want to create their account automatically 18:29:06 <pingou> so we don't have a clear date for the change yet 18:29:14 <nirik> ok 18:29:15 <pingou> but I'd hope for early october 18:29:16 <mirek-hm> FMN? 18:29:25 <pingou> mirek-hm: https://apps.fedoraproject.org/notifications/ 18:29:44 <nirik> basically a way to be notified as you like when fedmsg's happen. 18:29:55 <nirik> instead of always emails 18:30:30 <threebean> you can use it to get emails about coprs, for instance. 18:31:17 <mirek-hm> hmm can someone add it on list of trusted apps for fedoauth? 18:31:30 <nirik> I thought we already did? 18:31:46 <mirek-hm> It just asked me to review the login 18:31:54 <nirik> if not, we can... (althought we are in freeze) 18:32:09 <mirek-hm> true, it can wait 18:32:43 <nirik> anyhow, it's a handy setup. ;) 18:32:49 <nirik> #topic Open Floor 18:32:55 <nirik> ok, any items for open floor? 18:33:22 * tflink is curious if there has been any status change on the new app server and qa host 18:33:49 <nirik> tflink: qa09/virthost-comm03? 18:33:52 <tflink> yep 18:34:02 <nirik> smooge has been working to bring them up. 18:34:19 <nirik> I am not sure what the current status was. I know they are on the net and all, just need installing (which he was doign) 18:34:32 * tflink just wants to get some stuff moved over to the new hosts before warranties start expiring 18:34:43 <nirik> yeah, understandable. 18:34:54 <nirik> will poke him when he's around 18:35:04 <bochecha> nirik, what's the next step of moving distgit to ansible? I kind of lost track after my patches were merged (I see you fixed some errors I made, then it seems we're halfway through with moving to gitolite3?) 18:35:08 <tflink> we should be getting 2 more hosts in q2 and q3, still in process 18:35:45 <pingou> bochecha: nirik I can help until next week thursday if we want to move closer at gitolite3 18:36:10 <nirik> bochecha: so yeah, pkgs01.stg is in ansible now. It's rhel7. It has synced data on it. However, we have no gitolite2 in rhel7, so we wanted to look at moving to 3, but it's apparently a big jump. pingou looked into it, but I haven't had time yet. 18:36:30 <nirik> bochecha: also, cgit isn't working right, but thats likely seliux or something minor 18:36:49 <pingou> nirik: we don't need no cgit, we have spechub :-p 18:37:09 * pingou ducks 18:37:13 <nirik> pingou: :) 18:37:48 <bochecha> pingou, so what's missing to move to gitolite3? 18:38:12 <smooge> ugh sorry 18:38:28 <tflink> unless there's an issue I don't know of, moving to gitolite3 in the middle of a release that's already chaotic sounds like a bad idea 18:38:34 <nirik> hey smooge. tflink was asking about qa09/virthost-comm03.... any update? 18:38:34 <pingou> bochecha: basically in gitolite2 we have 1 conf file in /etc 18:38:46 <bochecha> tflink, right, we're only doing this on staging, not production 18:38:55 <pingou> bochecha: that conf file defines where the repos are located /srv/... 18:38:57 <tflink> ok, nvm then :) 18:39:08 <pingou> bochecha: in gitolite3 repos *must* be in ~/repositories 18:39:14 <nirik> tflink: well, mostly because we dont have the old version on epel7... but of course we can punt and add it 18:39:23 <smooge> The boxes got to mgmt working. I have to start setup and installation. I expect by Monday 18:39:38 <pingou> so since we use gitolite in such a way that each user has its own account on the machine, it means that we need each user to have ~/repositories 18:39:47 <pingou> oh and ~/.gitolite.rc btw 18:39:49 <tflink> nirik: I thought that the idea was to move production soon, missed the bit about stg 18:40:00 <bochecha> ah right 18:40:10 <pingou> so either we mess with $HOME 18:40:14 <bochecha> is there a reason why we don't use the recommended gitolite setup? (only one user) 18:40:24 <pingou> or we find a way to move to that ^ 18:40:35 <nirik> tflink: well, I would like to move prod (but not in freeze of course). We are mostly just exploring the options in stg right now. 18:40:50 <pingou> which I expect means: 1) a new fedpkg version 2) break all the clones of all the packages out there 18:41:05 <nirik> yeah, those don't seem... feasable 18:41:06 <bochecha> fwiw, at my previous dayjob I was using the "one user only" recommended setup of gitolite for my internal distgit, it was working well 18:41:17 <bochecha> pingou, ah right, the fetch/push urls with the user in them :-/ 18:41:22 <pingou> yup 18:41:33 <pingou> and that's where I stopped :) 18:41:39 <bochecha> well, a new fedpkg could rewrite those urls 18:41:45 <pingou> https://groups.google.com/forum/#!msg/gitolite/iSfd4b-UbhM/MDU2aP0DvCgJ 18:41:46 <pingou> ftr 18:42:04 <pingou> they do speak about us in there 18:42:29 <pingou> and they seem to advice to mess with $HOME 18:42:44 <nirik> yeah, so sounding like we should just get v2 branched for now at least 18:43:11 <bochecha> yeha, maybe gitolite2 (at least in th einfra repo) for el7 would make things simpler for now 18:43:12 <pingou> bochecha: which means, breaking all the old fedpkg around :/ 18:43:30 <bochecha> pingou, well, yeah, forcing an upgrade 18:43:34 <pingou> I wanted to check how hard it would be to build gitolite2 on koji, but I haven't checked it yet 18:43:43 <bochecha> (which is coming anyway, when we move to sha512 for the lookaside hashes) 18:44:02 <bochecha> but yeah, doing it twice is certainly not nice 18:44:06 <pingou> nirik: ^ if we force a fedpkg upgrade already, that might be a good time to do that 18:44:19 <bochecha> pingou, I don't htink we can do both at the same time 18:44:27 <pingou> that == move to the one gitolite user 18:44:30 <pingou> bochecha: why not? 18:44:45 <dgilmore> what does move to one gitolite user actually mean? 18:45:11 <bochecha> dgilmore, git clone gitolite@xxxxxxxxxxxxxxxxxxxxxx:foobar 18:45:11 <pingou> ssh://pingou@xxxxxxxxxxxxxxxxxxxxxx/fedocal -> ssh://gitolite@xxxxxxxxxxxxxxxxxxxxxx/fedocal 18:45:21 <dgilmore> bochecha: yeah no 18:45:31 <dgilmore> bochecha: never will that be okay 18:45:43 * pingou needs more info 18:45:43 <dgilmore> we want to know who commited the changes 18:45:52 <bochecha> we still do with the gitolite@ scheme 18:45:59 <pingou> dgilmore: we do know, gitolite handles that 18:46:03 <nirik> its just at a gitolite layer 18:46:08 <dgilmore> ewww 18:46:11 <dgilmore> no 18:46:12 <bochecha> all users are 'gitolite' on the server, but it's still mapped to the user's private key 18:46:26 <bochecha> dgilmore, it's the upstream recommended way, though 18:46:33 <dgilmore> bochecha: doesnt make it right 18:46:38 <bochecha> sure 18:46:42 <pingou> s/recommended/&designed/ 18:46:52 <bochecha> it's just making it hard/troublesome to do differently 18:47:05 <dgilmore> then we should look at other options 18:47:12 <dgilmore> or work with upstream to make it sane 18:47:38 <pingou> dgilmore: they explicitely removed what allowed to do things the way we did 18:47:41 <pingou> https://groups.google.com/forum/#!msg/gitolite/iSfd4b-UbhM/MDU2aP0DvCgJ 18:47:47 <bochecha> knowing we were doing it this way 18:47:51 <dgilmore> pingou: then we find a replacement 18:48:03 <bochecha> pingou, you handle the acls directly in spechub, ok? :) 18:48:13 * pingou jumps 18:48:18 <bochecha> \o/ 18:48:28 <pingou> no, ther other jump ;-) 18:49:09 <nirik> yeah, I think v2 for now and look for a longer term solution seems like what we will have to do. 18:49:10 <bochecha>
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
