============================================ #fedora-meeting: Infrastructure (2012-05-10) ============================================ Meeting started by nirik at 18:00:56 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2012-05-10/infrastructure.2012-05-10-18.00.log.html . Meeting summary --------------- * Robot Roll Call (nirik, 18:00:56) * New folks introductions and Apprentice tasks. (nirik, 18:03:00) * two factor auth status (nirik, 18:08:13) * pam_url is nearly ready to package up and use. (nirik, 18:11:17) * LINK: https://github.com/mricon/totp-cgi <- url for cgi (nirik, 18:11:53) * Staging re-work status (nirik, 18:14:49) * done! (nirik, 18:14:57) * Applications status / discussion (nirik, 18:15:43) * will coordinate app01.dev re-install. (nirik, 18:22:10) * Upcoming Tasks/Items (nirik, 18:37:21) * 2012-05-08 to 2012-05-22 FINAL FREEZE (nirik, 18:37:36) * 2012-05-10 - drop inactive fi-apprentices (nirik, 18:37:36) * 2012-05-11 - Skvidal out. (nirik, 18:37:36) * 2012-05-22 - F17 release (nirik, 18:37:36) * 2012-06-01 - nag fi-apprentices. (nirik, 18:37:36) * 2011-06-03 - gitweb-cache removal day. (nirik, 18:37:36) * 2012-06-08 OOW: osuosl01.fedoraproject.org (nirik, 18:37:38) * 2012-06-17 OOW: sign-vault02.phx2.fedoraproject.org (nirik, 18:37:41) * 2012-06-21 to 2012-07-04 Kevin is off on trains and boats. (nirik, 18:37:42) * FTBFS run starting on buildvm05-08 and 02. (nirik, 18:41:12) * Open Floor (nirik, 18:43:11) * ansible is in fedora/epel, please play with it. (nirik, 18:45:32) Meeting ended at 18:47:52 UTC. Action Items ------------ Action Items, by person ----------------------- * **UNASSIGNED** * (none) People Present (lines said) --------------------------- * nirik (94) * skvidal (50) * abadger1999 (33) * lmacken (26) * wolfkit (9) * Neldogz (9) * threebean (7) * jds2001 (7) * smooge (5) * zodbot (4) * whiterhino (3) * relrod (3) * herlo (2) * SilentBob (1) * dgilmore (1) * ricky (0) * mdomsch (0) * CodeBlock (0) -- 18:00:56 <nirik> #startmeeting Infrastructure (2012-05-10) 18:00:56 <zodbot> Meeting started Thu May 10 18:00:56 2012 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:56 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:56 <nirik> #meetingname infrastructure 18:00:56 <zodbot> The meeting name has been set to 'infrastructure' 18:00:56 <nirik> #topic Robot Roll Call 18:00:56 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken dgilmore mdomsch threebean 18:00:56 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge threebean 18:01:01 * relrod here 18:01:02 * skvidal is here 18:01:04 * abadger1999 here 18:01:08 * wolfkit is here 18:01:08 <smooge> crrowbot here 18:01:12 * lmacken 18:01:15 <dgilmore> hola 18:01:20 * jds2001 18:01:21 <Neldogz> neldogz is here 18:01:42 <herlo> here 18:02:07 * nirik waves 18:02:13 * SilentBob is here 18:03:00 <nirik> #topic New folks introductions and Apprentice tasks. 18:03:00 <nirik> If any new folks want to give a quick one line bio or any apprentices 18:03:00 <nirik> would like to ask general questions, they can do so now. Anyone? 18:03:36 * nirik waits a min for any apprentices 18:03:49 <Neldogz> Sure, I can start.. I am new here , have about 10 years of general IT experience, strong within the server and networking side 18:04:19 <nirik> welcome Neldogz. You interested in sysadmin type stuff? or application development? 18:04:50 <Neldogz> thank you nirik, I am mostly interested in sysadmin work. monitoring, patching, fixing issues 18:05:09 <skvidal> Neldogz: which monitoring frameworks have you used? 18:05:13 <nirik> great. see me in #fedora-admin after the meeting and we can see about setting you up. 18:05:29 <skvidal> (yah - what nirik said) :) 18:05:42 <Neldogz> will do! skvidal, i have used PRTG and HP SIM for monitoring 18:05:53 <Neldogz> I am also currently learning Nagios 18:06:01 <nirik> any other new folks? or apprentices wanting to talk about tasks? 18:06:10 <whiterhino> I am new as well and about the same stats are Neldogz and would also like to get more involved in the system admin side 18:06:38 <nirik> welcome whiterhino 18:07:09 <whiterhino> I have used PRTG and nagios in the past, I currently use icinga and cacti 18:07:12 <whiterhino> thanks 18:07:40 <nirik> we use nagios here... and it needs some rework that various people have looked into. ;) 18:08:06 <nirik> anyhow, moving along. 18:08:13 <nirik> #topic two factor auth status 18:08:21 <nirik> I think we have some movement in this this week? 18:08:27 <wolfkit> this is coming along quite nicely :) 18:08:44 <wolfkit> I have the pam_url module escaping usernames and passwords, and has been committed to the pam_url source tree 18:08:52 <nirik> excellent. 18:08:58 <wolfkit> there is an odd segfault on 64-bit machines with it.. but I'm still looking in to that 18:09:24 <nirik> wolfkit: would you be willing also to package it up for Fedora/RHEL? 18:09:49 <wolfkit> hehe, kind of one step ahead of you, was playing around with a Fedora package for it last night :) 18:10:00 <wolfkit> haven't tested it on RHEL, can do though 18:10:06 <nirik> cool. I would be happy to review if you like. 18:10:26 <nirik> I think the cgi side also had a bunch of work this week.. 18:10:33 <wolfkit> great! will file a ticket when it is ready 18:11:07 <skvidal> and some more items 18:11:07 <nirik> thanks 18:11:13 <skvidal> mricon did new patches to the totp cgi 18:11:17 <nirik> #info pam_url is nearly ready to package up and use. 18:11:18 <skvidal> that does the checking of the keys 18:11:32 <skvidal> specifically he added an 'encrypt the seed with the users passphrase/pin' 18:11:53 <nirik> https://github.com/mricon/totp-cgi <- url for cgi 18:11:54 <skvidal> so the user can have the opened seed on their end - but we only have the one encrypted with their pin/passphrase 18:12:11 <skvidal> this means we never have any unencrypted goo sitting on our system somewhere 18:12:25 <skvidal> now a user could get compromised, of course 18:12:32 <skvidal> but that just gives them one key 18:12:34 <skvidal> not all of them 18:12:38 <skvidal> s/key/seed/ 18:12:43 <nirik> great. 18:12:56 <skvidal> it is terribly neat, actuall 18:12:56 <skvidal> y 18:13:19 <nirik> provided the pins aren't easily brute forceable... ie, not like 4 char number only. 18:13:55 <nirik> ok, anything else on this? or shall we move on? 18:14:14 <skvidal> that's it 18:14:23 <skvidal> nirik: I think the pins can be whatever... 18:14:40 <nirik> yeah, so we can make it something reasonable (although I don't think they need as much as passwords) 18:14:46 <skvidal> agreed 18:14:47 <nirik> anyhow, moving along... 18:14:49 <nirik> #topic Staging re-work status 18:14:50 <skvidal> nod 18:14:53 <nirik> this is done! 18:14:57 <nirik> #info done! 18:15:05 <skvidal> yay! 18:15:06 <abadger1999> Cool! 18:15:09 <skvidal> and there was much rejoicin 18:15:11 <herlo> +1 18:15:11 <wolfkit> awesome :D 18:15:12 <nirik> let me know if you run into any problems with staging in the new world order. 18:15:43 <nirik> #topic Applications status / discussion 18:15:58 <nirik> any applications news this week? abadger1999 / lmacken / threebean / pingou / CodeBlock 18:16:22 <lmacken> not really, just a lot of bodhi2.0 hacking. 18:16:33 <lmacken> I still need to find a couple of hours to wrap up the packages/tagger deployment 18:16:52 <abadger1999> none from me 18:16:53 <nirik> lmacken: cool. We should still setup a meeting sometime for 2.0 discussions... perhaps after f17 goes gold... 18:17:25 <lmacken> nirik: sounds good 18:18:06 <threebean> nothing major new with messaging. lots of package reviews (almost done!) 18:18:16 <nirik> oh, I'll note that now in staging we no longer have a rhel5 host... but the only rhel5 thing we have left is app07 to run old community. 18:18:29 <nirik> so, hopefully that can putter along until we retire it. 18:18:29 <skvidal> yay for no rhel5! 18:18:36 <lmacken> no more py2.4 :) 18:19:22 <lmacken> sadly, 2.6 is already feeling old :( 18:19:34 <nirik> I think bapp01 and xen04 (soon to be stopped) and app07 are all we have left now. so, thats good. 18:20:16 <abadger1999> (community and smolt's cron job) 18:20:29 <nirik> oh, and app01.dev I guess... 18:20:33 <nirik> we need to redo that sometime. 18:20:46 * relrod can do that, just let me know when is good 18:21:03 <nirik> I think there's some active development going on on it... but I could be wrong... abadger1999 ? 18:21:04 <smooge> lmacken, its ok.. we can move to 2.8 when it comes out 18:21:15 <abadger1999> yeah 18:21:38 <abadger1999> relrod: I'll have to coordinate that -- backup stuff and put on the new hosts 18:22:04 <abadger1999> relrod: I'd also like to split it into two hosts -- a fas-development server (for developing fas, not a fas for the dev env) 18:22:10 <nirik> #info will coordinate app01.dev re-install. 18:22:10 <abadger1999> relrod: and a pkgdb development server. 18:22:24 <threebean> lmacken: we might think about building bodhi2 on py3. pyramid is already compatible (and mako and sqlalchemy). 18:22:32 <abadger1999> relrod: But if you wanted to create the new hosts.. I could just start moving the code over? 18:22:54 <relrod> abadger1999: Sounds good. Will start on that tonight after finals or tomorrow. 18:22:54 <lmacken> threebean: good call, I'll create a py3 virtualenv for my local development. 18:23:00 <abadger1999> relrod: Cool. 18:23:19 <abadger1999> uhhh... 18:23:24 <nirik> is there a python3 in rhel/epel6? 18:23:29 <lmacken> yup 18:23:29 <abadger1999> Not sure if I'd support that unless 18:23:34 <threebean> :D 18:23:44 <abadger1999> we do something about the py3 version in RHEL5/6 vs Fedora 18:23:47 <abadger1999> err 18:23:54 <abadger1999> just epel6 18:24:16 <abadger1999> unles dmalcolm already updated? 18:24:19 * nirik doesn't see it. 18:24:27 * abadger1999 checks whats available 18:24:37 * abadger1999 doesn't either 18:24:53 <abadger1999> so... we'd need a whole new stack of deps... 18:25:01 <lmacken> I think he's working on it 18:25:04 <abadger1999> lots of packaging work if you want to go that route. 18:25:07 <lmacken> and yeah, it'll require a lot of packaging tweaks 18:25:12 <lmacken> but, inevitable tweaks. 18:25:14 <abadger1999> most spec files will have conditionals 18:25:31 <abadger1999> should be reasonable to get those changed. 18:26:08 <abadger1999> and since it's not in yet, you won't have to worry about it being a really old version of py3 18:26:10 <nirik> how do things like mod_wsgi handle multiple python versions? 18:26:29 <abadger1999> otoh... it does mean that we'll have to support both py2 and py3. 18:26:34 <abadger1999> nirik: They don't 18:26:42 <abadger1999> so we'd need two sets of app servers too.... 18:26:51 <nirik> yeah, so we would need more servers... 18:26:51 <nirik> yeah 18:26:56 <abadger1999> logisitcs of this seems more and more suspect. 18:27:03 <lmacken> yeah, true :) 18:27:08 <jds2001> why two sets? 18:27:16 <jds2001> instead of two instances? 18:27:23 <abadger1999> jds2001: TG1 and TG2 will never be able to shift over to python3 18:27:31 <abadger1999> because some of there deps are never going to port. 18:27:35 <nirik> jds2001: you mean multiple httpds? 18:27:39 <wolfkit> perhaps not necessarily building bodhi2 specifically for py3, but testing on py3 to ensure it's working / compatible with it for a later deployment? 18:27:40 <jds2001> nirik: yeah 18:27:44 <threebean> lmacken: build it with python-six so bodhi2 can run on py3 someday... someday. 18:27:47 <threebean> wolfkit: yeah 18:27:56 <lmacken> threebean: yeah, python-six is probably the way to go 18:28:06 <abadger1999> so until all of those are ported to different toolkits, we'd need to support modules for python2 and modules for python3. 18:28:15 <nirik> I suppose that could work, but it's always a bit odd to do that... ie, never know which server restarts when you do a restart, etc. 18:28:15 <abadger1999> <nod> 18:28:51 <abadger1999> threebean: six +1... That could be a good way forward -- also the next py3 release will make u"string" valid again 18:28:54 <abadger1999> so that will help 18:29:15 <threebean> :) 18:29:18 <jds2001> nirik: many moons ago at my previous job, I'd ran 20+ httpd instances on one box. 18:29:25 <nirik> crazy. ;) 18:29:30 <jds2001> and could restart each independently :) 18:29:38 <nirik> anyhow, worth investigating... we don't have to decide now. 18:29:46 <nirik> any other application news? 18:29:50 <lmacken> speakin of mod_wsgi, I've been pinging upstream about my hash seed patch. dead air :( 18:30:09 <lmacken> in the mean time, we could potentially do the apache init script hack to enable hash seed randomization 18:30:27 <lmacken> or, we could ship a custom mod_wsgi with my patch 18:30:55 * lmacken looks to see if our python in production already supports it 18:31:03 <nirik> probibly the init script hack would be easier as a hotfix... custom mod_wsgi would require us to keep on our toes updating. 18:31:10 <lmacken> yep 18:31:22 <nirik> but also note that we are in freeze... so we need to be careful what we change 18:31:25 <lmacken> I already put the diff for that init script patch in the trac ticket 18:31:35 <lmacken> yeah, probably will have to wait in that case 18:32:01 <nirik> ok. 18:32:03 <threebean> oh -- ryansb contributed an ircbot for fedmsg. fun! 18:32:28 <nirik> cool.... irc message busing. ;) 18:32:55 <lmacken> yeah, our python already supports -R... should be simple to enable it for our apps. 18:33:13 <nirik> excellent. 18:33:15 <lmacken> (even though there is a bug in the python hash randomization http://bugs.python.org/issue14621) 18:33:35 <lmacken> I doubt any of our apps rely on dict ordering, but we'll still want to test everything in staging first 18:34:18 <nirik> yeah, perhaps we could do that soon and deploy after freeze... or if it looks fine, just do a freeze break and push it out. 18:34:22 <lmacken> my init script patch is here: https://fedorahosted.org/fedora-infrastructure/ticket/3169 18:35:08 <nirik> yeah 18:35:22 <lmacken> oh wait, I was looking at the wrong python... looks like we don't have the patch in production 18:35:35 <nirik> ah, ;( 18:35:50 <lmacken> hopefully a yum update should pull it in though... not positive 18:36:03 <nirik> I thought I saw that update go by 18:36:23 <nirik> but I guess not 18:36:51 <nirik> lets discuss further outside of meeting I guess. I can look into where that update is. 18:37:01 <nirik> moving along... 18:37:21 <nirik> #topic Upcoming Tasks/Items 18:37:36 <nirik> #info 2012-05-08 to 2012-05-22 FINAL FREEZE 18:37:36 <nirik> #info 2012-05-10 - drop inactive fi-apprentices 18:37:36 <nirik> #info 2012-05-11 - Skvidal out. 18:37:36 <nirik> #info 2012-05-22 - F17 release 18:37:36 <nirik> #info 2012-06-01 - nag fi-apprentices. 18:37:36 <nirik> #info 2011-06-03 - gitweb-cache removal day. 18:37:38 <nirik> #info 2012-06-08 OOW: osuosl01.fedoraproject.org 18:37:41 <nirik> #info 2012-06-17 OOW: sign-vault02.phx2.fedoraproject.org 18:37:42 <nirik> #info 2012-06-21 to 2012-07-04 Kevin is off on trains and boats. 18:37:44 <nirik> thats what I have for upcoming. 18:37:59 <nirik> anyone want to schedule something or note some upcoming task/work? 18:38:06 <skvidal> I'll note something 18:38:28 <skvidal> I am getting buildvm-05 - 08 setup today - and I will be starting a ftbfs run on them 18:38:40 <skvidal> I'll be basing the run from lockbox01 into some disk space there 18:38:51 <nirik> skvidal: cool. Is this against rawhide I assume? 18:38:56 <skvidal> yeah it will be 18:39:02 <Neldogz> nirik: I belive we can close easyfix ticket 3231. After the restart of the unbound service, nagios appears to be performing the check successfully. https://admin.fedoraproject.org/nagios/cgi-bin//extinfo.cgi?type=2&host=unbound-telia01&service=Unbound+443%2Ftcp 18:39:39 <nirik> Neldogz: yeah, odd. It was doing it for a while, then stopped. ;) Oh well. 18:40:02 <nirik> skvidal: anything we need to know if we need to stop it or the like? 18:40:12 <skvidal> nirik: no 18:40:15 <Neldogz> I wish we could pinpoint what exactly was causing the problem. 18:40:22 <skvidal> nirik: just kill the processes running as me on lockbox01 18:40:31 <skvidal> nirik: and you can either kill or reboot buildvmhost-02 18:40:42 <skvidal> it might eat up some disk space on lockbox01 18:40:48 <skvidal> I'm looking to make sure that won't happen now 18:40:48 <nirik> Neldogz: me too. oh well, we will see if it happens again I guess. 18:40:53 <nirik> skvidal: ok. 18:41:12 <skvidal> nirik: hmm - not a lot of disk there, is there... 18:41:12 <nirik> #info FTBFS run starting on buildvm05-08 and 02. 18:41:18 <skvidal> not 02 18:41:25 <smooge> question.. when does skvidal get back 18:41:31 <skvidal> buildvm-05 -> 08 - which are all on buildvmhost-02 18:41:36 <skvidal> smooge: I'll be back monday 18:41:39 <nirik> ah, right, sorry. Could be more clear there. 18:41:42 <skvidal> smooge: my little brother is graduating from college 18:41:50 <skvidal> so I'm away for that this weekend 18:41:55 <jds2001> skvidal: mine just did :) 18:42:00 <skvidal> I should be available via phone some of the time 18:42:01 <skvidal> jds2001: :) 18:42:07 <skvidal> but I wouldn't bet on it 18:42:12 <smooge> you kids 18:42:36 <Neldogz> nirik: can someone that has access re-enable the notifications for the tls/ssh dns check to unbound-telia01 18:42:47 <nirik> Neldogz: yeah, I can see after the meeting. 18:42:57 <Neldogz> cool 18:43:11 <nirik> #topic Open Floor 18:43:16 <nirik> anyone have items for open floor? 18:43:35 <skvidal> nirik: I have one item, maybe? 18:43:41 <nirik> sure, shoot... 18:43:45 <skvidal> ansible made it into epel-6 and fedora now as pkgs 18:43:51 <skvidal> so it is easier for folks to play/test with 18:43:56 <skvidal> it is still evolving 18:44:03 <skvidal> but I wanted to encourage folks to mess with it 18:44:16 <nirik> cool. 18:44:16 <skvidal> and complain baout things if you find issues 18:44:16 * nirik makes a note to try it out here. 18:44:23 <smooge> will be messing with it next week as I redeploy my home system 18:44:25 <skvidal> I'm using ansible's api to drive the ftbfs stuff 18:44:38 <skvidal> and using the playbooks to setup the buildvm boxes I'm doing today 18:44:48 <skvidal> but more eyes/complaints are good 18:44:51 <skvidal> thx 18:45:32 <nirik> #info ansible is in fedora/epel, please play with it. 18:46:21 <nirik> ok, anything else or shall we call it a meeting? 18:47:49 <nirik> ok, thanks for coming everyone! 18:47:52 <nirik> #endmeeting
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
