============================================ #fedora-meeting: Infrastructure (2012-04-26) ============================================ Meeting started by nirik at 18:00:01 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2012-04-26/infrastructure.2012-04-26-18.00.log.html . Meeting summary --------------- * Robot Roll Call (nirik, 18:00:01) * New folks introductions and Apprentice tasks. (nirik, 18:03:53) * two factor auth status (nirik, 18:05:36) * Staging re-work status (nirik, 18:06:14) * Applications status / discussion (nirik, 18:06:43) * we moved gather-easyfix into production. Thanks pingou! (nirik, 18:07:39) * tagger and packager deployment is almost done. (nirik, 18:08:01) * ACTION: nirik to follow up on census progress. (nirik, 18:16:05) * dgilmore working on composedb app. (nirik, 18:19:11) * Upcoming Tasks/Items (nirik, 18:20:03) * 2012-04-29 to 2012-05-03 - Kevin out in the wilds of NM (nirik, 18:20:11) * 2012-05-01 to 2012-05-15 - F17 Final Freeze. (nirik, 18:20:12) * 2012-05-01 - nag fi-apprentices. (nirik, 18:20:12) * 2011-05-03 - gitweb-cache removal day. (nirik, 18:20:12) * 2012-05-09 - Check if puppet works on f17 yet. (nirik, 18:20:12) * 2012-05-10 - drop inactive fi-apprentices (nirik, 18:20:12) * 2012-05-15 - F17 release (nirik, 18:20:14) * Open Floor (nirik, 18:23:37) Meeting ended at 18:36:09 UTC. Action Items ------------ * nirik to follow up on census progress. Action Items, by person ----------------------- * nirik * nirik to follow up on census progress. * **UNASSIGNED** * (none) People Present (lines said) --------------------------- * nirik (70) * abadger1999 (18) * mdomsch (18) * skvidal (15) * dgilmore (6) * zodbot (4) * pingou (3) * lmacken (2) * threebean (2) * smooge (2) * codemaniac (1) * tibbs (1) * ianweller (1) * ricky (0) * CodeBlock (0) -- 18:00:01 <nirik> #startmeeting Infrastructure (2012-04-26) 18:00:01 <zodbot> Meeting started Thu Apr 26 18:00:01 2012 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:01 <nirik> #meetingname infrastructure 18:00:01 <zodbot> The meeting name has been set to 'infrastructure' 18:00:01 <nirik> #topic Robot Roll Call 18:00:01 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken dgilmore mdomsch threebean 18:00:01 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge threebean 18:00:09 * abadger1999 here 18:00:18 <tibbs> Wow, that time already? 18:00:47 * mdomsch has divided attention, as usual 18:00:55 <codemaniac> \o all 18:01:08 * ianweller here 18:01:17 <nirik> morning everyone. 18:01:24 * skvidal is here 18:01:26 <skvidal> or something 18:02:33 <nirik> we are working on hosted migration, which didn't go all that smoothly... ;( So, probibly this will be a short meeting. ;) 18:03:34 * threebean is here 18:03:53 <nirik> #topic New folks introductions and Apprentice tasks. 18:03:54 <nirik> If any new folks want to give a quick one line bio or any apprentices 18:03:54 <nirik> would like to ask general questions, they can do so now. Anyone? 18:05:24 <nirik> ok... moving on then 18:05:36 <nirik> #topic two factor auth status 18:05:38 <nirik> any news here? 18:05:44 <skvidal> no 18:05:48 <skvidal> think about this week :) 18:06:08 <nirik> yeah, no kidding. ;) 18:06:14 <nirik> #topic Staging re-work status 18:06:26 <nirik> I am waiting on this until we have bapp01 retired... 18:06:31 <nirik> so, hopefully soonish. 18:06:43 <nirik> #topic Applications status / discussion 18:06:47 <nirik> any application news? 18:07:05 <nirik> abadger1999 / lmacken / threebean / pingou / CodeBlock 18:07:18 <lmacken> threebean and I are almost done with the proper tagger & packages deployment 18:07:18 <abadger1999> been working on smolt all week :-( 18:07:19 <threebean> lmacken pushed tagger into production 18:07:36 <abadger1999> I'm getting ready to toss it out the airlock. 18:07:39 <nirik> #info we moved gather-easyfix into production. Thanks pingou! 18:07:41 <pingou> working on hyperkitty and kittystore now :) 18:07:50 <pingou> thank you nirik for taking care of this :) 18:08:01 <nirik> #info tagger and packager deployment is almost done. 18:08:08 <lmacken> got quite a few emails today about people wanting to help fix bugs in bodhi, thanks to the easyfix stuff :) 18:08:15 <pingou> nice! :) 18:08:21 <nirik> info smolt is a pile. 18:08:27 <nirik> oops... forgot the #. ;) 18:08:55 <nirik> abadger1999: so where is it now? runs on app07, but poorly/ 18:09:16 <abadger1999> nirik: I'm setting it up in pupet to run on app07 18:09:29 <abadger1999> but it's generating a lot of 0 length files for stats. 18:09:33 <abadger1999> otoh, I looked on bapp01 18:09:40 <abadger1999> and although there's less 0 length files... 18:09:43 <nirik> abadger1999: some of them always were 0 I thought? 18:09:51 <abadger1999> over half of hte files there are 0 length as well. 18:10:09 <nirik> yeah. 18:10:17 <abadger1999> so I'm not sure if this is new or if there's some timing issue or what. 18:10:43 <nirik> does the main stats.html one appear? 18:10:44 <abadger1999> at this point, I'd like to just push it out since we already want to get rid of smolt. 18:11:00 <nirik> yeah. 18:11:13 <nirik> push out whats working, call it good, work on announcing sunsut 18:11:15 <nirik> sunset even 18:11:18 <abadger1999> -rw-r--r-- 1 root root 0 Apr 26 18:01 stats.html 18:11:23 <nirik> :( 18:11:37 <abadger1999> vs 18:11:39 <abadger1999> -rw-r--r-- 1 root root 191843 Apr 25 00:18 stats.html 18:11:48 <nirik> yeah, thats the page many people hit... 18:11:51 <abadger1999> the latter is what's on bapp01; former is what's on app07 18:12:35 <nirik> so, perhaps we take the last bapp01 one, slap a "This data is from 2012-04-25 and will not be updated, please see the census project" and leave it static? 18:12:46 <abadger1999> We could do that. 18:12:55 <abadger1999> disable the cron job 18:13:10 <abadger1999> okay. I'll do that. 18:14:04 <nirik> some people might not like that, but if we are sunsetting it anyhow... 18:14:41 <nirik> any objections? other ideas? 18:14:46 <abadger1999> What's the latest update on using census? 18:15:17 <nirik> I've not heard much recently. 18:16:05 <nirik> #action nirik to follow up on census progress. 18:16:13 <nirik> any other general application news? 18:16:30 * dgilmore is about to start on a composedb app 18:16:56 <nirik> dgilmore: cool. Are you doing that yourself? or working with others? 18:17:29 <nirik> anything we can do to assist at this point? 18:17:42 <dgilmore> nirik: just myself 18:18:02 <dgilmore> trying to work out whether to use Django TurboGears or something else 18:18:17 <dgilmore> im hoping that it will be ready for f18 dev 18:18:32 <dgilmore> development 18:18:37 <dgilmore> i.e when we branch 18:18:54 <nirik> cool. Might write up a plan/outline... might be some folks will want to help out with it. 18:19:11 <nirik> #info dgilmore working on composedb app. 18:20:03 <nirik> #topic Upcoming Tasks/Items 18:20:11 <nirik> #info 2012-04-29 to 2012-05-03 - Kevin out in the wilds of NM 18:20:12 <nirik> #info 2012-05-01 to 2012-05-15 - F17 Final Freeze. 18:20:12 <nirik> #info 2012-05-01 - nag fi-apprentices. 18:20:12 <nirik> #info 2011-05-03 - gitweb-cache removal day. 18:20:12 <nirik> #info 2012-05-09 - Check if puppet works on f17 yet. 18:20:12 <nirik> #info 2012-05-10 - drop inactive fi-apprentices 18:20:14 <nirik> #info 2012-05-15 - F17 release 18:20:21 <nirik> note that I will be out early next week... 18:20:36 <smooge> I can cover freeze tickets and such 18:20:37 <nirik> so if anyone needs anything from me before then, catch me today or friday. 18:21:21 <nirik> anything else upcoming folks want to schedule or note? 18:21:43 <nirik> oh, did I get the freeze wrong. 18:22:31 <nirik> yeah, it's a week out from that on the 08th 18:23:37 <nirik> #topic Open Floor 18:23:43 <nirik> anyone have anything for open floor? 18:24:11 * nirik will close out in a minute if nothing comes up 18:24:17 <mdomsch> MM brokenness? 18:24:24 <mdomsch> sorry I was out of pocket on Tuesday 18:24:59 <mdomsch> not sure why it chose that day to blow up and not before, if there really were a bunch of invalid values in host.bandwidth_int 18:24:59 <nirik> mdomsch: oh yeah. 18:25:07 <nirik> us either. ;( 18:25:13 <nirik> that did seem to fix it though... 18:25:14 <mdomsch> I have a patch in MM 1.4 to force that column NOT NULL 18:25:30 <nirik> we also hotfixed the script to error on those moving forward. 18:25:38 <nirik> so if it happens again we should be able to see it right away. 18:25:39 <mdomsch> I suspect there are similar kinds of failures latent 18:26:07 <skvidal> fun! 18:26:17 <nirik> on the plus side, I have a much better understanding now of how all the parts fit and work in mm. ;) 18:26:26 <mdomsch> well, that's something :-) 18:26:48 <mdomsch> thanks all for scrambling to fix it 18:26:51 <skvidal> and to that end - there sure seem to be a lot of moving parts in MM 18:27:19 <mdomsch> skvidal: yes... 18:27:24 <skvidal> I added a hotfix to MM so that if we start seeing tracebacks resulting in a BAD Request again 18:27:30 <skvidal> we'll at least get a piece of the traceback 18:28:21 <mdomsch> skvidal: it's the result of being our highest request volume application, and my first attempt at a sizeable program in python 18:28:30 <skvidal> I wasn't chastising you 18:28:45 <skvidal> I know exactly how things evolved - it's not something to be blamed for 18:29:00 <nirik> to be fair, all our apps logging is... not ideal. ;) 18:29:02 <skvidal> as nirik said - it took a fair amount of poking for us to figure out where it was broken 18:29:09 <mdomsch> sqlobject caching, python threading, conspired to make me split out the mirrorlist_server into its own cache, separate from the database 18:29:24 <mdomsch> and yes, almost no logging 18:29:44 <skvidal> I guess what I would like to see happen is this 18:29:55 <skvidal> could we force some bad data into the db 18:30:02 <skvidal> and see where things go sideways? 18:30:14 <skvidal> b/c as it is we were scrambling to figure out on which system things actually started breaking 18:30:23 <nirik> we could do that in stg? 18:30:29 <mdomsch> sure... 18:30:55 <nirik> in our copious spare time. ;) 18:30:57 <mdomsch> wouldn't need a whole lot of entries 18:31:25 <mdomsch> I've been doing my MM 1.4 work lately in a private instance that I hope to make front linux.dell.com for its few mirrors 18:31:39 <mdomsch> it's been handy for finding bugs, but it's not a full fuzzing approach 18:32:49 <mdomsch> and yes, exposing the tracebacks somehow, instead of eating them, would be useful for debugging 18:33:00 * nirik nods. 18:33:06 <nirik> ok, anything else, or shall we call it a meeting? 18:33:59 <mdomsch> call it 18:35:11 <smooge> hit the button nirik-frank 18:36:09 <nirik> #endmeeting
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
