============================================ #fedora-meeting: Infrastructure (2014-07-10) ============================================ Meeting started by nirik at 18:00:25 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2014-07-10/infrastructure.2014-07-10-18.00.log.html . Meeting summary --------------- * aloha (nirik, 18:00:26) * New folks introductions and Apprentice tasks (nirik, 18:02:07) * Applications status / discussion (nirik, 18:09:48) * application updates this week: elections, pkgdb2, fedoauth, bodhi (prod) (nirik, 18:14:02) * in stg: fedora-tagger, datagrepper, taskotron (almost) (nirik, 18:14:33) * development, cnucnu web renamed to anitya (nirik, 18:14:49) * Sysadmin status / discussion (nirik, 18:19:43) * smooge working on log01 migration (nirik, 18:20:34) * relrod working on noc migration (nirik, 18:20:41) * nagios/alerts recap (nirik, 18:25:41) * Upcoming Tasks/Items (nirik, 18:30:30) * LINK: https://apps.fedoraproject.org/calendar/list/infrastructure/ (nirik, 18:30:30) * smooge and nirik will be out at the phx2 datacenter july 22 to 24th (nirik, 18:31:08) * threebean on vacation july 26 through august 2nd (threebean, 18:32:03) * relrod away at a conference July 22-23 (relrod, 18:34:23) * Open Floor (nirik, 18:35:14) Meeting ended at 18:44:01 UTC. Action Items ------------ Action Items, by person ----------------------- * **UNASSIGNED** * (none) People Present (lines said) --------------------------- * nirik (75) * pingou (26) * ootbro (17) * threebean (16) * tflink (16) * zodbot (8) * relrod (8) * oddshocks (6) * hammad_ (5) * lmacken (5) * selvakumarm (3) * smooge (3) * abadger1999 (2) * lanica (2) * lalit_ (1) * webpigeon (1) * janeznemanic (1) * charul_ (1) * dgilmore (1) * puiterwijk (0) * mdomsch (0) -- 18:00:25 <nirik> #startmeeting Infrastructure (2014-07-10) 18:00:25 <zodbot> Meeting started Thu Jul 10 18:00:25 2014 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:25 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:26 <nirik> #meetingname infrastructure 18:00:26 <nirik> #topic aloha 18:00:26 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk 18:00:26 <zodbot> The meeting name has been set to 'infrastructure' 18:00:26 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean 18:01:15 <tflink> .hellomynameis tflink 18:01:16 <zodbot> tflink: tflink 'Tim Flink' <tflink@xxxxxxxxxx> 18:01:29 * threebean 18:01:29 <relrod> here 18:01:31 * lmacken 18:01:32 <janeznemanic> hi 18:01:36 <selvakumarm> Hi 18:01:37 * hammad_ is here 18:01:47 * lalit_ here 18:01:50 <nirik> morning everyone. 18:01:54 * charul_ here 18:02:00 <ootbro> or good afternoon? (or evening?) 18:02:02 <dgilmore> afternoon 18:02:03 * pingou 18:02:06 <pingou> evening 18:02:07 <nirik> #topic New folks introductions and Apprentice tasks 18:02:08 <relrod> sitting at the park tethering - it's a beautiful day, 80F :D 18:02:13 <nirik> it's always morning on irc. ;) 18:02:31 <nirik> any new folks like to introduce themselves or apprentices with questions, comments or ideas? 18:02:41 * lanica is here for the meeting. 18:03:30 <ootbro> nirik: any (final?) comments on my last e-mail about the map? I plan to start digging through the zone files for proxy info after the meeting. 18:04:07 <nirik> ootbro: I don't think I had any more... I've been very swamped of late tho. ;) 18:04:27 <oddshocks> Here 18:04:28 <ootbro> I understand .... no problem..... 18:04:53 <threebean> ootbro: thanks for working on it. 18:05:01 <threebean> it will be useful to have 18:05:13 <nirik> indeed. ;) 18:05:15 <ootbro> you're welcome ...... yeah..... it will answer some of my still-newbie questions 18:05:30 <smooge> good day to you all 18:06:40 <nirik> ok, any other new folks or apprentice questions? moving on in a few... 18:06:58 <ootbro> for everyone else.... the first part of the work is a basic (pictoral) diagram that shows all the data centers (with location notations), with the 10.5 and 192 networks tagged. 18:07:39 <ootbro> .... and an indication of which which centers have a proxy and what services are forwarded through each proxy 18:08:04 <ootbro> (It's the last part that will have me digging through the DNS zone files.) 18:08:10 <ootbro> <end> 18:08:31 <selvakumarm> ootbro: where will i find the diagram ?? 18:08:47 <ootbro> I plan to put it somewhere on the fp.o wiki 18:09:47 <selvakumarm> please do the same and circulate the location.. thanks 18:09:48 <nirik> #topic Applications status / discussion 18:09:55 <nirik> any application news this week or upcoming? 18:10:01 <pingou> new elections in prod 18:10:07 <pingou> new pkgdb2 in prod 18:10:29 <pingou> cnucnuweb has been renamed anitya, dev instance updated: http://209.132.184.188/anitya/ 18:10:30 <lmacken> new bodhi went to prod last week I think, minor bug fixing around python-bugzilla 18:10:37 <tflink> taskotron staging is mostly deployed, waiting on a firewall rule change ticket before it's done 18:10:45 <pingou> (the backend was pretty much completely re-written) 18:10:52 <threebean> I've got two stuck in staging at the moment. 18:10:59 <pingou> busy week :) 18:11:02 <threebean> adimania's datagrepper work is good -> https://fedorahosted.org/fedora-infrastructure/ticket/4393 18:11:13 <threebean> but I'm having a hrad time getting it all set oh rhel7 18:11:15 <nirik> cool. lots of stuff. 18:11:28 <nirik> tflink: whats left for prod? that vpn access... and ? 18:11:32 <threebean> also, fedora-tagger. new release in staging. one of the features broke on postgres so we're holding off for the moment. 18:11:43 <oddshocks> Fedimg is almost production ready. fedimg01 and fedimg01.stg machines were set up + playbooks thanks to threebean. 18:11:50 <lmacken> threebean: related to fedmsg (wrt rhel7 issues) ? 18:12:04 <threebean> lmacken: nah, just lots of stuff that never got branched. 18:12:09 <lmacken> ah 18:12:11 <tflink> nirik: client setup, mostly. so a few more rhel7 installs, vm creations and hostname changes 18:12:21 * webpigeon sneaks in 18:12:26 <lmacken> threebean: you've gotta pave the way for us ;) 18:12:40 <pingou> +1, it's nice :) 18:12:53 <nirik> tflink: cool. Hopefully not too bad to get done. 18:13:25 <tflink> nirik: yeah, all stuff that has been done before in dev+stg so hopefully not too bad :) 18:13:41 <nirik> we also have a new fedoauth version in prod 18:14:02 <nirik> #info application updates this week: elections, pkgdb2, fedoauth, bodhi (prod) 18:14:33 <nirik> #info in stg: fedora-tagger, datagrepper, taskotron (almost) 18:14:49 <nirik> #info development, cnucnu web renamed to anitya 18:15:28 <nirik> anything else on the application front? 18:15:35 <pingou> I've been going through the pkgdb1 tickets and closing what could be, so we have 32 tickets left 18:15:44 <nirik> tflink: is there a deadline to get prod taskotron replacing autoqa? 18:15:45 <abadger1999> Yay! 18:15:50 <pingou> which are either still valid or which needs input from someone else :) 18:15:53 <nirik> excellent. 18:15:56 * pingou glances at abadger1999 ;-) 18:16:00 <abadger1999> pingou: Are we going to just migrate whatever is left at the end of hte purge? 18:16:07 <nirik> pingou: oh, we still need to orphan that one inactive maintainers packages. 18:16:18 <pingou> nirik: should work now from the pkgdb-cli in git 18:16:28 <pingou> server side it should work 18:16:31 * pingou crosses his fingers 18:16:43 <nirik> pingou: cool. I can try after the meeting? 18:17:00 <pingou> nirik: either that or tomorrow, I won't last long after the meeting 18:17:17 <tflink> nirik: I had wanted it to be by branch, honestly 18:17:23 <pingou> nirik: if you try, run it with --debug and if it fails I can look at fixing it tomorrow morning 18:17:28 <nirik> pingou: ok 18:18:01 <pingou> tflink: is there a target branch to get taskotron in prod and replacing autoqa? :) 18:18:17 <nirik> tflink: well, hopefully soon then. 18:18:25 <tflink> nirik: but realistically, I leave for vacation 2 weeks from tomorrow. I'd like to have it done with a week to spare before then if possible 18:18:32 <tflink> pingou: target branch? 18:18:37 <pingou> tflink: f21, f22? 18:18:48 <pingou> f19? (we're late for that one ^^) 18:18:50 <nirik> tflink: yeah, we go into freeze on the 22nd too, so ideally before then would be very good. 18:18:51 <tflink> pingou: f21 branch :-P 18:19:32 <nirik> ok... anything else on the application side? 18:19:43 <nirik> #topic Sysadmin status / discussion 18:19:54 <nirik> we have been moving more instances to rhel7 and ansible. 18:20:20 <nirik> we are down to 95 hosts in puppet. 18:20:30 <pingou> cool 18:20:34 <nirik> #info smooge working on log01 migration 18:20:41 <nirik> #info relrod working on noc migration 18:20:47 <relrod> I almost have nagios moved into ansible, will work on it more later today 18:20:48 <relrod> yes that 18:21:12 <nirik> cool. 18:21:21 <nirik> relrod: could you also at some point move noc03 off virthost02? 18:21:30 <nirik> vh02 is one of the ones we are going to be retiring 18:21:53 <nirik> perhaps vh17 would be a good place? 18:21:56 <relrod> nirik: hah. yeah, I was just using it for testing - it should hopefully be easy to just change the host and rerun the playbook 18:22:04 <nirik> yep 18:22:32 <relrod> I only picked vh02 because noc01 is currently there 18:22:52 <tflink> smooge: any news on racking for virthost-comm03.qa and qa09.qa? 18:22:53 <nirik> I moved/redid our rawhide and branched compose machines. After a few hiccups they are working fine now. 18:23:08 <nirik> tflink: they have been racked. ;) we just need to get them networked and installed. 18:23:10 <smooge> they are racked. I need to set them u 18:23:29 <nirik> smooge: oh, what happened to those retrace machines? where were we on them? 18:23:51 <smooge> I have asked michael friedman for tracking numbers 18:24:05 <nirik> might ping again, it's been a while since we heard. 18:25:41 <nirik> #topic nagios/alerts recap 18:25:56 <nirik> .tiny https://admin.fedoraproject.org/nagios/cgi-bin//summary.cgi?report=1&displaytype=3&timeperiod=last7days&smon=5&sday=1&syear=2014&shour=0&smin=0&ssec=0&emon=5&eday=15&eyear=2014&ehour=24&emin=0&esec=0&hostgroup=all&servicegroup=all&host=all&alerttypes=3&statetypes=2&hoststates=3&servicestates=56&limit=25 18:25:56 <zodbot> nirik: http://tinyurl.com/q8j48o9 18:26:26 <threebean> I like this meeting routine ;p 18:26:28 <nirik> so, typical telia stuff. I am working on moving everything off telia. 18:26:37 <nirik> threebean: should we adjust the notifs backlog? 18:26:46 <nirik> it seems to get hammered by big clumps? 18:26:55 <threebean> it does. and we could. 18:27:04 <pingou> .tiny https://admin.fedoraproject.org/collectd/bin/graph.cgi?hostname=notifs-backend01.phx2.fedoraproject.org;plugin=fedmsg;plugin_instance=hub;type=queue_length;type_instance=FMNConsumer_backlog;begin=-3000000 18:27:04 <zodbot> pingou: http://tinyurl.com/ocdbwf7 18:27:20 <threebean> I think the big one was the mass branch this past week. 18:27:39 <threebean> pingou: that goes too far back. it includes peaks that wash out the latest activity. 18:27:39 <nirik> yeah, then the secondary arches following that 18:27:58 <pingou> .tiny https://admin.fedoraproject.org/collectd/bin/graph.cgi?hostname=notifs-backend01.phx2.fedoraproject.org;plugin=fedmsg;plugin_instance=hub;type=queue_length;type_instance=FMNConsumer_backlog;begin=-900000 18:28:00 <zodbot> pingou: http://tinyurl.com/pjgob9a 18:28:01 <pingou> over 10 days 18:28:06 <nirik> yeah, our collectd has been acting up. 18:28:12 <threebean> hm, :/ 18:28:16 <nirik> I think it's due to new rhel7 hosts using a different collectd version 18:28:43 <nirik> we need to get a log01 up with the new server (which smooge is working on) 18:29:14 <nirik> the other thing I see is packages03 hitting oom lately. Not sure what the story is there. 18:30:26 <nirik> it's been a bit more quiet the last few days, so we can wait and see. 18:30:30 <nirik> #topic Upcoming Tasks/Items 18:30:30 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/ 18:30:38 <nirik> anyone have upcoming items they would like to note or schedule? 18:31:08 <nirik> #info smooge and nirik will be out at the phx2 datacenter july 22 to 24th 18:31:34 <nirik> Our freeze for alpha is supposed to start the 22nd, but I might make it start the 24th and give us time to get things down out there. 18:31:48 <lanica> What kind of tasks will you be working on? Just auditing or...? 18:31:48 <nirik> not sure how much distruptive we have pending tho 18:32:03 <threebean> #info threebean on vacation july 26 through august 2nd 18:32:19 <nirik> lanica: pulling old machines out, setting up some new storage, possibly doing firmware updates, checking whats connected where. 18:32:48 <nirik> we may be renetworking things if we get some new network space by then 18:33:27 <nirik> lanica: basically anything that we can do that needs hands on 18:34:14 <nirik> threebean: just in time to rest up before flock. ;) 18:34:23 <relrod> #info relrod away at a conference July 22-23 18:34:51 <relrod> Will have my phone though 18:34:54 <nirik> tis the season of summer travel. ;) 18:35:12 <threebean> :) 18:35:14 <nirik> #topic Open Floor 18:35:24 <nirik> anyone have anything for open floor? questions, comments, ideas? 18:36:06 <ootbro> one general question to the experienced admins ..... what are some good (free) resources for learning shell scripting and python? 18:36:39 <nirik> for python there's a 'dive into python' book thats supposed to be pretty good. 18:36:56 <oddshocks> A lot of folks love Learn Python The Hard Way which is free online 18:37:02 <hammad_> Hello ,I am hammad, working on fedora college. We are almost done with it, and pushed the code to fedora-infra. 18:37:09 <hammad_> Working on GsoC 18:37:27 <pingou> hammad_: btw, you may want to see with your mentor if you want to keep the repo being a 'fork' 18:37:39 <pingou> in the future, for pull-request it can quickly become annoying 18:37:55 <hammad_> yeah, let me talk with him ,maybe we can move it permenantly 18:37:56 <ootbro> oddshocks: the price is certainly right -- and I wouldn't have another book jammed into the home office. 18:37:58 <hammad_> to fedoraa infra 18:38:17 * pingou has the fedora-tagger example in mind 18:38:23 <tflink> ootbro: "learn python the hard way" is a good python resource 18:38:41 <tflink> or "the hitchiker's guide to python" 18:38:58 <tflink> but that's more of a reference, lpthw is designed to teach 18:39:11 <threebean> oo, I recommend http://pyvideo.org/ 18:39:22 <ootbro> tflink: oddshocks: with both of you recommending lpthw, I'll probably go that route at first. 18:39:42 <tflink> ah, I missed that it was already mentioned :) 18:39:48 <nirik> I can't think of any shell books off hand. 18:39:49 <ootbro> not a problem... :) 18:40:00 * nirik doesn't learn well from books... I learn better by doing. 18:40:00 <tflink> there's a 'learn shell the hard way' as well 18:40:38 <oddshocks> tflink: great minds 18:40:45 <nirik> anyhow, if nothing else, will close out in a minute or so. 18:40:56 <oddshocks> ootbro: IMO the best resource for shell scripting is sometimes Stack Exchange :P 18:40:58 <ootbro> nirik: I do, too, but I need to do a little reading first.... plus I'm probably several months away from needing to learn either 18:41:21 <oddshocks> ootbro: just search "how do I do the thing in <language>" and it's probably already been answered, and if not, you can ask the question yourself 18:41:30 <nirik> that too 18:41:31 <tflink> nvm, not a shell book - more of a cli crash course 18:41:51 <ootbro> my thanks to all for the suggestions. I'll cull the URLs and titles from the meeting minutes. 18:42:05 <tflink> I've always liked http://www.tldp.org/LDP/abs/html/ for bash, though 18:42:20 <nirik> ah, tldp... there's something I have not seen in a while. 18:42:29 <ootbro> I can program in other languages, but know now shell or python at this point. 18:42:34 <ootbro> "know no" 18:43:58 <nirik> alright, thanks for coming everyone! 18:44:01 <nirik> #endmeeting
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
