============================================ #fedora-meeting: Infrastructure (2014-06-19) ============================================ Meeting started by nirik at 18:12:35 UTC. The full logs are available at fedora-meeting/2014/fedora-meeting.2014-06-19-18.12.log.html . Meeting summary --------------- * aloha (nirik, 18:12:36) * New folks introductions and Apprentice tasks. (nirik, 18:13:27) * Applications status / discussion (nirik, 18:16:56) * LINK: https://apps.fedoraproject.org/github2fedmsg/ (threebean, 18:17:07) * fedoauth is updated in stg. Please test it against as many sites as you can... it's going to production this weekend. (nirik, 18:17:21) * copr upgrade also coming up tomorrow. There will be a short outage. (nirik, 18:23:56) * Sysadmin status / discussion (nirik, 18:31:25) * rhel7 setup almost in place to start moving machines to it. Will work on kickstarts next week. (nirik, 18:36:01) * ansible using ssh pipeline/controlpersist on lockbox, should be much faste.r (nirik, 18:36:18) * smooge and nirik will be heading out to phx2 datacenter in late july... 22nd to 25th (nirik, 18:36:44) * nagios recap (nirik, 18:37:48) * Upcoming Tasks/Items (nirik, 18:41:39) * LINK: https://apps.fedoraproject.org/calendar/list/infrastructure/ (nirik, 18:41:39) * Open Floor (nirik, 18:42:47) Meeting ended at 18:46:03 UTC. Action Items ------------ Action Items, by person ----------------------- * **UNASSIGNED** * (none) People Present (lines said) --------------------------- * nirik (63) * oddshocks (19) * tflink (14) * roshi (10) * threebean (7) * ootbro (7) * ursabot (6) * smooge (5) * danofsatx-work (3) * puiterwijk (1) * randomuser (1) * abadger1999 (1) * lmacken (1) * ausmarton (1) * janeznemanic (1) * mpduty (1) * lorddemon (1) * mdomsch (0) * pingou (0) * relrod (0) * dgilmore (0) -- 18:12:35 <nirik> #startmeeting Infrastructure (2014-06-19) 18:12:35 <ursabot> Meeting started Thu Jun 19 18:12:35 2014 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:12:35 <ursabot> Useful Commands: #action #agreed #help #info #idea #link #topic. 18:12:35 <nirik> #meetingname infrastructure 18:12:35 <ursabot> The meeting name has been set to 'infrastructure' 18:12:36 <nirik> #topic aloha 18:12:36 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk 18:12:36 <ursabot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean 18:12:46 <oddshocks> woohoo 18:12:52 <nirik> who all is around for a late starting, net splitting meeting? ;) 18:12:53 * oddshocks present and accounted for 18:13:09 * ausmarton here 18:13:15 * puiterwijk is around but busy 18:13:17 <abadger1999> Greetings 18:13:27 <nirik> #topic New folks introductions and Apprentice tasks. 18:13:28 * threebean is here 18:13:32 * danofsatx-work is nuking sketty 18:13:38 * lmacken 18:13:44 <danofsatx-work> and there's the beep... 18:13:46 <nirik> any new folks around who want to introduce themselves? or apprentices with questions, comments or items to note? 18:14:10 * roshi is here 18:14:18 <janeznemanic> hi 18:14:20 * randomuser is milling around 18:14:30 <roshi> just lurking trying to get a better understanding of our infra :) 18:14:48 <roshi> learn what I can about how we do things on the infra side 18:15:06 <oddshocks> oh god 18:15:07 <oddshocks> why 18:15:09 <roshi> :) 18:15:20 <oddshocks> 2x netsplit combo 18:15:44 <oddshocks> lost kevin and patrick again 18:15:46 * roshi thought you were meaning "oh god why?" to checking out our infra 18:15:56 <oddshocks> roshi: oh no, it's not that bad, I promise ;) 18:16:21 <oddshocks> we just lost about 30 connections to netsplit for the second time in 5 mins 18:16:22 <roshi> :D 18:16:28 <oddshocks> same ones 18:16:39 <nirik> ursabot: ping 18:16:39 <ursabot> pong 18:16:39 <oddshocks> and they're back 18:16:39 <nirik> yeah... 18:16:42 <nirik> freenode isn't too staable today 18:16:56 <nirik> #topic Applications status / discussion 18:17:00 * tflink shows up late, apologies 18:17:01 <nirik> any applications news? 18:17:07 <threebean> https://apps.fedoraproject.org/github2fedmsg/ 18:17:20 <threebean> pushed that out this past week. 18:17:21 <nirik> #info fedoauth is updated in stg. Please test it against as many sites as you can... it's going to production this weekend. 18:17:42 <nirik> threebean: pretty cool. Is there any kind of overview of projects connected to it? or it's all per user? 18:18:03 <threebean> all per user, but we could make an overview dashboard thing if it would be useful 18:18:21 <nirik> just my curiosity, dunno how important that is. 18:18:44 * threebean nods 18:18:58 <oddshocks> Fedimg is making its strides! Working out some libcloud issues with #libcloud, but things are looking good! Got all the fedmsg meta code merged in to the fedmsg meta infra repo... docs started on RTFD... makin' progress. that's all for me 18:19:20 <nirik> oddshocks: cool. 18:19:49 <threebean> oddshocks: your fedmsg.meta stuff is released too and on its way to the updates-testing yum repos 18:19:52 <nirik> oddshocks: oh, I wonder... do we want to look at having it upload to the infrastructure private cloud too? dunno how useful it would be, but we could look into that if it is. 18:20:23 <oddshocks> threebean: awesom! 18:20:34 <oddshocks> nirik: I 100% do 18:20:58 <nirik> should just need an account that can upload I would suspect. 18:22:10 <nirik> any other app news? 18:22:10 <oddshocks> nirik: our goal is definitely automatic uploads to external AND internal cloud providers. I was originally uploading to our internal Fedora FTP server too but I was told that is The Wrong Thing To Do and that I should just have nightly builds uploaded to the internal fedora FTP server, but I am 100% down for internal cloud image uploads 18:22:35 <oddshocks> nirik: but any info anyone can get me on getting the images on our internal cloud server is super-helpful 18:22:38 <oddshocks> and can be done with ease 18:22:42 <oddshocks> :) 18:23:04 <nirik> oddshocks: sure, we can talk out of meeting on it sometime... if we do nightlies we should also expire/delete older ones (ie, I don't think we want to keep them forever) 18:23:17 <oddshocks> Definitely. Sounds good. 18:23:56 <nirik> #info copr upgrade also coming up tomorrow. There will be a short outage. 18:25:03 * tflink doesn't know if this counts as application status (or if it's too far out) but we're hoping to get taskotron farther deployed this week and next 18:25:10 <nirik> cool. 18:25:57 <nirik> tflink: oh, there's an issue with resultsdb-stg01... I tried to put it in the stg group, but it can't talk to stg fas. ;( I have a RHIT ticket to fix that... 18:26:08 <nirik> we do want the *stg* qa stuff to be in infra stg right? 18:26:20 <nirik> (for fas, and talking to proxy01.stg and koji01.stg, etc?) 18:27:30 <tflink> nirik: resultsdb doesn't have auth right now, so fas isn't a huge issue 18:27:49 <nirik> well, for local accounts/sudo 18:27:57 <tflink> yeah, I wasn't thinking about that 18:28:13 <tflink> I'm fine eihter way, stg means that my yubikey doesn't work but I can deal with that 18:28:13 <nirik> I can get it sorted out. I would expect we want them to interact with stg 18:28:22 <nirik> ok 18:28:39 <tflink> unless the 2fa issue with stg fas was worked out 18:28:41 <nirik> hopefully the proxy stuff works for you now. 18:28:49 <tflink> er 2fa with yubikey and stg fas 18:29:02 <nirik> I'm not sure which issue? wanting one yubikey to work with the same slot for both? 18:29:15 <tflink> for a while, yubikey couldn't be used with fas stg for 2fa 18:29:28 <tflink> I was told to use google authenticator instead 18:29:40 <nirik> huh, I haven't tried, but I can... 18:30:07 <tflink> I hooked the load testing setup up to resultsdb-stg for a test 18:30:21 <tflink> so it'll get tested out more shortly, even if the names don't quite match :) 18:30:38 <tflink> ie, taskotron-dev.fp.o working with taskotron.stg.fp.o/resultsdb 18:30:42 <nirik> ok. 18:31:25 <nirik> #topic Sysadmin status / discussion 18:31:33 <nirik> ok, lets see on the sysadmin side... 18:31:58 <nirik> I have all our ducks in a row finally for rhel7 setup... so we will be setting up kickstarts and moving to rhel7 instances soon where we can. 18:32:32 <nirik> We will have to see where we are application stack wise before moving any of the application instances over. 18:32:44 <nirik> and we also talked about python3 sometime. 18:32:45 * oddshocks nods 18:33:04 <lorddemon> \me supee 18:33:34 <nirik> but a lot of simpler stuff can move to 7 sooner... like download servers, virthosts, the mailman instances (so we can deploy hyperkitty with rhel7 instead of f19) 18:33:55 * mpduty is here 18:34:01 <ootbro> nirik: does it look like things are back on track? 18:34:04 <ootbro> I'm still hanging on 18:34:04 <ootbro> roshi: I'm starting work on getting that documentation created 18:34:11 <ootbro> \ still watching over the meeting attempting to start 18:34:28 <nirik> ootbro: we started and have had a few minutes with no net split. ;) 18:34:45 <roshi> which documentation ootbro? 18:34:51 * roshi doesn't recognize your nick 18:35:07 <ootbro> nirik: thanks.. I've been watching the lines scroll past here 18:35:16 <nirik> I've gotten ansible on lockbox using controlpersist and ssh pipelining, so playbook runs should be much faster. This is using a openssh scratch build with that support. 18:35:25 <ootbro> roshi: documentation on how the Fedora infrastructure is laid out 18:35:43 <roshi> ah, sweet 18:36:01 <roshi> sorry, I ask a lot of people for documentation on things and can't keep it all straight in my head :) 18:36:01 <nirik> #info rhel7 setup almost in place to start moving machines to it. Will work on kickstarts next week. 18:36:18 <nirik> #info ansible using ssh pipeline/controlpersist on lockbox, should be much faste.r 18:36:24 <ootbro> not a problem..... I picked up the project a few weeks ago, with the announcement on here 18:36:44 <nirik> #info smooge and nirik will be heading out to phx2 datacenter in late july... 22nd to 25th 18:37:48 <nirik> #topic nagios recap 18:38:19 <nirik> ?tiny https://admin.fedoraproject.org/nagios/cgi-bin//summary.cgi?report=1&displaytype=3&timeperiod=last7days&smon=5&sday=1&syear=2014&shour=0&smin=0&ssec=0&emon=5&eday=15&eyear=2014&ehour=24&emin=0&esec=0&hostgroup=all&servicegroup=all&host=all&alerttypes=3&statetypes=2&hoststates=3&servicestates=56&limit=25 18:38:20 <ursabot> nirik: http://tinyurl.com/q8j48o9 18:38:32 <nirik> our bug alerter is virthost-comm02 this week. 18:38:46 <nirik> it's got some kind of disk issue going on thats making it slow. 18:38:59 <nirik> we are waiting for it to finish raid check and then likely will try a reboot. 18:39:36 <tflink> nirik: when were you thinking the reboot would be? 18:39:47 <nirik> tflink: not sure yet... this raid check is taking forever. ;( 18:40:04 <nirik> [==================>..] check = 91.9% (447933184/487298048) finish=653.5min speed=1003K/sec 18:40:04 <smooge> I think we will be rebooting it before the raid check is done 18:40:12 <smooge> oh wait 18:40:18 <nirik> it's getting close 18:40:18 <smooge> thats different from when I looked 18:40:28 <nirik> tflink: so, perhaps tomorrow? 18:40:36 <smooge> that is 1 MB/sec on a bunch of 3 GB/sec disks 18:40:45 <smooge> of course they are busy 18:40:56 <tflink> ok, I think I have some stuff on there that's not set up to rebound on reboot so please let me know when you reboot it 18:40:56 <nirik> yeah, it's definitely not right. ;) 18:41:20 <nirik> tflink: can do... we also need to coordinate with arm folks as the arm koji instance is on that virthost too 18:41:39 <nirik> #topic Upcoming Tasks/Items 18:41:39 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/ 18:41:56 <nirik> anyhing upcoming anyone would like to note or schedule? 18:42:47 <nirik> #topic Open Floor 18:42:53 <nirik> anyone have items for open floor? 18:44:16 <nirik> ok, if nothing else will end the meeting in a minute, hopefully before the next net.split. ;) 18:44:24 <threebean> :) 18:44:47 <danofsatx-work> my email issues are fixed, and I'll be RHCE certified next Friday ;) 18:44:59 <nirik> danofsatx-work: congrats 18:46:00 <nirik> ok, thanks for coming everyone. Do continue in #fedora-admin, #fedora-apps and #fedora-noc. ;) 18:46:03 <nirik> #endmeeting
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
