============================================ #fedora-meeting: Infrastructure (2013-09-26) ============================================ Meeting started by nirik at 19:00:39 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2013-09-26/infrastructure.2013-09-26-19.00.log.html . Meeting summary --------------- * welcome everyone (nirik, 19:00:39) * New folks introductions and Apprentice tasks. (nirik, 19:03:08) * Applications status / discussion (nirik, 19:06:18) * nuancier is rolling out soon! :) (nirik, 19:07:07) * LINK: https://fedoraproject.org/wiki/Infrastructure/RFR/Copr - any question to this? (mirek, 19:07:40) * LINK: https://fedorahosted.org/fedora-infrastructure/ticket/4023 and https://fedoraproject.org/wiki/Infrastructure/RFR/Copr (nirik, 19:11:36) * bodhi fix for duplicate updates hopefully out (nirik, 19:28:18) * coprs going to discuss best deployment path and revisit next week (nirik, 19:28:32) * nuancier going to go with manual cache/thumbnailing for now, revisit after the upcoming voting. (nirik, 19:28:55) * Sysadmin status / discussion (nirik, 19:31:57) * new ansible on lockbox with some nice new features. (nirik, 19:32:14) * close to having the new cisco ucs blade server working (nirik, 19:35:26) * Upcoming Tasks/Items (nirik, 19:39:20) * LINK: https://apps.fedoraproject.org/calendar/list/infrastructure/ (nirik, 19:39:21) * Open Floor (nirik, 19:42:01) Meeting ended at 19:49:20 UTC. Action Items ------------ Action Items, by person ----------------------- * **UNASSIGNED** * (none) People Present (lines said) --------------------------- * nirik (101) * pingou (42) * mirek (18) * frankieonuonga (11) * threebean (9) * lmacken (9) * anshprat (6) * robyduck (5) * relrod (5) * croberts (5) * zodbot (4) * ynchen (3) * oddshocks (3) * Xeladem (2) * tflink (1) * adimania (1) * abadger1999 (1) * ausmarton (1) * abompard (1) * smooge (1) * mdomsch (0) * puiterwijk (0) * dgilmore (0) -- 19:00:39 <nirik> #startmeeting Infrastructure (2013-09-26) 19:00:39 <zodbot> Meeting started Thu Sep 26 19:00:39 2013 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:39 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:00:39 <nirik> #meetingname infrastructure 19:00:39 <zodbot> The meeting name has been set to 'infrastructure' 19:00:39 <nirik> #topic welcome everyone 19:00:39 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk 19:00:39 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean 19:00:52 * relrod here 19:00:52 * tflink is lurking 19:00:53 * mirek waves 19:00:54 * pingou here 19:00:57 * threebean here 19:01:00 * anshprat here 19:01:02 * adimania is here 19:01:24 * robyduck is also here 19:01:38 <frankieonuonga> i am here as always 19:01:40 * croberts croberts is here 19:01:40 * ausmarton is here 19:01:59 <Xeladem> here too :) 19:02:08 <frankieonuonga> i dont know how to do that i am here thing 19:02:20 <pingou> /me\ 19:02:21 <nirik> frankieonuonga: /me whatever 19:02:24 * pingou sorry 19:02:30 <pingou> /me sorry # damn it! 19:02:41 * ynchen 19:02:42 <mirek> :) 19:02:47 <nirik> :) 19:02:48 * ynchen is here 19:02:51 * Xeladem waves 19:02:54 <nirik> ok, lets go ahead and get started then... 19:03:05 * frankieonuonga is here 19:03:08 <nirik> #topic New folks introductions and Apprentice tasks. 19:03:18 <nirik> any new folks like to introduce themselves? 19:03:25 <nirik> or apprentices with questions or comments? 19:03:38 <ynchen> hi i just joined this week, i looking to join the sysadmin-noc 19:03:51 <robyduck> Yes, me, I'm Robert Mayr from Italy, would like to help out also in the infra team :) 19:04:04 <anshprat> am working on speeding up askbot presently.. 19:04:13 <pingou> robyduck: awesome, welcome :) 19:04:15 * croberts waves hi to robyduck 19:04:19 <nirik> welcome everyone. ;) 19:04:29 <robyduck> Hi pingou, nice to meet you here :) 19:04:32 <threebean> anshprat: thanks for that.. that will be nice :) 19:04:47 * threebean waves "hello" to robyduck 19:04:56 * pingou wonders how threebean does that 19:05:02 <robyduck> hey threebean :) 19:05:04 * frankieonuonga waves to the new crew 19:05:12 <robyduck> pingou ^^ 19:05:45 <nirik> as always do chime in with any questions as we go... and #fedora-admin, #fedora-noc, and #fedora-apps are the place(s) to watch for things to do... 19:06:18 <nirik> #topic Applications status / discussion 19:06:26 <nirik> what do we have application wise this week or upcoming? 19:06:37 <pingou> nuancier 19:06:58 <pingou> we have a stagging version working: https://apps.stg.fedoraproject.org/nuancier 19:07:07 <nirik> #info nuancier is rolling out soon! :) 19:07:18 <pingou> and pkgdb2 w/ pkgdb1 data http://209.132.184.188/ 19:07:27 <nirik> cool. 19:07:29 <pingou> please test, break and let me know how it behaves :) 19:07:34 <nirik> what are the next steps on pkgdb2? 19:07:40 <mirek> https://fedoraproject.org/wiki/Infrastructure/RFR/Copr - any question to this? 19:08:03 <pingou> for pkgdb2: polishing a little still, finish the API authentication question, pkgdb-cli update 19:08:09 <pingou> more unit-tests 19:08:23 <pingou> and that should be it 19:08:29 <nirik> pingou: wow. cool. 19:08:51 <nirik> mirek: yeah... I still have marked to reply to your message about resources... 19:08:54 <pingou> but it takes ~3h to convert the db to the new model :( 19:08:54 <nirik> just been swamped. ;) 19:09:08 <pingou> so we'll need to schedule some down time when we do the migration 19:09:15 <nirik> thats fine. 19:09:32 <pingou> I'm also thinking to send a request for testing to the devel list 19:09:44 <pingou> that I can get bug report and RFE as early as possible :) 19:09:45 <nirik> pingou: +1 19:09:48 <threebean> good idea 19:09:59 <mirek> I'm not sure what is next in process of RFR, should you vote about it? or what? or can I ask for that DB...? 19:10:18 <nirik> mirek: I have a number of questions. ;) 19:10:41 <nirik> are the frontend/backend things we want to run in our normal infrastructure? or are they best still being in the cloud? 19:10:42 * oddshocks late 19:10:49 * pingou has still a point about nuancier, but will wait for after the copr section 19:10:57 * anshprat too 19:11:00 <mirek> nirik: still in cloud. 19:11:22 <mirek> nirik: but there is nothing cloud specific 19:11:36 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/4023 and https://fedoraproject.org/wiki/Infrastructure/RFR/Copr 19:12:11 <nirik> so, I wonder if this would be a good fit for something to be in the other RH cloud that we were offered resources in... since it needs more disk than we have around. 19:12:49 <nirik> also, can you expand on whats in the currently used disk? 19:12:58 <nirik> is it keeping everything ever built? or ? 19:12:58 <mirek> nirik: I did not get response from rvokal about that, I'm asking him every other day thou 19:13:40 <pingou> (how would the "new" cloud interact with the db?) 19:13:45 <nirik> mirek: well, I can talk with Dan Yocum, who is who I was talking to about it... They don't have external ip's yet tho, which we would need. 19:13:48 <pingou> our db, a dedicated node db? 19:13:55 <mirek> nirik: right now it keeps all builds. I have to come with some policy which build will be deleted 19:14:29 <nirik> if it's in the cloud, it would need it's own db also in the cloud most likely. 19:14:37 <nirik> because our db host is not externally reachable. 19:14:57 <mirek> nirik: well ask him :) you can make it only better :) 19:15:01 <nirik> which I guess we could get around via vpn, but I'd rather not. 19:15:25 <pingou> so we can also make it more easily accessible to mirek 19:15:31 <nirik> mirek: will do. I can cc you on email. 19:16:17 <mirek> I can prepare playbook for frontend (which use db) to be deployed as normal fedora service and backend can stay in cloud 19:16:29 <frankieonuonga> nirik:can the cc be to copr-devel 19:16:37 <mirek> we would need just 22 and 443 ports between those two machines 19:16:39 <frankieonuonga> keeps everyone in the loo[ 19:16:43 <frankieonuonga> loop i mean 19:16:57 <nirik> well, this is an implementaion detail, but I can send whatever I find out there too. 19:17:07 * nirik will subscribe 19:17:48 <lmacken> I did a bodhi masher update yesterday which will hopefully fix the duplicate notice updateinfo.xml issue 19:18:03 <nirik> mirek: that might work... the frontend would need credentials to talk to the cloud, but we could probibly setup that. 19:18:20 <nirik> lmacken: cool. ;) pesky timestamps. 19:18:24 <pingou> mirek: but then the frontend should be packaged :) 19:18:33 <mirek> nirik: ok, I will still count with 3 options, ordinary fedora service, staying in fed.cloud, or that hosting, we will see what will look better next week 19:18:44 <mirek> pingou: it is already 19:18:53 <nirik> pingou: happily it just passed review. ;) 19:18:53 <mirek> pingou: even with selinux :) 19:19:06 <mirek> nirik: thanks for that btw 19:19:17 <pingou> \ó/ 19:19:17 <nirik> no problem. 19:19:21 <pingou> champagne! 19:19:25 <threebean> hooray! 19:19:47 <nirik> mirek: where is the storage used for packages? off the frontend? 19:19:57 <mirek> nirik: on backend 19:20:08 <nirik> ok 19:20:08 <pingou> mirek: I still owe you a couple of patch for the API/CLI 19:20:43 <mirek> pingou: and I'm still waiting :) 19:20:54 <nirik> I think we may want a pretty agressive pruning policy... but we can see I suppose. 19:21:44 <mirek> yes, I will discuss it on copr-devel and copr deployment is delayed to next week, do you agree? 19:21:46 <nirik> mirek: any other coprs questions we should look at for now? 19:21:57 <nirik> yep. I need to subscribe there, but will do so. :) 19:22:05 <nirik> abadger1999: did you have any other copr questions? 19:22:41 <abadger1999> nope 19:22:56 <nirik> cool. Any other upcoming application news? 19:23:03 <pingou> about nuancier 19:23:20 <pingou> so it generate thumbnails for the pictures 'candidates' to an election 19:23:30 <pingou> atm these pictures are installed by hand on each node 19:23:49 <pingou> while the cache is generated by the frontend and thus only on one node 19:23:59 <nirik> dumb question: are these the same size/format the wiki uses? 19:24:10 <anshprat> pingou: I found a bug and an issue in nuancier.. ping you after the meeting.. 19:24:21 <pingou> but we can easily generate the cache by hand on each nodes, so this might be the simplest solution for the moment for nuancier 19:24:23 <pingou> anshprat: please :) 19:24:58 <pingou> in the long term, when nuancier will handle upload of the candidates, then we should probably look into a shared storage 19:25:06 <pingou> nirik was proposing glusterfs 19:25:19 <pingou> (eof) 19:25:21 <nirik> we could do gluster or just do a nfs netapp volume... 19:25:40 <nirik> we already have one for the wiki, we could also just reuse part of that... 19:26:06 <pingou> either way is fine for me, my question is more, do we want to have this already or should we wait for nuancier full feature? 19:26:13 <nirik> but If you are ok with manual for now, we can just revisit it later. 19:26:37 <nirik> when is the go-live date? 19:26:40 <pingou> nirik: the pictures themselves are pretty big (not sure what the wiki uses, but likely the same), the thumbnails are 128x128px 19:26:43 <nirik> for the upcoming election? 19:26:45 <pingou> nirik: Monday 19:27:09 <anshprat> pingou: is there a way to download and try out the images ? 19:27:22 <nirik> yeah, I'd say manual for now, revisit after this election. 19:27:24 <anshprat> (full res I mean) 19:27:43 <pingou> anshprat: yes, right click on the 'google' icon when hovering over the image 19:27:48 <pingou> nirik: wfm, thanks :) 19:28:18 <nirik> #info bodhi fix for duplicate updates hopefully out 19:28:32 <nirik> #info coprs going to discuss best deployment path and revisit next week 19:28:55 <nirik> #info nuancier going to go with manual cache/thumbnailing for now, revisit after the upcoming voting. 19:29:03 <lmacken> mashing in koji discussions underway :) 19:29:10 <threebean> cool :) 19:29:19 <nirik> yeah. :) I added some thoughts to that thread. 19:29:25 <nirik> would be great to move that into koji 19:29:49 <lmacken> from a bodhi perspective, it has to end up in /mnt/koji anyway, so bodhi could simply flip symlinks 19:30:14 * lmacken should chime in on the thread :) 19:30:17 <nirik> yeah, it's a lot of data to shlep around too... and needs enough space to write it. 19:30:44 <lmacken> yeah, moving it from the builders to the hub, etc, would be a lot of work 19:30:47 <nirik> lmacken: how would we handle signing? just like now? get a list, sign, push and hopefully everything in there is signed? 19:31:08 <lmacken> signing doesn't have to change at all, right? 19:31:18 <nirik> yeah, it could be the same 19:31:57 <nirik> #topic Sysadmin status / discussion 19:32:03 <nirik> ok, on the sysadmin side... lets see. 19:32:14 <nirik> #info new ansible on lockbox with some nice new features. 19:32:29 <nirik> One of the new nice things in ansible 1.3.x is 'accelerate' mode. 19:32:44 <threebean> its fireball but without the zeromq libs :) 19:32:47 <nirik> I think we should just look at enabling that everywhere and using it. It makes things much much faster 19:32:53 <threebean> +1 19:32:56 <nirik> yep. It's more simple... no zmq 19:33:14 <pingou> what was zmq used for already? 19:33:24 <nirik> it does need python-keyczar on each node. 19:33:30 <nirik> but thats got only a few deps. 19:33:33 <lmacken> 2-6x faster than ssh apparently 19:33:55 <lmacken> ah, all we do is add 'accelerate: true' to our playbooks?? 19:34:07 <nirik> gallery playbook: with accel: 32sec, without: 1minute 50 seconds. 19:34:13 <relrod> wow 19:34:17 <threebean> its fun :) 19:34:32 <nirik> lmacken: and install python-keyczar on the hosts being playbooked 19:34:41 <lmacken> cool 19:35:06 * abompard coming in late 19:35:26 <nirik> #info close to having the new cisco ucs blade server working 19:35:44 <nirik> I have one of the blades asking for dhcp correctly now... but then it ignores the ip. ;( 19:36:03 <nirik> I'm going to schedule 2 outages for next week: 19:36:47 <nirik> 1. on tuesday for buildsys stuff. Will move some guests around on bvirthosts, and also will move lookaside cache on pkgs01 to netapp storage, and also apply updates/reboots 19:37:23 <nirik> 2. on wed for class a/non buildsys stuff. moving some guests to a new virthost15 and updates/reboots. 19:37:36 <nirik> I'll file outages for them and send announcements later today. 19:37:50 <nirik> If there's anything else someone needs an outage for let me know. 19:38:39 <nirik> We are trucking thru the backlog of stuff waiting for the freeze to be over... 19:38:51 <nirik> if anyone sees tickets that were waiting on that that haven't been applied yet, please note them. ;) 19:39:13 <nirik> anything else on the sysadmin side? 19:39:20 <nirik> #topic Upcoming Tasks/Items 19:39:21 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/ 19:39:33 <nirik> calendar should be up to date on our milestones now. ;) 19:39:38 * nirik figured out how to edit it 19:39:45 <pingou> \ó/ 19:40:17 <nirik> just to note: I am going to be out next friday, all the following week and then back the monday after that. 19:40:32 <nirik> I'll possibly check emails and say hi, but won't be on vastly much. ;) 19:40:42 <oddshocks> nirik: will someone else run the meeting? 19:40:56 <frankieonuonga> nirik: enjoy 19:41:00 <pingou> nirik: have fun :) 19:41:01 <frankieonuonga> the break 19:41:07 <nirik> yeah, need to line someone up. ;) Usually smooge does. 19:41:13 <oddshocks> cool :) 19:41:55 <nirik> our next freeze is after I get back... 19:42:01 <nirik> #topic Open Floor 19:42:02 <relrod> I should be around if he can't be 19:42:14 <nirik> anything else anyone would like to discuss or note? 19:42:20 <nirik> relrod: thanks. 19:42:47 <croberts> I am going to send in a commit for the FAS newslink and get that ticket closed 19:42:59 <croberts> and file the upstream bug that you suggested nirik 19:43:01 <nirik> croberts: excellent. 19:43:03 <frankieonuonga> i can help relrod 19:43:16 <nirik> croberts: there's a few more wiki related tickets if you are looking for more. ;) 19:43:35 <croberts> nirik: nice :) I will check out the trac and look at them 19:43:58 <nirik> cool. 19:44:58 <relrod> I'm in the process of trying to set up a dpsearch instance and get frankieonuonga access to it, but our temp instance playbooks are running into various issues with the ansible update 19:45:17 <smooge> sorry been cleaning up. 19:45:47 <nirik> relrod: I can assist some too... I cleaned up some various issues with other playbooks. 19:46:16 <nirik> smooge: no worries. 19:46:22 <nirik> smooge: anything to note for the meeting? 19:46:27 * frankieonuonga agreeing that relrod has been of great assistance 19:47:08 <relrod> frankieonuonga: :) 19:48:19 <nirik> ok, will close out in a min if nothing else... 19:49:18 <nirik> Thanks for coming everyone. Do continue in our normal channels. :) 19:49:20 <nirik> #endmeeting
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
