=================================== #fedora-meeting: FESCO (2010-06-15) =================================== Meeting started by mjg59 at 19:35:11 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2010-06-15/fesco.2010-06-15-19.35.log.html Meeting summary --------------- * init process (mjg59, 19:36:01) * kylem and nirik unable to attend today (mjg59, 19:36:30) * #351 Create a policy for updates - status report on implementation (mjg59, 19:36:47) * ACTION: mclasen to look over critpath documentation (mjg59, 19:40:18) * #382 Implementing Stable Release Vision (mjg59, 19:41:12) * LINK: https://fedoraproject.org/wiki/Stable_release_updates_vision (mjg59, 19:49:10) * ACTION: fesco to continue to update https://fedoraproject.org/wiki/Stable_release_updates_vision_implementation_ideas and aim to produce a comprehensive policy for the board (mjg59, 19:56:41) * #387 (mjg59, 19:57:18) * #387 compile list of supported CPUs and react to recent loss of support for Geode LX on F13 (mjg59, 19:57:24) * AGREED: dsd_ to post a patch to disable 686 assembler optimisations for glibc for f13 (mjg59, 20:39:30) * AGREED: more research on the details of building i586 separately to be carried out before deciding on f14 (mjg59, 20:40:18) * #394 F14Feature: MeeGo 1.0 https://fedoraproject.org/wiki/Features/MeeGo_1.0 (mjg59, 20:41:18) * AGREED: Meego 1.0 is an F14 feature (mjg59, 20:46:12) * #395 F14Feature: Sugar 0.90 https://fedoraproject.org/wiki/Features/Sugar_0.90 (mjg59, 20:46:27) * AGREED: Sugar 0.90 is an F14 feature (mjg59, 20:47:47) * #396 F14Feature: systemd - https://fedoraproject.org/wiki/Features/systemd (mjg59, 20:48:02) * AGREED: Systemd is a feature for F14 (mjg59, 21:00:57) * Open Floor (mjg59, 21:04:26) * FES ticket updates - https://fedorahosted.org/fedora-engineering-services/report/6 (mjg59, 21:05:07) * Open Floor (mjg59, 21:07:00) Meeting ended at 21:08:52 UTC. Action Items ------------ * mclasen to look over critpath documentation * fesco to continue to update https://fedoraproject.org/wiki/Stable_release_updates_vision_implementation_ideas and aim to produce a comprehensive policy for the board -- 19:35:11 <mjg59> #startmeeting FESCO (2010-06-15) 19:35:11 <zodbot> Meeting started Tue Jun 15 19:35:11 2010 UTC. The chair is mjg59. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:35:11 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:35:17 <mjg59> #meetingname fesco 19:35:17 <zodbot> The meeting name has been set to 'fesco' 19:35:36 <mjg59> #chair mjg59 mclasen notting SMParrish_mobile ajax pjones cwickert 19:35:36 <zodbot> Current chairs: SMParrish_mobile ajax cwickert mclasen mjg59 notting pjones 19:35:52 <mjg59> We're missing kyle and nirik 19:36:01 <mjg59> #topic init process 19:36:05 <pjones> zodbot uses LC_COLLATE=en_US . Ew. 19:36:13 <mclasen> kyle said he might not make it (still in UK) 19:36:20 <mjg59> Yeah 19:36:22 <mjg59> Ok 19:36:30 <mjg59> #info kylem and nirik unable to attend today 19:36:37 <mjg59> Ok, shall we start? 19:36:47 <mjg59> #topic #351 Create a policy for updates - status report on implementation 19:36:48 <pjones> oh, wait, I'm backwards. 19:36:50 <mjg59> .fesco 351 19:36:51 <zodbot> mjg59: #351 (Create a policy for updates) - FESCo - Trac - https://fedorahosted.org/fesco/ticket/351 19:37:02 <mjg59> Anyone got anything to say on this this week? 19:37:15 <notting> no change on my end from this week. jlaska was doing some critpath documentation 19:37:51 <mjg59> Anyone else? 19:37:57 <mclasen> did lukes bodhi update happen ? 19:38:06 <mjg59> lmacken: Around? 19:38:10 * mclasen should know this, but doesn't... 19:38:17 <jlaska> notting: feel free to make suggestions, I'm missing important critpath information for maintainers -- http://fedoraproject.org/wiki/Critical_Path_Packages 19:39:06 * mclasen will look it over after the meeting too 19:39:32 * cwickert is here... 19:40:02 <jlaska> mclasen: thx 19:40:18 <mjg59> #action mclasen to look over critpath documentation 19:40:33 <mjg59> Anyone else? Or shall we move on? 19:40:57 <mjg59> (going once.. twice...) 19:41:12 <mjg59> #topic #382 Implementing Stable Release Vision 19:41:13 * cwickert thinks that the KDE list of critical path is way to short 19:41:34 <mjg59> cwickert: We've had that conversation - unfortunately it seems difficult to come up with a better one that doesn't cover pretty much the entirity of KDE 19:42:18 <mjg59> And an overly broad critpath would likely serve as a deterrent to maintainers at the moment. If KDE ends up looking bad compared to the rest of the distribution, then with luck that's something they'll pay attention to 19:42:32 <cwickert> mjg59: well, the Xfce critpath covers half of Xfce, the LXDE one too 19:42:44 <pjones> cwickert: sure, but those are both smaller than kde, right? 19:43:07 <cwickert> yes, but the percentage is way higher 19:43:12 <cwickert> anyway, lets move on 19:43:45 <mjg59> Ok, so we have the wiki page at https://fedoraproject.org/wiki/Stable_release_updates_vision_implementation_ideas 19:43:56 <mjg59> We haven't really added much to that 19:44:29 <mjg59> I'm not convinced that IRC is the best way to discuss adding ideas, but a private mailing list also doesn't seem reasonable 19:44:43 <SMParrish_mobile> Think I added something to it 19:44:49 <mjg59> Maybe people could just scribble some more stuff in there and annotate it on the discussion page if necessary? 19:44:57 <mjg59> It'd be nice to have something to bring to the board at some point 19:45:05 <mjg59> (Yeah, I know that I'm guilty here) 19:45:26 <cwickert> sorry for the stupid question, but I was away for the last weeks... what about the "only security updates and bugfixes" rule? are we trying to drive our users away from Fedora? 19:45:56 <ajax> that's a bit of a bait-y question, but. 19:45:58 <cjb> cwickert: you get the minus points for assuming bad faith. 19:45:58 <ajax> here's the thing 19:46:32 <ajax> if releases are continuously updated with new features, then what is to distinguish them from rawhide 19:46:37 <cwickert> I mean, new versions in a release are one thing that makes us different from the rest of the distributions and according to several polls, users want this. IMO this is a completely different issues from bad/no testing and low quality 19:46:53 <pjones> cwickert: you skipped "far too many updated packages" 19:47:15 <cwickert> pjones: "too many" for whom? 19:47:21 <pjones> for users? 19:47:39 <pjones> If I obeyed packagekit I'd be rebooting my desktop machine 3 times per day for all of F13. 19:47:43 <cwickert> if the users want the updates, then it cannot be "too many" 19:48:01 <cwickert> strange, I only once a day if at all. 19:48:05 <mclasen> people for whom software updates are not the central piece of their fedora experience and interest 19:48:15 <mjg59> Wait. Hang on. 19:48:25 <mjg59> "Stable releases should not be used for tracking upstream version closely when this is likely to change the user experience beyond fixing bugs and security issues." 19:48:26 <cwickert> mclasen: updates are an offer, not a duty. 19:48:30 <pjones> (also, this is far off topic at the moment) 19:48:31 <mjg59> This is from the Board's statement 19:48:46 <pjones> mjg59: right. it's not up for discussion. 19:48:48 <mjg59> It's a position we've been asked to implement 19:49:04 <cwickert> mjg59: you have the link to the boards statement? 19:49:08 <mclasen> cwickert: until the next security update pulls them all in... 19:49:09 <notting> i last updated on 06-05. i have 141 updates waiting. 19:49:10 <mjg59> https://fedoraproject.org/wiki/Stable_release_updates_vision 19:49:19 <LinuxCode> cwickert, too many updates that add new features are not a good user experience 19:49:30 <mjg59> We've had this discussion many times now 19:49:36 <mjg59> I don't think there's a great deal we can add to it here 19:49:43 <LinuxCode> specially as it is, imho, more important to make sure our current stuff works 19:49:44 <cwickert> LinuxCode: again, this is your definition of "too many" updates 19:49:51 <mjg59> The Board have taken a position, and we get to implement it 19:50:02 <LinuxCode> cwickert, updates are fine, as long as they dont add too many new features 19:50:04 <cwickert> LinuxCode: making sure stuff works is a completely different issue 19:50:09 <mjg59> If we disagree with that position then we're free to tell them that we disagree, but unless they change their mind then we still get to implement it 19:50:22 <LinuxCode> cwickert, agreed 19:50:25 <LinuxCode> mjg59, yeh 19:50:29 <mjg59> But I don't think that this is the right place to have that conversation 19:50:34 <cwickert> LinuxCode: but the users WANT the new features, that's why they use Fedora and not Ubuntu, SUSE and whatnot 19:50:53 <mclasen> lets not repeat that discussion here and now 19:50:57 <pjones> derail: success! 19:51:02 <notting> mjg59: unless we want to take the position that 'fesco refuses to work on it'. which, as of the last couple of meetings, we are *not* taking that position. 19:51:13 <mjg59> notting: Right 19:51:21 <ajax> cwickert: that's a reasonable position, but i think what we're aiming for there is "if you want that, use rawhide" 19:51:41 <gholms> ...or the branched release, if any 19:52:09 <mjg59> But in terms of the implementation, does anyone want to add anything at this point? 19:52:26 <mjg59> Or shall we just try to be better at sticking ideas in this week, and try to come up with a cohesive plan? 19:53:38 <mjg59> Ok, moving on? 19:54:02 <cwickert> one question and I will stop arguing. what abput parrot and rakudo for example. people want the monthly updates and they want it on a stable release and not in rawhide. what about these kind of updates? 19:54:21 <mjg59> cwickert: One option is to provide separate repositories for well-bounded package subsets 19:54:32 <mjg59> Let people opt-in to more active updates 19:54:45 <cwickert> mjg59: we already have a yum plugin for that 19:54:46 <pjones> cwickert: if there's an overwhelming reason for some specific case, put it on the agenda and we can discuss it as an exception to the normal process? 19:54:49 <mjg59> As long as there's little interaction between them and the rest of the distribution, that should be pretty safe 19:55:02 <pjones> cwickert: that is, write a specific proposal for an exception to the process and put it on the agenda? 19:55:21 <mjg59> And, yeah, if it's the case that basically all users of a package are going to want updates, then maybe it'd be outside the bounds of the proposal 19:55:23 <cwickert> pjones: I have asked this several times, it was discussed and there was no solution. 19:55:40 <mjg59> cwickert: We're still very much at the stage of trying to brainstorm out an implementation 19:55:42 <pjones> but you're asking vague questions, not coming up with a specific proposal 19:55:46 <notting> cwickert: there's an option for updates-features on the wiki page referenced above 19:55:57 <cwickert> notting: sounds fair 19:55:58 <mjg59> We're not trying to make anything impossible at this point 19:56:41 <mjg59> #action fesco to continue to update https://fedoraproject.org/wiki/Stable_release_updates_vision_implementation_ideas and aim to produce a comprehensive policy for the board 19:56:54 <mjg59> Sound ok? 19:56:56 <cjb> n 19:56:58 <cjb> oops 19:57:04 <pjones> mjg59: yeah 19:57:18 <mjg59> #topic #387 19:57:24 <mjg59> #topic #387 compile list of supported CPUs and react to recent loss of support for Geode LX on F13 19:57:27 <mjg59> .fesco 387 19:57:28 <zodbot> mjg59: #387 (compile list of supported CPUs and react to recent loss of support for Geode LX on F13) - FESCo - Trac - https://fedorahosted.org/fesco/ticket/387 19:57:52 <mjg59> So by the sounds of it, this problem is due to the glibc spec file passing more aggressive optimisation arguments to the linker? 19:58:10 <dsd_> not the spec file 19:58:15 <mclasen> but jakub seems to be unwilling to change that 19:58:17 <dsd_> the hand-crafted Makefile system inside of glibc upstream 19:58:19 <pjones> the configure script or somesuch, but yeah 19:58:28 <pjones> dsd_: right 19:58:29 <mjg59> Sorry, right 19:58:35 <mgoetze> assembler, not linker 19:58:40 <dsd_> and its assembler optimizations 19:58:45 <Guest13239> well, the problem is more general, but the symptom is caused by the glibc scripting. 19:59:14 <mjg59> Right, so other than the facts I was entirely correct 19:59:31 <mjg59> Jakub makes a reasonable point that it's entirely plausible that gcc will start doing the same at some point 19:59:45 <pjones> well, not during f13 one would hope 19:59:49 <mjg59> One would hope 19:59:57 <pjones> and if it does, ... 20:00:01 <mjg59> So we can either "fix" the glibc build system 20:00:10 <mjg59> (Which the maintainer is unenthusiastic about, but still) 20:00:18 <mjg59> Or we can change to start building for 586 again 20:00:21 <ajax> notting: what are we winning by chopping off 586? 20:00:35 <notting> actual performance gains, as i recall 20:00:39 <mjg59> We made a conscious decision to go from 586 to 686, so that doesn't seem immediately appealing 20:00:48 <dsd_> i presume that if we fix both F13 and F14+, we are talking about 2 separate fixes. which one are we discussing? 20:00:49 <mclasen> as a f13 hotfix, patching glibc seems reasonable to me; long-term, jakub's argument is right 20:00:57 <mjg59> And I don't think it's worth dropping back to 586 just to support a cpu that's genuinely a 686 20:01:07 <pjones> mclasen: I'm leaning that same way 20:01:36 <mjg59> Yeah, I think we should just make the change in F13 20:01:53 <dsd_> which change? 20:02:02 <cjb> dsd_: are you interested in coming up with a glibc patch we could review? (the glibc makefile change to avoid asm optimizations.) 20:02:05 <mjg59> And I think we should encourage the binutils and gcc maintainers to let us know if optimisation parameters will change the set of supported CPUs 20:02:05 <ajax> so the story for f14 is... secondary arch? i586? 20:02:23 <Guest13239> a decision to not fix this in the long term = a decision to drop support for the Geode LX in Fedora. 20:02:25 <mjg59> ajax: Well, it's also entirely possible that f14 won't change things 20:02:27 <mclasen> dsd_: make glibc not pass -mtune=i686 to as 20:02:32 <dsd_> cjb: yes, its a trivial patch that i can make this afternoon 20:02:41 <ajax> Guest13239: excluded middle, boo. 20:03:33 <Guest13239> it's ok to decide that, just be aware of what you are deciding. 20:03:40 <mjg59> Guest13239: Correct, a decision not to support machines that don't implement 686 nopl is a ecision to drop Geode LX support 20:03:46 <cwickert> I don't think that it is right to do something because upstream will do it "at some point". we have a policy to follow upstream, but this means to honor upstreams decision and not to go ahead. dropping the support for the geode would be a shame as long as the OLPC is our largest deployment 20:04:11 <ajax> mjg59: which is not the same as "build two, one for 586 and one for 686" 20:04:13 <ajax> which is my point 20:04:19 <pjones> cwickert: I'm not sure those numbers will stand up to scrutiny, but I agree with the sentiment. 20:04:22 <Guest13239> In F12 the decision was explicitly made to drop the i586 but to retain the Geode LX. That is the decision that's being considered for revision if no fix is put in place for this bug. 20:04:33 <pjones> Guest13239: yes, we know. 20:04:44 <notting> have we discussed with kylem and davej the possibility of carrying the emulation patch? 20:05:00 <pjones> not really; last week we all pretty much all thought that was a non-started 20:05:00 <mjg59> Guest13239: ...we're explicitly talking about fixing this bug. 20:05:03 <pjones> especially upstream 20:05:44 <mclasen> notting: upstream first, yadda yadda 20:05:56 <notting> we *do* carry the occasional patch, though 20:05:59 <pjones> yeah 20:06:04 <pjones> davej: any thoughts? 20:06:14 <cjb> notting: according to https://bugzilla.redhat.com/show_bug.cgi?id=579838#c15 , this particular patch is very not a good idea. 20:06:16 <notting> (sorry, i have to drop off. will read scrollback) 20:06:21 <mjg59> notting: Thanks! 20:06:45 <davej> I tried to get it upstream a while back, but it turned into a wankfest of "we should generically support missing instructions" 20:07:24 <mjg59> Yeah, upstream has ideas about how they want this done 20:07:33 <davej> and it's a non-trivial amount of work 20:07:34 <mjg59> In theory it could be integrated with the kvm instruction emulation stuff 20:07:48 <ajax> in practice maybe it'd be nice to have something that works instead. 20:07:59 <davej> you'd think that, but people suck 20:08:19 <cjb> note that "works" doesn't really describe the emulation patch, since it's turning a NOP into a gigantic kernel trap. 20:08:39 <mclasen> and just like the kernel people don't like emulation, the toolchain people probably don't like a new IS for geode ? 20:08:52 <cjb> mclasen: it's not geode only. 20:08:52 <ajax> cjb: well you could always fix mmap(PROT_EXEC) to hunt for NOPL and patch... 20:08:59 <ajax> (note: do not do this) 20:09:40 <pjones> ajax: I think I just threw up a little. 20:09:56 <ajax> pjones: there's pepto in the break room 20:10:22 * LinuxCode gets a glass of water for pjones 20:10:38 <ajax> so: do we want to carry this patch (or a similar one) for f14, or do we want to drop geode from f14, or do we want to do something else to support geode in f14. 20:11:00 <pjones> it seems like there's a better glibc fix we're missing in this discussion 20:11:51 <pjones> but maybe there's not. 20:11:58 <ajax> i can't really think of one. 20:12:07 <pjones> I dunno, something that doesn't involve switching the definition of i686 every 10 minutes. 20:12:14 <mjg59> Is it worth lobbying for a "586+cmov" gcc and binutils architecture? 20:12:27 <pjones> mjg59: that's never worked before... 20:12:34 <cjb> pjones: to include an undocumented instruction that isn't in the spec. :/ 20:12:40 <dsd_> last week it was pointed out that geode LX does not support all types of cmov 20:12:41 <pjones> cjb: exactly. 20:12:49 <ajax> pjones: i think they've actually been pretty consistent, as long as you consider 686 to mean Pentium Pro 20:12:54 <cjb> dsd_: is that actually an issue? 20:12:58 <dsd_> (another reason to stop pretending that geode LX is i686, in my opinion) 20:13:02 <cjb> (do we emit any of the unsupported types?) 20:13:05 <dsd_> cjb: not sure.. and if it isnt, it might be in 6 months time 20:13:27 <cjb> dsd_: seems separate to me.. I've never seen any problem with that. 20:13:30 <mjg59> Well, in any case. Are we decided on what to do for f13? 20:13:55 <dsd_> cjb: similarly Fedora worked up til F13, and now we have a problem. same could happen again 20:14:31 <LinuxCode> would it not be possible to just freeze glibc in terms of version ? 20:14:31 <cjb> dsd_: (it could, but we don't have any evidence that it will from the last couple of years, so it's hardly likely.) 20:14:39 <ajax> i think we were pretty clearly unanimous for f13: patch glibc. 20:14:42 <LinuxCode> just throwing this in 20:14:44 <cjb> mjg59: sounds like dsd_ comes up with a patch and attaches to the bug 20:14:55 <mclasen> dsd_: that can always happen; best prevention against that is continuous integration 20:14:56 <mjg59> Yeah 20:15:01 <dsd_> i'll make that patch this afternoon 20:15:07 <mjg59> So then, there's a question for the long term 20:15:21 <pjones> LinuxCode: not terribly effective. 20:15:21 <mjg59> We can't guarantee that this won't happen again unless we revert to i586 20:15:25 <LinuxCode> pjones, i see 20:15:32 <LinuxCode> just a wild idea 20:15:55 <ajax> and it sounds like we do not want to revert to 586 for the distro as a whole. 20:15:58 <cjb> mjg59: it sounds like none of us actually want that to happen :) 20:15:58 <mjg59> Right 20:16:18 <pjones> yeah, I think we all agree that's generally undesirable. 20:16:22 <ajax> so i'm just going to throw that whole "secondary arch" thing out there again 20:16:47 <ajax> we haven't had one since ppc fell off the map, really. 20:16:47 <Guest13239> I haven't figured out what's undesirable about 586 support. Tossing whole architectures over the wall to get a <1% speedup never struck me as wise. 20:16:50 <mjg59> So from my point of view, Geode developers get an emulation patch upstream, Geode developers get a geode architecture into gcc and binutils or we do secondary arch for i586 20:17:09 <pjones> ajax: all of 32-bit x86 ? We've got no precedent for subarches as secondaries, and that might be... complicated. 20:17:14 <mclasen> don't we have some way to install IS-variants of libraries, anyway ? like no-mmx variants, etc 20:17:22 * mclasen fuzzy on how that works 20:17:39 <ajax> mclasen: it's not especially clear, period. 20:17:45 <pjones> mclasen: that's a good question - could we not do an i586 glibc? 20:18:05 <mjg59> pjones: Sure, but any other package that passes the same build options may hit the same issue 20:18:09 <pjones> which would be fine unless kernel (or other things) start using the flags that get them nopl 20:18:09 * gholms rings the 20 minute bell 20:18:09 <ajax> pjones: if gcc starts invoking gas with -march... 20:18:27 <pjones> ajax: yeah 20:18:30 <ajax> vote to continue (+1 from me) 20:18:34 <pjones> +1 to continue 20:18:36 <dsd_> continue 20:18:51 <mclasen> continue 20:18:56 <pjones> ajax: I think the gcc issue there might be a "ask them to not do that" and/or "kill that horse when we come to it" sortof thing. 20:18:58 <SMParrish_mobile> Continue 20:19:43 <ajax> that's four (dsd doesn't count, not fesco, sorry dude). but i'll take it as majority since we're short. 20:19:57 <ajax> pjones: they're going to balk at that. every 0.1% on specint... 20:20:08 <pjones> ajax: and anything else in userland gets to make a choice of "don't do that manually" or "also ship a i586..." 20:20:15 <pjones> ajax: I'm still talking about F13 here 20:20:26 <pjones> F14 may need a differently nuanced answer. 20:21:09 <ajax> pjones: fair. we _could_ potentially do an i586 build of glibc for f13; we did it before, after all. 20:21:18 <pjones> I think it's reasonable to ask gcc not to randomly switch stuff like that intra-release. 20:21:40 <ajax> and i think that's actually a point in favor of subarch as secondary arch; yum knows that if you installed i586 it's probably because you meant it and can't do i686 20:21:40 <mclasen> yeah, stable updates, and all that... 20:22:03 <cjb> ok. so would we apply dsd's patch to F13 and F14, pending a better outcome? 20:22:21 <cjb> (I mean, F13 and devel) 20:22:50 <ajax> f13 certainly. not sure devel is necessary yet. 20:22:55 <pjones> ditto ajax 20:23:30 <pjones> so maybe we should be discussing subarch secondary for F14 20:23:33 <LinuxCode> ajax, but would you push the packages into the same repos for i686 ? 20:23:39 <pjones> but let's discuss F13 to completion _before_ we get on that. 20:23:55 <pjones> LinuxCode: we certainly could. 20:24:05 <cwickert> IMHO F13 is most important, but if we want to encourage the OLPC folks to continue developing on Fedora, we need F14 as well 20:24:05 <LinuxCode> might make it more transparent 20:24:07 <ajax> pjones: i think we _ought_ not, but we could. 20:24:32 <LinuxCode> ajax, question is if that technically works 20:24:47 <ajax> alright then, f13 proposal: patch glibc to not emit nopl because geode lx is supported in f13. 20:24:52 <ajax> (+1 from me) 20:24:57 <mjg59> +1 20:25:02 <pjones> +1 20:25:02 <mclasen> +1 20:25:16 <davej> s/glibc/gcc/ no ? 20:25:26 <ajax> davej: no because right now only glibc does it. 20:25:31 <davej> ugh 20:26:01 <SMParrish_mobile> +1 20:26:15 <ajax> +5, majority, motion carries. 20:26:17 <ajax> so! f14. 20:27:06 <pjones> F14 ... um... shit. 20:27:12 <ajax> i don't think 586 belongs in the same repos as 686 anymore. the isos and boot images would conflict. 20:27:36 <ajax> or you'd build them out of 586 and then something would have to figure out if you could do real i686 20:27:40 <ajax> and that's dumb too 20:27:51 <pjones> Is asking olpc to (help) maintain a secondary arch that's essentially "i686 + i586 rebuilds of anything that doesn't work as i686" going to cause them to run screaming to debian? 20:28:00 <pjones> it's probably not all that much work... 20:28:05 <LinuxCode> pjones, hehe 20:28:14 <mjg59> Well, we should probably do a better job of documenting what building a secondary arch involves 20:28:19 <mclasen> they didn't seem enthusiastic last week... 20:28:53 <ajax> pjones: if gcc changes to invoke gas with optimizations, "anything that doesn't work" is "everything" 20:28:53 <mjg59> Bootstrapping from 686 should be trivial - the only real problematic case I can think of is static libraries 20:28:53 <dsd_> pjones: (speaking as OLPC community member) i dont think anyone at OLPC has that experience. also OLPC resources are veeeery tight. 20:28:58 <dsd_> what kind of work is involved? 20:29:05 <pjones> ajax: yes, but it's really simple rebuilds 20:29:09 <mjg59> jwb: You around? 20:29:27 <jwb> mjg59, will be in a bit 20:29:41 <pjones> dsd_: basically rebuilding every package inside an i586 mock builder and then doing a compose. 20:29:44 <mgoetze> if i may make a comment as a debian user :) while i would welcome olpc users i can also share our solution: debian is 486+ but we allow optimized libs in e.g. /lib/i686/cmov/libc.so.6 20:29:48 <mjg59> jwb: Ok - we were just curious as to the amount of effort you'd guess would be involved in running 586 as a secondary 20:29:59 <gholms> Would that proposal be similar to ppc64's being "ppc, with 64-bit exceptions?" 20:30:00 <pjones> dsd_: I think spot + dgilmore have more details that make that a lot easier than I just made it sound. 20:30:13 <mclasen> mgoetze: thats the is-variant-libraries thing I alluded to earlier 20:30:15 <pjones> gholms: unless gcc changes the options it passes to gas, yes 20:30:25 <cjb> (koji-shadow can make it easier, aiui) 20:30:27 <dsd_> i'm quite ignorant about the Fedora infrastructure, but wouldnt this be trivial to do on the official koji side? 20:30:30 <mclasen> mgoetze: doesn't help for forbidden instructions in apps or kernels, though... 20:30:38 <dsd_> just adding 1 more build profile with 1 little change of -march flag 20:30:41 <pjones> mgoetze: the "sunos 2.5" solution, eh? 20:30:56 <mgoetze> mclasen: the question is, how badly do you need these optimizations... 20:31:03 <pjones> mgoetze: think about openoffice 20:31:09 <LinuxCode> dsd_, but somebody still has to glance over things to make sure packages have built 20:31:20 <mjg59> mgoetze: We did some work on this in a previous release and decided it was worth it 20:31:29 <pjones> dsd_: yeah, for i586 you wouldn't need to set up your own builders 20:31:32 <dsd_> LinuxCode: ok, the "glacing over and testing" can certainly be left to OLPC community 20:31:41 <ajax> dsd_: the only issue with doing that (besides storage, which we may not have to burn) is that primary arches are mandatory for build success. 20:31:58 <ajax> so a build failure on i586 would mean no update for any arch 20:32:09 <pjones> ajax: sure, but the number of build failures that are i586 vs i686 only is going to be /fairly/ low. 20:32:16 <dsd_> i would suspect it would be tiny 20:32:16 <LinuxCode> puts a bigger burden on maintainers 20:32:22 <LinuxCode> pjones, yeh 20:32:28 <LinuxCode> minimal 20:32:29 <mjg59> So in the worst case scenario that i686 nopls become common, we're faced with two options - kernel emulation and a secondary arch, right? 20:32:31 <dsd_> we're talking about a pretty safe change, in my opinion 20:32:32 <mclasen> ajax: a x86 variant is not going to be a big issue for build failures, compared to ppc, no ? 20:32:38 <mgoetze> mjg59: i mean would it still be good enough if you only have it in certain important/performance-critical libs 20:32:38 <pjones> it's not going to be like sparc or ppc where helper binaries take a shit with a bus error 20:32:42 <dsd_> and fedora already shipped for a while with -march=i586 20:32:46 <pjones> mclasen: exactly 20:32:48 <ajax> pjones/mclasen: agreed, not likely to cause new failures. 20:32:48 * mgoetze goes back into his debian corner now :) 20:32:49 <mjg59> mgoetze: There's several binaries that matter 20:32:49 <LinuxCode> dsd_, well, if infra has capacity 20:32:54 <cjb> mjg59: I think you'd take a performance nose-dive if you were emulating them *and* they were everywhere. 20:33:04 <jwb> so 20:33:05 <ajax> the storage thing is pretty real though. 20:33:14 <pjones> ajax: yeah. 20:33:16 <jwb> if you mean a full secondary koji instance 20:33:16 <mjg59> cjb: Yeah, so if that's a problem then an arch rebuild would seem to be it 20:33:21 <pjones> Oxf13: hey, do you know anything relevant here? 20:33:34 <cjb> jwb: then you could koji-shadow, right? 20:33:39 <jwb> it's not trivial. there is setup, hardware, shadowing, and then doing the releng aspects all on your own 20:33:44 <pjones> jwb: I think we just need to auto-rebuild them in the normal koji instance. 20:33:47 <LinuxCode> + mirrors 20:33:59 <ajax> cjb: at best, you could fix up every page where you find nopls, as you hit them. so you'd only take the trap the first time. but that's a full disassembler in the kernel... 20:34:01 <jwb> cjb, koji-shadow helps but it is not a perfect solution 20:34:02 <pjones> jwb: they should build fine on x86 builders, after all 20:34:25 <cjb> ajax: nod. 20:34:29 <jwb> pjones, you mean reuse existing koji as a secondary instance? 20:34:36 <pjones> jwb: yeah, basically 20:34:40 <jwb> koji can't do that 20:34:40 <Oxf13> *twitch* 20:34:43 <jwb> builders talk to one hub 20:34:56 <dsd_> are we decided against adding i586 as a primary arch? 20:34:58 <Oxf13> right, we don't have ways of having builders talk to multiple hubs, or having multiple builders on a single machine 20:35:00 <pjones> I don't mean "set up another hub" either. 20:35:08 <jwb> pjones, then you mean primary 20:35:13 <pjones> dsd_: I think that's gone as an assumption so far. 20:35:19 <dsd_> i understand the concerns about storage on the current infra. apart from that, it would be a really good option i think 20:35:28 <kalev> I think a whole lot more easier for all involved parties would be to revert back to i586 as main architecture 20:35:28 <jwb> koji doesn't have the concept of 'primary/secondary' at the hub level 20:35:30 <dsd_> i am admittedly talking from the sidelines though, you guys are the expects 20:35:33 <Oxf13> We could add i586 as another build arch, which would mean another compose arch and.... 20:35:44 <mclasen> what was this talk about 'subarch' earlier ? 20:35:59 <mclasen> is that a variant where we don't build everything, but only stuff that needs it ? 20:36:00 <pjones> Oxf13: could we do it as another build arch without doing it as another compose arch? and then let them handle compose as a secondary arch? 20:36:02 <mjg59> mclasen: If we could just provide a subset of packages that were built with 586 20:36:04 <mclasen> do we have that capability ? 20:36:08 <mjg59> mclasen: Except that's potentially "everything" 20:36:18 <mclasen> you can't predict the future... 20:36:30 <mclasen> if it turns into everything, subarch turns into secondary arch... 20:36:31 <Oxf13> pjones: maybe. I'm a bit fuzzy if we can have i586 and i686 show up as two completely separate arches 20:36:43 <pjones> mclasen: no, but it stands to reason that there's a fairly high chance it'll turn to "everything" 20:37:07 <ajax> mclasen: i think "subarch" before was being used in the sense of "rpm's notion of i686 being a superset of i586" 20:37:08 * gholms rings the 15-minute bell 20:37:27 <pjones> ajax: mclasen: yeah, ajax is right about what I meant when I said "subarch as a secondary" 20:37:30 <dgilmore> problem with doing i586 and i686 is that everything will end up getting the i686 rpms in teh buildroot and thinsg taht get statically linked will link against i686 and not i586 objects 20:37:34 <ajax> i think we have things we need to know before we can move on: 20:37:40 <ajax> (ie, decide on f14) 20:37:52 <ajax> - whether we can do 586 and 686 separately within koji as primaries 20:38:02 <ajax> - what running a secondary would really entail 20:38:09 <ajax> - what the storage requirements would be 20:38:23 <mclasen> I concur. we need to do some fact-finding and revisit f14 next week 20:38:26 <ajax> so i move to _not_ continue, and come back in a week with more info 20:38:32 <mjg59> Yeah, that sounds fair 20:38:41 <cjb> sounds right to me too, thanks. 20:38:43 <pjones> ajax: reasonable. 20:38:43 <mjg59> Anyone disagree? 20:38:44 <dsd_> who are we expecting to do this work? 20:38:45 <ajax> (i won't be here next week, but that just makes it not my duty) 20:38:52 <pjones> I won't either. 20:39:04 * gholms reminds mjg59 that the minutes still need an #agreed for f13 20:39:16 <dgilmore> ajax: we cant do them seperatly in a single koji 20:39:26 * mclasen is willing to do some digging 20:39:30 <mjg59> #agreed dsd_ to post a patch to disable 686 assembler optimisations for glibc for f13 20:39:49 <gholms> Thanks 20:39:52 <cjb> mclasen: thanks! 20:40:00 <dsd_> thanks mclasen 20:40:17 <dsd_> (i'm testing the patch now) 20:40:18 <mjg59> #agreed more research on the details of building i586 separately to be carried out before deciding on f14 20:40:26 <mjg59> Everyone ok? 20:40:40 <cjb> Yep, thanks for your time, all. 20:41:01 <mjg59> Ok 20:41:05 <pjones> maybe we should add to ajax's list of things we need to know: - do gcc plan to change that switch by default? 20:41:18 <mjg59> #topic #394 F14Feature: MeeGo 1.0 https://fedoraproject.org/wiki/Features/MeeGo_1.0 20:41:35 * LinuxCode gets big eyed 20:41:49 <cjb> pjones: (Jakub kinda threatened it in the bug -- "gcc should switch to telling as to use i686 optimized code for -march=i686, so all packages will eventually have i686 nops.") 20:41:51 <mjg59> Looks fine to me. Anyone have any concerns? 20:42:10 <pjones> cjb: oh, pooh. 20:42:27 * mclasen is a bit dubious on this kind of feature in general 20:42:38 <mclasen> fedora is an OS, meego is an OS 20:42:47 <mclasen> meego-in-fedora is what ? 20:42:53 <LinuxCode> is this referring to Intels new chips or something ? 20:42:54 <pjones> mclasen: likewise, but at the same time: so what? why stop them? 20:42:55 <mjg59> A trademark violation? 20:43:13 <drago01> a sub OS ;) 20:43:16 * LinuxCode is confused 20:43:18 <mclasen> yeah, don't mean that as stop-energy 20:43:39 <mjg59> It may have to be something like "Fedora 14 implements the Meego software platform" or some such 20:43:42 <mclasen> just some lingering doubt about features that don't necessarily enhance fedora, but replace it 20:43:56 <drago01> LinuxCode: this is a merger of moblin and maemo has nothing to do with "new chips" 20:44:02 <LinuxCode> drago01, I know 20:44:07 <cjb> mclasen: they're talking about the "MeeGo Netbook UX", which is just a UI, I think. 20:44:20 <LinuxCode> I was just confused, why thats there in the first place 20:44:43 <mjg59> Shall we vote? 20:44:45 <ajax> the documentation is broken, at a minimum. one place they say yum install, another they say yum groupinstall. 20:44:53 <pjones> I'm +1 on letting them do their thing in their packages. 20:44:57 <ajax> but the intent is clear 20:44:59 <mclasen> cjb: I'd say thats about as meaningful as stating 'vista is just a UX' 20:45:12 <cwickert> mclasen: if this is not a feature, I wonder how your polkit authentication agent reorganization can be a feature 20:45:20 <LinuxCode> be nice if we would get Hildon into Fedora itself 20:45:22 <ajax> and yeah, have fun i guess. +1. 20:45:44 <mjg59> Yeah, +1 20:45:47 <cwickert> +1 20:45:48 <mclasen> I'm +/- 0 here 20:45:59 <SMParrish_mobile> +1 20:46:05 <pjones> that's 5 20:46:12 <mjg59> #agreed Meego 1.0 is an F14 feature 20:46:27 <mjg59> #topic #395 F14Feature: Sugar 0.90 https://fedoraproject.org/wiki/Features/Sugar_0.90 20:46:31 <mjg59> .fesco 395 20:46:31 <zodbot> mjg59: #395 (F14Feature: Sugar 0.90 https://fedoraproject.org/wiki/Features/Sugar_0.90) - FESCo - Trac - https://fedorahosted.org/fesco/ticket/395 20:46:48 <mjg59> Entertainingly similar 20:46:53 <pjones> indeed. 20:46:59 <ajax> up, enter 20:47:03 <mjg59> +1 20:47:05 <LinuxCode> hehe 20:47:10 <cwickert> ...and one more reason to have Fedora work on the Geode 20:47:13 <pjones> I'm +1 on letting them do their thing in their packages. 20:47:16 <cwickert> +1 20:47:29 <SMParrish_mobile> +1 20:47:32 <mclasen> +1 20:47:41 <ajax> 5, done. 20:47:45 * mclasen votes refreshingly self-inconsistent 20:47:47 <mjg59> #agreed Sugar 0.90 is an F14 feature 20:47:52 <LinuxCode> mclasen, haha 20:48:02 <mjg59> #topic #396 F14Feature: systemd - https://fedoraproject.org/wiki/Features/systemd 20:48:05 <mjg59> .fesco 396 20:48:06 <zodbot> mjg59: #396 (F14Feature: systemd - https://fedoraproject.org/wiki/Features/systemd) - FESCo - Trac - https://fedorahosted.org/fesco/ticket/396 20:48:10 <mclasen> LinuxCode: think of the kids, instead of the telcos 20:48:13 <mclasen> or something 20:48:21 <LinuxCode> mclasen, biased towards kids ? 20:48:23 <LinuxCode> lol 20:48:30 <ajax> boy do i ever not care about init. 20:48:34 <pjones> yeah. always try to aim right at them. 20:48:44 <cwickert> does Lennart really want this systemd to be a feature?? AFAIKS it was Rahul who made the wiki page 20:48:52 * mclasen thinks the feature is a bit premature 20:48:57 <mjg59> Is the aim really to replace upstart in the F14 cycle? 20:49:11 <mclasen> rahul was very enthusiastic and wrote it, lennart hasn't really declared his intentions about f14 yet 20:49:11 <mjg59> (And is anyone here to speak on behalf of this feature today?) 20:49:15 <mezcalero> cwickert: i have made quite a few changes to the feature page yesterday 20:49:23 * mclasen sees mezcalero and kay in the room 20:49:25 <mjg59> mezcalero: Oh, good, you are here 20:49:32 <pjones> if lennart and rahul really want to make systemd installable in parallel with upstart, I think that's okay. If the goal is to replace upstart, I think this is silly. If they don't know the goal yet, ... 20:49:42 <mjg59> mezcalero: Is the intent to replace upstart for F14? 20:49:53 <mezcalero> so, basically my own opinion on this i have explained on the wiki page and basically goes like this: 20:49:54 <cwickert> mezcalero: when we talked on Linuxtag this sounded a bit different, you were not that enthusiastic 20:50:20 <cwickert> (only 5 days before today) 20:50:24 <mezcalero> yes, it is ambitious, but so far things look rosy, and we can at least try 20:50:33 <mclasen> lets hear lennarts opinion, shall we ? 20:50:39 <cwickert> mezcalero: if you want to do it, go for it! 20:50:40 <pjones> mezcalero: try _what_, exactly? 20:50:57 <mezcalero> if fesco agrees on this, then we should set a date where we decide whether we should adopt it by default 20:51:06 <mezcalero> pjones: the feature page says "default" 20:51:09 <mjg59> mezcalero: Well, strictly speaking that's feature freeze 20:51:20 <mezcalero> mjg59: well, yes, but i'd like to see an earlier date for that 20:51:23 <mjg59> Sure 20:51:27 <LinuxCode> quote "we do not recommend users to convert to using systemd services yet for this release" ??? 20:51:41 <drago01> LinuxCode: has not been any different for upstart 20:51:41 <mezcalero> so, i would like to keep the options open, but i'd like to see the testing for the case "systemd as default" 20:51:42 <mjg59> LinuxCode: We haven't recommended that people use upstart native scripts yet either 20:51:43 <ajax> LinuxCode: which is what we do with upstart. 20:51:52 <LinuxCode> k 20:51:56 <LinuxCode> kk 20:51:58 <LinuxCode> sorry for noise 20:51:59 <mjg59> mezcalero: The feature owner is always free to decide to stop pushing it part way through the release 20:52:13 <ajax> and i'm happy to say no to anything you like ;) 20:52:33 <pjones> I don't have a problem with you trying to make sure it's ready to _try_ to use as default fairly early 20:52:41 <mezcalero> mjg59: well, yes, but i want to make clear that with this proposal this case is not unlikely 20:52:47 <mjg59> mezcalero: If you're confident that it'll behave itself, I don't have any problem with aiming for it to replace upstart - but I'd like to see the initial transition happen early, so we have good confidence that it works for everyone 20:52:47 <pjones> I think we need to have a longer discussion as to if that's what we're going to do though, even if you do have it ready. 20:53:02 * cwickert just wanted to make sure this feature is supported by the developer. if so, it would really be a great feature and we should support mezcalero 20:53:34 <mjg59> As long as features *work*, I think Fedora is a good place to push them 20:53:37 <LinuxCode> the feature sounds good to me 20:53:38 <mezcalero> i think what be very valuable is to get the testing 20:53:49 <mezcalero> and if then it turns out not to fly for f14 20:53:53 <pjones> mjg59: that being said, that's a compelling argument for making it optional 20:53:54 <mezcalero> then we take a step back 20:53:55 <mclasen> mezcalero: can I boot rawhide with the current packages today ? 20:53:59 <mezcalero> and we are prepared to step back 20:54:07 <pjones> default is a much more in depth discussion that hasn't really been had here. 20:54:11 <mezcalero> mclasen: yes, you should 20:54:12 * drago01 notes -ENOSELINUX (yet) 20:54:15 <mjg59> So I'd lean +1 on the assumption that we'll look very carefully at it part way through the cycle to determine whether we think it's plausible to ship it 20:54:28 <mezcalero> mclasen: i run f13 with minor updates though, not rawhide itself 20:54:34 * mclasen notes a nose in ENOSELINUX 20:54:53 <LinuxCode> I dont think it should be default though 20:54:54 <mezcalero> well, the NOSELINUX this is a prob, but i think not too hard to fix 20:55:06 <mezcalero> i.e. mostly we need dwalsh to get the policy right 20:55:13 <mezcalero> upstart has no selinux support atm at all 20:55:17 <mezcalero> except for a policy 20:55:20 <mjg59> mezcalero: Are you aware of any other cases where it's not at reasonable feature parity? 20:55:27 <mjg59> s/feature/integration/ as appropriate 20:55:48 <mezcalero> i want closer selinux integration in the long run (i.e. that systemd loads the policy, instead of the initrd, but that's not really important now) 20:55:49 <cwickert> dwalsh is very responsive for this kind of changes, this shouldn't prevent us from changing 20:55:52 <mgoetze> i believe russell coker has previously offered to patch "any init system" to support selinux (though perhaps that was in a debian context) 20:56:02 <mezcalero> mjg59: i listed the missing parts on the page 20:56:17 <mjg59> mezcalero: Ok, that's what you'd consider to be the full list? 20:56:21 <mjg59> mezcalero: Just wanted to make sure 20:56:31 <mezcalero> mjg59: yes, i am pretty sure it is complete 20:56:35 <mjg59> Ok 20:56:39 <mjg59> Anyone else have any questions? 20:56:53 <mezcalero> mjg59: oh, and regarding the item "We need sysv compatible implementations for /bin/reboot, /bin/shutdown an" part: we can actually split that off upstart 20:56:54 <ajax> do i get to see before/after numbers for boot speed? 20:56:56 <mezcalero> and use that 20:57:05 <pjones> bootcharts would be good 20:57:08 * mclasen is going to work with mezcalero on getting the feature page fully staffed in terms of roadmap, testing, and fallback 20:57:09 <mezcalero> so the absolute minimal adoption would require only the man pages 20:57:24 <mezcalero> pjones: i actually have plenty of those 20:57:33 <pjones> I'm +1 to letting you go ahead as a feature with the caveat that I'm not necessarily +1 to switching the default. 20:57:41 <LinuxCode> mezcalero, be nice to see those on the page 20:57:46 <pjones> (and I actually lean against it so far) 20:57:55 <mjg59> pjones: I think the default is something that we can revisit partway through the cycle 20:58:05 <mjg59> It's obviously hard for us to make that decision now 20:58:09 <pjones> mjg59: yeah 20:58:16 <mezcalero> pjones: however, to be frank they are not particularly impressive on rotating media without readahead, which is why i haven't published them; and readahead is currently upstartified and i haven't deupstartidized it yet 20:58:54 <mclasen> obviously, we can only make the default-or-not decision at feature freeze time 20:58:56 <mezcalero> on ssd systemd is much nicer though, regardless of readahead 20:59:00 <mjg59> Ok. So for now, should we just vote on it as a feature rather than inherently as a default? 20:59:01 <drago01> well doing everything in parallel is pretty much expected to end in "disk seek madness" 20:59:01 <ajax> +1 to go ahead and see how far this gets. 20:59:05 <cwickert> +1 for systemd and trusting mezcalero 20:59:05 <drago01> on rotating media 20:59:11 <mezcalero> mclasen: well, but it would be nice if rawhide would ship it enabled by default 20:59:12 <drago01> but readahead should help 20:59:17 <mezcalero> mclasen: just to get it tested 20:59:23 <mclasen> mezcalero: sure, if it boots :-) 21:00:02 <wacka> mezcalero: how would you enable it by default, create a /sbin/init symlink or mangle the boot loader config? 21:00:07 <mezcalero> so yepp, again i am aware this is ambitious, but i'd really like to see the testing and i am happy to revert it later if this really doesn't work out 21:00:14 <mclasen> shall we vote on switching rawhide to systemd for some time during f14, separately ? 21:00:16 <mjg59> Ok, I'm +1 21:00:36 <mjg59> Any other votes? We're at +3 so far. 21:00:43 <mclasen> +1 also 21:00:45 <mezcalero> and even if we don't pull it off for f14, the work is not lost after all, and will be good for f15 21:00:45 <SMParrish_mobile> +1 21:00:48 <mjg59> Ok 21:00:57 <mezcalero> wacka: symlink 21:00:57 <mjg59> #agreed Systemd is a feature for F14 21:01:01 <mezcalero> ah, yay! 21:01:05 <mezcalero> yippieh 21:01:07 <pjones> mclasen: I think we need to discuss if switching for the release is a goal before we talk about switching for some time in the middle. 21:01:13 <mjg59> mezcalero: And if you break everything, we break your legs 21:01:34 <mclasen> pjones: the rawhide testing would be valuable, even if it remains optional in f14 21:01:40 <mjg59> Ok, that's the end of the tickets 21:01:44 <mezcalero> mjg59: as long as you don't break my typing fingers i am fine 21:01:45 <mclasen> at least thats how I understand mezcaleros request 21:02:00 <mjg59> I, uh, can't remember how to do the engineering services report. Does anyone have anything on that? 21:02:02 <mezcalero> mclasen: yes, you understood me ;-) 21:02:11 <pjones> mclasen: true, but that same rawhide time is a scarcity I'm not sure is wise to spend on our non-default init. 21:02:38 <mezcalero> one of the main reasons i want to see this as default is btw that i can ask people to ship native service files 21:02:51 <mezcalero> if it is not default convincing people to do that will be much harder 21:03:09 <pjones> I think if that's the _best_ argument for switching init systems, we'd have to be crazy. 21:03:57 <wacka> mezcalero: is the config file format stable enough already or do you expect incompatible changes? 21:04:26 <mjg59> #topic Open Floor 21:04:34 <gholms> FES? 21:04:45 <mezcalero> wacka: at this point i see nothing i'd want to change 21:04:47 <mjg59> Sorry, yes 21:05:07 <mjg59> #topic FES ticket updates - https://fedorahosted.org/fedora-engineering-services/report/6 21:05:17 <mezcalero> wacka: and the format is well-known, and understood as it is just .desktop 21:05:31 <mezcalero> wacka: so it's not exactly something completely new which needs to settle 21:05:44 <wacka> sure, but the keywords you use etc are systemd specific 21:05:52 <mjg59> Anyone want to talk about FES? 21:06:32 <mjg59> Man. And I went to all the trouble of finding that URL and everything 21:06:51 <gholms> mmcgrath isn't here... 21:06:54 <mjg59> Yeah 21:06:57 <mjg59> Oh well 21:06:57 <LinuxCode> indeed 21:07:00 <mjg59> #topic Open Floor 21:07:01 <LinuxCode> hes off 21:07:08 <mjg59> We'll come back to that next week 21:08:36 <mjg59> Well, isn't everyone dull 21:08:37 <mjg59> Ok 21:08:43 <LinuxCode> lol 21:08:46 <gholms> [a cricket chirps in the distance] 21:08:46 <mjg59> Thanks, everyone! 21:08:52 <mjg59> #endmeeting
Attachment:
signature.asc
Description: PGP signature
-- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel
