=========================================== #fedora-meeting: proventesters (2011-09-21) =========================================== Meeting started by nirik at 19:00:29 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2011-09-21/proventesters.2011-09-21-19.00.log.html Meeting summary --------------- * init process/agenda (nirik, 19:00:29) * What's required of proventesters? (nirik, 19:03:49) * LINK: http://fedoraproject.org/wiki/Proven_tester (nirik, 19:03:50) * Resources (nirik, 19:06:37) * LINK: https://admin.fedoraproject.org/updates/rss/rss2.0?comments=True (nirik, 19:09:24) * F14 (nirik, 19:13:01) * LINK: https://admin.fedoraproject.org/updates/rss/rss2.0?package=ipython&release=F16 (nirik, 19:24:38) * LINK: http://lists.fedoraproject.org/pipermail/test/2011-September/thread.html (Southern_Gentlem, 19:29:57) * AGREED: will meet next week at 18UTC. (nirik, 19:58:16) Meeting ended at 19:58:37 UTC. Action Items ------------ Action Items, by person ----------------------- * **UNASSIGNED** * (none) People Present (lines said) --------------------------- * nirik (106) * Southern_Gentlem (27) * mcloaked (26) * Viking-Ice (22) * tflink (21) * Cerlyn (14) * OldFart (14) * brunowolff (5) * athmane (5) * zodbot (3) * jsmith (3) -- 19:00:29 <nirik> #startmeeting proventesters (2011-09-21) 19:00:29 <zodbot> Meeting started Wed Sep 21 19:00:29 2011 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:29 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:00:29 <nirik> #meetingname proventesters 19:00:29 <nirik> #topic init process/agenda 19:00:29 <zodbot> The meeting name has been set to 'proventesters' 19:00:37 <nirik> any proventesters around today? 19:00:45 <OldFart> yes. 19:00:56 <tflink> yep 19:01:05 <nirik> cool. 19:01:16 * nirik will wait a few minutes and see if we get any more folks. 19:01:43 <mcloaked> I am here - 19:01:53 <nirik> welcome 19:01:56 <Cerlyn> I'm here 19:02:34 * Southern_Gentlem 19:02:44 <nirik> excellent. A few folks. ;) Anyone here _not_ currently a proventester? 19:02:55 * jsmith is not 19:03:10 <nirik> jsmith: cool. It's easy to become one... 19:03:20 <jsmith> nirik: I imagine it is :-) 19:03:30 <jsmith> Don't mind me though... I'm mostly just lurking 19:03:43 <nirik> ok, let me do some intro materal and then we can move on to real business... 19:03:49 <nirik> #topic What's required of proventesters? 19:03:50 <nirik> http://fedoraproject.org/wiki/Proven_tester 19:04:01 <nirik> So, the above wiki page is the main landing page about proventesters. 19:04:48 <nirik> so, you can find info there about joining and what proventesters need to do. 19:05:09 <nirik> basically to join you have to agree that you have read and understand what you are supposed to do. ;) 19:05:37 <nirik> Any questions on process or the like? should be able to answer them by reading that page... (and it's links) 19:06:33 <nirik> ok, I'd like to talk process/tools for a minute. 19:06:37 <nirik> #topic Resources 19:06:50 <nirik> If folks aren't currently using them or know about them: 19:06:51 <mcloaked> Quick question: do you know how many proventesters there are now? 19:07:03 <nirik> mcloaked: yeah, I looked this morning. There's 80 people in the group. 19:07:11 <Cerlyn> A better question might be how many are active 19:07:20 <nirik> yeah, thats much harder to determine. ;( 19:07:37 <tflink> we could grab recent bodhi comments and compare to the list of members 19:07:45 <nirik> * fedora-easy-karma makes it easy to submit karma on updates if you have a bunch installed. Very useful. 19:07:54 <mcloaked> True - though does bodhi keep a track count of comments/karma additions? 19:08:01 <nirik> * bodhi has rss feeds if you want to know about just one packges updates. 19:08:13 <nirik> mcloaked: not sure. 19:08:25 <tflink> mcloaked: not sure what it keeps track of by user, I'm usually grabbing data for updates 19:08:34 <nirik> * bodhi command line can be great for pulling down updates that aren't in testing yet. 19:08:37 * brunowolff is here, but was doing work stuff when the meeting started. 19:08:59 <nirik> There's also an overall comments rss feed 19:09:21 <Southern_Gentlem> ?brb 19:09:24 <nirik> https://admin.fedoraproject.org/updates/rss/rss2.0?comments=True 19:09:34 <mcloaked> Yes often getting rpms that have yet to reach the repo is useful. 19:10:04 <tflink> either way, it is doable - even if that does involve some ugly code that pulls a crapton of data from bodhi :) 19:11:05 <nirik> bodhi also has a metrics page: https://admin.fedoraproject.org/updates/metrics/?release=F16 19:11:53 <nirik> So, I thought I would focus on f14 today... but I'm happy to discuss anything else folks feel is related and important... 19:12:57 * nirik listens to crickets. 19:13:01 <nirik> #topic F14 19:13:31 <nirik> so, there's currently 12 pending security updates in f14 19:13:50 <nirik> and 15 updates in critpath 19:14:01 <nirik> (there is some overlap there) 19:14:29 <nirik> Does anyone here have f14 instances still? ;) 19:14:29 * nirik has a vm 19:14:42 <tflink> my laptop is still running F14 19:14:46 <OldFart> I have F14 19:14:48 * Cerlyn has VMs and XOs 19:14:56 <Southern_Gentlem> ?can there be a mailing list other than testing that gets all the noise from qa 19:15:10 <brunowolff> I don't really have any f15 instances. Just an old laptop, that I only use for special stuff and will upgrade probably the next time I touch it. 19:15:31 <brunowolff> Everything I normally use is f16 or f17 now. 19:15:39 <nirik> Southern_Gentlem: what email/info do you want to get thats lost in the noise currently? 19:16:01 <OldFart> The last time I looked at F14 security updates, I didn't feel qualified to test them. 19:16:08 <brunowolff> And I mostly have older hardware, so haven't been too interested in setting up VMs. 19:16:52 <nirik> some of them would seem to be pretty easy... like the kernel... 19:17:19 <nirik> but others are more esoteric. 19:17:19 <nirik> Would having more test plans help? 19:17:42 <tflink> for some things, yes 19:17:54 <OldFart> Yeah, the kernel, but the fixes are not known to me. I can +1 from casual use. Is that OK? 19:17:56 <tflink> I've run across more than one update that I haven't the foggiest idea how to test 19:18:11 <Southern_Gentlem> nirik, stuff for f14 and f15 that is waiting for testing 19:18:13 <Cerlyn> A large part of my testing is not so much based on test plans, but trying to replicate the issue that was fixed 19:18:39 <tflink> how to hit the code that was changed to verify that it doesn't explode, rather 19:19:08 <nirik> Southern_Gentlem: so the updates-testing report? could you filter on that? or perhaps we could add a topic for it to make it easier to filter on? 19:19:11 <Cerlyn> yes; it doesn't make sense in my view to test the kernel for example if I don't have a computer with the updated module 19:19:17 <mcloaked> For kernel I still think it is useful to get comments if there are no regressions, but it would be helpful to have notes to remind people to install not just the appropriate kernel rpm but also the related perf, kernel-headers etc (I know perf has gone from the kernel builds for f16)? 19:19:32 <nirik> OldFart: yes. You don't need to verify everything, you need to verify that it shows no regressions for you and isn't breaking. 19:19:43 <Southern_Gentlem> nirik, to have the updates testing report sent to a proven testers list 19:20:23 <nirik> that seems like kinda a waste for an entire list, IMHO 19:21:00 <Southern_Gentlem> nirik, i think the reason stuff is not getting tested is no notification system for the proven testers 19:21:32 <Southern_Gentlem> we all get busy doing our things and that falls through the cracks 19:21:35 <OldFart> Used to be mail notifications, but too many complained about spam.... 19:21:36 <nirik> for critpath packages, remember you simply need to test: http://fedoraproject.org/wiki/Critical_path_action#Actions 19:22:16 <nirik> OldFart: right. 19:22:41 <mcloaked> If a proventester is subscribed to the test list then a list of critpath and security updates gets sent out fairly regularly? 19:22:47 <nirik> I am sure if we started mailing proventesters on every updates push people would complain again. 19:22:48 <Southern_Gentlem> nirik, or a compromise is that the package has sat there with no testing and the maintainer sends it to a mailing list 19:22:55 <nirik> mcloaked: yes, every push 19:23:19 <OldFart> Feasible to have an opt in mailing list just for testers? 19:23:26 <nirik> Subject: Fedora N updates-testing report 19:23:34 <nirik> OldFart: I would call the 'test' list that. 19:23:53 <OldFart> OK... 19:24:03 <Cerlyn> Goes back to my email earlier yesterday/today on wanting to be able to subscribe to package updates like one can for bug reports and commits. (I was trying to phrase it in ways that won't overload infrastructure) 19:24:19 <nirik> Cerlyn: would the rss feeds meet that need? 19:24:38 <nirik> https://admin.fedoraproject.org/updates/rss/rss2.0?package=ipython&release=F16 19:24:49 <Cerlyn> nirik: The full updates list per release would, if an RSS reader exists which can filter 19:24:58 <Cerlyn> I don't want to subscribe to every sugar* package out there 19:25:10 <mcloaked> By the way if you add yourself to the bugzilla report for a package you are interested in then you will get mail for new comments - and that will usually include mail reporting when a fix package has been built. 19:25:12 <Cerlyn> that's a burden on infrastructure, and I was an infrastructure person 19:25:44 <Cerlyn> mcloaked: But does the package owner have to approve those memberships? I was trying to figure out the system 19:25:44 <nirik> right, so you want a collection of packages, but in one feed or the like... 19:25:51 <Southern_Gentlem> test list to me is more for QA and (yes proventesters can follow that if they wish be it looks like alot of spam for people who want to test updates for current releases 19:26:03 <brunowolff> This is from back a bit, but perf isn't gone, the sub-package name changed to kernel-tools. 19:26:47 <mcloaked> Cerlyn: I think the package owner does not need to approve if you add yourself to the bug cc: list 19:27:02 <nirik> Cerlyn: no approval needed, just a bugzilla account. 19:27:24 <nirik> Cerlyn: you can also add yourself in pkgdb to packages you are interested in to be cc'ed on all bug reports against that package. 19:27:33 <nirik> (get ready for lots of email on some packages though) 19:27:43 <Cerlyn> It's a side topic but I noticed a lot of packages didn't have a QA contact in pkgdb 19:27:55 * Viking-Ice sneaks in... 19:27:58 <nirik> Cerlyn: I don't think that field is used for anything. ;( 19:28:19 <Cerlyn> bugzilla uses it IIRC 19:28:27 <mcloaked> brunowolff: thanks - about perf->kernel-tools 19:28:56 <nirik> Southern_Gentlem: I guess if there's enough interest for a new list for the updates emails, we could try it... perhaps we should ask on the test list if there is enough interest? 19:29:55 <Southern_Gentlem> example 19:29:57 <Southern_Gentlem> http://lists.fedoraproject.org/pipermail/test/2011-September/thread.html 19:30:29 <Southern_Gentlem> you have so scroll past the first page to see anything about f15 or f14 updates needing tested 19:30:46 <nirik> Cerlyn: it's set to extras-qa@xxxxxxxxxxxxxxxxx for all fedora bugs... which goes to... /dev/null. ;) 19:31:13 <Cerlyn> One potentially also could generate a report on packages which have not received feedback after some time, and email that, as a warning that the packager can now push at will 19:31:17 <nirik> Southern_Gentlem: yeah, but couldn't you filter through it for the ones you want? or you mean many people aren't subscribed and are looking at the web archive? 19:32:30 <Southern_Gentlem> nirik, so why dont separate the list so just the released releases updates that need testing and not devel 19:32:40 <Viking-Ice> if you are part of the QA community you should be subscribed and all QA task requested should be posted to the test list not -devel 19:33:15 <Southern_Gentlem> nirik, i dropped the test list because of 60 messages being delivered to get the ones that deal with me 19:33:21 <nirik> Southern_Gentlem: well, I fear splitting that out would just result in another list that not many people would subscribe to, since most folks interested are already subscribed to test. 19:33:32 <nirik> but I could be wrong. 19:33:53 <tflink> either way, sounds like time to ask for input from test@ ? 19:33:59 <mcloaked> nirik: my feeling is that you are right... 19:34:01 <Southern_Gentlem> nirik, stuff isnt getting tested tells me that most feel the test list is to noisey 19:34:05 <Viking-Ice> not really we dont want split which which results in divided sub community 19:34:17 <Southern_Gentlem> Viking-Ice, you already have one 19:34:21 <nirik> yeah, lets ask for input on the test list... and see if there's enough folks who want another list... 19:34:24 <Viking-Ice> bugzapper 19:34:44 <Viking-Ice> if that's the route you are heading have at it 19:35:10 <nirik> so, moving back to f14. ;) What can we do to better handle these? would folks like to go thru one by one in the meeting? or should we look at hard to test ones? easy to test ones? 19:35:16 <Viking-Ice> we struggle hard to not divide the zapper stuff was due to autoqa only discussion on QA 19:35:17 <mcloaked> There could be other reasons - like people are busy, or for install testing when there are major issues giving bad or unbootable installs it can slow things down. 19:35:44 <nirik> I think it's mostly that people don't know they can test/provide feedback 19:36:00 <nirik> visibility is just low. 19:36:10 <tflink> you mean among proventesters or among fedora users? 19:36:22 <mcloaked> Is there any chance that f14 is being sidelined by people trying to test later (eg f16) packages and getting short of time? 19:36:28 <nirik> tflink: fedora users. 19:36:50 <tflink> I think part of it is worry that they'll tank their system 19:37:03 <nirik> I know as a package maintainer when I fix a bug, ofter the reporter will say "great, works here" in the bug, but won't add karma unless I specifically ask them to, and often not even then. 19:37:14 * athmane is late 19:37:22 <mcloaked> tflink: do testers mainly use production systems in their testing or spare machines? 19:37:31 <tflink> have no idea 19:37:38 <nirik> mcloaked: yes, f14 suffers from many things: 19:37:48 <tflink> I know that my F14 system is a production machine that I use every day 19:37:52 <nirik> * fedora is a fast moving distro, so most of the energy is on f16+ 19:38:10 <nirik> * the people running f14 just want it to keep working, so don't want to test dangerous things. 19:38:18 <OldFart> I have F14, F15, F16 and rawhide on one machine...no VMs and try to test when I see the updates testing report. But, 19:38:20 <nirik> * f14 people are less likely to be contributors I suspect... 19:38:35 <mcloaked> In my own case for potentially "tanking" tests I use a machine that won't matter if it breaks - but for production machines I will test packages selectively. 19:38:54 <OldFart> most often I am not familiar enough with the package or, if familiar, with the bug or security fix. 19:39:15 <athmane> imho f14 (fedora old stable in general) needs a less strict update policy 19:39:27 <nirik> OldFart: for that perhaps the test list or #fedora-qa could help... ie, someone who knows the package could help others test it 19:39:49 <mcloaked> OldFart: you can still install a package in critpath and check if there are no regressions - that is valid process. 19:39:49 <nirik> athmane: EBOYONDTHESCOPEOFTHISMEETING. ;) 19:40:15 <OldFart> Test cases would help....but they add a burden to the developer/maintainer. 19:41:10 <nirik> sure, but they are great to have... 19:41:22 <nirik> but most of them time currently, we don't have them 19:42:05 <nirik> as a side note, some of the f14 critpath updates should drop off the report soon... some of the obscure X drivers are dropping out of critpath 19:42:32 <nirik> qxl and openchrome 19:43:21 <mcloaked> On testcases - sometimes people could meet on here and discuss how to test a particular package perhaps? Might help where there is no testcase available? 19:43:41 <nirik> I would think these would be not too bad to verify critpath functionality on: kernel, gnupg2, curl, udev, 19:43:51 <nirik> mcloaked: absolutely. 19:44:09 <OldFart> +1 19:44:36 <nirik> that was the last item I had for today: should we bother meeting again? are meetups helpfull or useful in any way? 19:45:02 <athmane> there's a ticket for criticalpath test cases creation => https://fedorahosted.org/fedora-qa/ticket/154 19:45:13 <OldFart> Helpful....more should show up over time 19:45:17 <Viking-Ice> it should be mandantory for maintainers to provide how to debug their component and provide test cases 19:45:20 <mcloaked> Also I guess there is #fedora-qa for discussions? 19:46:11 <nirik> mcloaked: yep. 19:46:38 <nirik> athmane: good link... 19:46:47 <mcloaked> I guess that the log of meetings such as this can be seen when they are sent out to the test list - so they are useful even if only small numbers 19:46:54 <nirik> Viking-Ice: not really enforceable... but in an ideal world... 19:47:05 <athmane> we have ~15 test case of 432 critical path pkg 19:47:12 <tflink> forcing test cases != good test cases 19:47:37 <Southern_Gentlem> and no testing ==no testing 19:47:38 <tflink> does not necessarily lead to, anyways 19:47:39 <nirik> athmane: long way to go for sure. 19:48:10 <Viking-Ice> nirik, why not 19:48:14 <mcloaked> Also worth stressing that if testing leads to problems seen then it is important to file bz.... not just a comment in bodhi 19:48:16 * nirik just finished rebooting his f14 vm... 19:48:22 <nirik> mcloaked: +1 19:48:34 <Viking-Ice> nirik if you want your component in fedora you need to provide us with that info 19:48:43 <nirik> Viking-Ice: what do you do if they don't? 19:48:51 <nirik> and what if they are horrible at testing? 19:48:56 <Viking-Ice> dont introduce the component 19:49:12 <tflink> what do you do about the bad test cases that just introduce noise? 19:49:30 <Southern_Gentlem> all of this is not addressing the issues 19:49:32 <Viking-Ice> if the packager and or maintainer does not know how to debug and test his own component <--- 19:49:51 <nirik> Southern_Gentlem: suggestions? 19:50:01 <tflink> knowing how to debug and being willing to write out test cases are not the same thing 19:50:22 <Viking-Ice> mcloaked, reporters have come and gone mostly because they dont get responded to in bugzilla 19:50:42 <mcloaked> For some kinds of test fails are quite tough to get useful debug info - eg boot crash - how to get backtrace, install crash - how to get bactrace - similar for serious kernel issues when they happen even if they are rare. 19:50:56 <Southern_Gentlem> nirik, other than breaking out another list that has what needs testing nope 19:51:13 <nirik> Southern_Gentlem: ok, will float that idea to the test list... or did you want to? 19:51:17 <Southern_Gentlem> i see notification to proven testers as the bigest problem 19:51:40 <nirik> we didn't really see much more testing back when we spamed all proventesters I don't think... 19:51:46 <nirik> but there was likely fewer back then. 19:52:04 <Southern_Gentlem> nirik, go ahead but i see most of the test list are testing devel and could care less about the currcent releases or we wouldnt be in this shape 19:52:28 <Viking-Ice> yeah somepeople have thrown updates-testers of that list 19:52:34 <mcloaked> Within bodhi you can get a list of security packages such as at https://admin.fedoraproject.org/updates/F14/security?_csrf_token=0834a0859819089ae1a83c6e8e96ed3b63a57934 19:52:38 <Viking-Ice> as in GA testers 19:52:45 <nirik> sure, for mentioned reasons there's less interest in the stable releases. But really there's not that many critpath/security ones. 19:53:01 <mcloaked> Might be nice if there was also a link to current critpath package updates per Fedora version too? 19:53:17 <nirik> I really think it wouldn't take too much for a group to process thru them... I think it's reachable. 19:53:51 <Southern_Gentlem> nirik, i think a most a week email of list of packages waiting to be tested in the current releases would be easy 19:53:57 <Viking-Ice> still reporters/testers are faced with nonresponsive maintainers 19:54:04 <Viking-Ice> and EOL 19:54:07 <tflink> mcloaked: you mean other than https://admin.fedoraproject.org/updates/critpath?unapproved=True&release=F16 19:54:34 <Viking-Ice> seriously when are we going to target the root of the problem instead of working around it 19:54:41 <nirik> Southern_Gentlem: well, emailing the updates-testing reports to another list would be easy... 19:55:10 <Southern_Gentlem> nirik, only when stuff needs to be tested no status after that 19:55:11 <mcloaked> tflink: Excellent - I was unaware of that construction! I will certainly use it in the future. 19:55:12 <nirik> Viking-Ice: because there's no solution to that unless you intend to hire and task people with full time development/qa? 19:55:27 <Viking-Ice> drop the ownership model 19:55:34 <Viking-Ice> get the devs working together etc 19:55:49 <nirik> Southern_Gentlem: that would be a lot harder, as it would need bodhi code changes. ;( 19:55:50 <Viking-Ice> necessary and needed change 19:56:08 <nirik> Viking-Ice: beyond the scope of this meeting. 19:56:08 <mcloaked> tflink: It would be even nicer if that list of unapproved critpath packages was a link on the left side of the bodhi interface! 19:56:21 <nirik> Anyhow, 4 min left until next meeting... 19:56:29 <nirik> should we meet again next week at this time? another time? not at all? 19:56:30 <tflink> I'm not convinced that would work all that well, my experience is that when there's no ownership, nothing gets done 19:56:38 <Viking-Ice> nirik, yeah bury the root cause once more, nonresponsive maintainers are the root of alot of problems 19:56:48 <tflink> mcloaked: yeah, you have to click through a bunch of stuff to find that link 19:56:52 <Southern_Gentlem> nirik, and only updates-testing for current releases not devel 19:57:02 <nirik> Southern_Gentlem: devel has no updates-testing. ;) 19:57:19 <Southern_Gentlem> nirik, what does f16 go to at the moment 19:57:19 <OldFart> I'm ok with a meeting next week....prefer an hour earlier. 19:57:25 * nirik waits for votes. will close the meeting in a minute or two 19:57:30 <Southern_Gentlem> +1 19:57:34 <tflink> I'm also ok with next week 19:57:41 <mcloaked> I believe I can make next week same time 19:57:46 <nirik> next week at 18? 19:57:50 <athmane> +1 19:57:55 <Viking-Ice> +1 19:57:57 <Cerlyn> + 19:58:04 <Southern_Gentlem> +1 19:58:11 <OldFart> +1 can't stay for the hour 19:58:12 <mcloaked> +! 19:58:16 <nirik> #agreed will meet next week at 18UTC. 19:58:31 <nirik> Thanks for coming everyone... continue discussion over in #fedora-qa and on list... 19:58:37 <nirik> #endmeeting
Attachment:
signature.asc
Description: PGP signature
-- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
