Here's the recap of the Fedora 18 Graphics Test Week. Note that there are no Fedora 17 Graphics Test Week numbers here because there wasn't one! The last one was Fedora 16. So the 'outcome' numbers have the results from the F16 test week added. As expected, we got more tests and more bugs filed this time than we did in F16; the F16 numbers were low because I didn't get the publicity and organization done properly. We got it done better this time, with many thanks to Martin Holec for covering some stuff I missed (and also for running a ton of tests himself, which pumps the numbers up too :>) Here's the numbers on tests conducted and bugs filed, with all the historical numbers for comparison. I counted each non-example line in the main results matrix as a 'test conducted'. I didn't count the runs in the 'bonus' 3D matrix, in order to provide a more fair comparison with previous releases. In future, once we have enough releases with the bonus 3D tests included, I may start using those numbers. This might affect the bugs-to-tests ratio somewhat. f11 nouveau: 104 tests, 42 bugs - ratio 0.40 f12 nouveau: 53 tests, 34 bugs - ratio 0.64 f13 nouveau: 78 tests, 26 bugs - ratio 0.33 f14 nouveau: 39 tests, 8 bugs - ratio 0.21 f15 nouveau: 83 tests, 55 bugs - ratio 0.66 f16 nouveau: 16 tests, 8 bugs - ratio 0.50 f18 nouveau: 54 tests, 43 bugs - ratio 0.80 f11 radeon: 55 tests, 46 bugs - ratio 0.84 f12 radeon: 61 tests, 81 bugs - ratio 1.33 f13 radeon: 48 tests, 33 bugs - ratio 0.69 f14 radeon: 32 tests, 18 bugs - ratio 0.56 f15 radeon: 66 tests, 38 bugs - ratio 0.58 f16 radeon: 12 tests, 8 bugs - ratio 0.67 f18 radeon: 32 tests, 24 bugs - ratio 0.75 f11 intel: 23 tests, 21 bugs - ratio 0.91 f12 intel: 29 tests, 31 bugs - ratio 1.07 f13 intel: 38 tests, 38 bugs - ratio 1.00 f14 intel: 33 tests, 28 bugs - ratio 0.84 f15 intel: 37 tests, 25 bugs - ratio 0.68 f16 intel: 10 tests, 7 bugs - ratio 0.70 f18 intel: 27 tests, 7 bugs - ratio 0.26 Interesting that Intel is doing really well this cycle. The hardware tested seems to be a reasonable range, so it's not that we were only testing one small set of hardware that happens to work well. Radeon is broadly in line with historical trends. nouveau is at its 'buggiest' level ever, which is maybe worrying, though it's worth noting that quite a lot of the bugs filed related to the advanced 3D tests, and two of them were for a crash in extremetuxracer which is nothing to do with nouveau. As for participation, we had 113 tests run this time, a huge improvement on the 38 from F16, about even with F14, but still a bit below the other releases from F11 to F15, so we can still make things better for F18. Here's the chart for how reported bugs are handled, adding the results from the F16 events (this chart gets updated on a one-cycle delay, because it's concerned with how the bugs filed are handled after a reasonable amount of time for the developers to look at them, and as noted above, there was no F17 event, so we're adding the F16 results this time). A full explanation of what this tracks and how it's calculated in the F13 recap at https://lists.fedoraproject.org/pipermail/test/2010-April/090271.html and the F16 recap at https://lists.fedoraproject.org/pipermail/test/2011-November/104677.html . Also, if you try to reproduce these numbers, note that you'll want to remove the bugs 123456 and 234567 from the lists - they're used in the 'example' entries in the results matrix. The numbers for older releases are copied/pasted, not re-generated each time. F16 may gain a slight advantage here from having had a year for its bugs to be fixed, not six months. f11 nouveau: 42 bugs, 4 open, 8 closeddupe, 24 closedfixed, 6 closedunfixed - 70.59% f12 nouveau: 34 bugs, 11 open, 8 closeddupe, 14 closedfixed, 1 closedunfixed - 53.85% f13 nouveau: 27 bugs, 17 open, 6 closeddupe, 3 closedfixed, 1 closedunfixed - 14.29% f14 nouveau: 8 bugs, 5 open, 3 closeddupe - 0% (small sample) f15 nouveau: 58 bugs, 27 open, 13 closeddupe, 13 closedfixed, 5 closedunfixed - 28.89% f16 nouveau: 8 bugs, 4 open, 3 closeddupe, 1 closedunfixed - 0% (small sample) f11 radeon: 46 bugs, 14 open, 10 closeddupe, 19 closedfixed, 3 closedunfixed - 52.78% f12 radeon: 81 bugs, 19 open, 32 closeddupe, 28 closedfixed, 2 closedunfixed - 57.14% f13 radeon: 36 bugs, 28 open, 3 closeddupe, 5 closedfixed, 0 closedunfixed - 15.15% f14 radeon: 18 bugs, 13 open, 0 closeddupe, 3 closedfixed, 2 closedunfixed - 16.67% f15 radeon: 38 bugs, 17 open, 10 closeddupe, 9 closedfixed, 2 closedunfixed - 32.14% f16 radeon: 8 bugs, 5 open*, 1 closeddupe, 1 closedfixed, 1 closedunfixed - 14.29%* (small sample) f11 intel: 21 bugs, 7 open, 1 closeddupe, 12 closedfixed, 1 closedunfixed - 60% f12 intel: 31 bugs, 7 open, 12 closeddupe, 12 closedfixed, 0 closedunfixed - 63.16% f13 intel: 42 bugs, 26 open, 4 closeddupe, 11 closedfixed, 1 closedunfixed - 28.95% f14 intel: 28 bugs, 21 open, 4 closeddupe, 1 closedfixed, 2 closedunfixed - 4.17% f15 intel: 27 bugs, 8 open, 7 closeddupe, 12 closedfixed, 0 closedunfixed - 60% f16 intel: 7 bugs, 5 open, 1 closeddupe, 1 closedfixed, 0 closedunfixed - 16.67% (small sample) The low number of bugs makes these numbers pretty unreliable, really, but obviously the results aren't great for any of the drivers, the bottom line is exactly two bugs got fixed so far as a result of the F16 test days, which obviously isn't what we're aiming for. In the end though it's probably a result of the overall neglect of the event in that cycle, so we'll have to wait for the F18 numbers - where the bug count is up again - to draw any conclusions. Re the asterisks on radeon - one of the 'open' bugs is #698011, which is actually CLOSED UPSTREAM but counted under 'open' because it's open upstream. It's actually under active development right now, it looks like a patch that fixes it was posted at the end of September. So probably, if I'd done this recap a week later, that bug would be counted as closedfixed not open, and the radeon numbers would look better. Many thanks as always to all those who contributed tests and to our plucky X developers and triagers, especially: Adam Jackson Dave Airlie Jerome Glisse Ben Skeggs Matej Cepl Martin Holec The raw list of bugs filed at each of the F18 events follows. Nouveau ------- 745202 NEW - gnome-shell does not display correctly with NV3x adapters - multicolor corruption of panel, Shell-style menus and text [nvfx] 752613 NEW - Nivida 100M on Lenovo W520 Wont boot if using Discrete Graphics 835648 NEW - kernel freeze after [drm] Initialized drm 1.1.0 20060810 on T520 with Quadro NVS 4200M 848488 NEW - [abrt] extremetuxracer-0.4-9.fc17: size: Process /usr/bin/etracer was killed by signal 11 (SIGSEGV) 855560 NEW - F18 Live Alpha : nVidia GeForce 8600M GT, graphic driver (nouveau): very poor performance (vesa is fluid) 858503 NEW - display shows garbage after suspend/resume on W520 860215 NEW - [abrt]: WARNING: at kernel/power/suspend_test.c:53 suspend_test_finish+0x86/0x90() 860228 NEW - Rendering / performance issues with polymol on GeForce 310 (GT218 / NVA8) 860248 NEW - Fedora 18 AMD 64 Nouveau Desktop Suspend Fails 860256 NEW - NV41 won't suspend 860259 NEW - [abrt] extremetuxracer-0.4-10.fc18: pp::Font::advance: Process /usr/bin/etracer was killed by signal 11 (SIGSEGV) 860265 NEW - NV44 Xorg segfault after suspend 860270 NEW - pure virtual function called crash 860360 NEW - Distorted text after system booted 860389 NEW - NVC8 wont suspend 860405 NEW - Video with nVidia Corporation G73 [GeForce 7300 GT] does not come back after suspend 860407 NEW - system hangs while playing a video via totem and on pressing the windows key 860477 NEW - nouveau Xorg crash following boot (GTX 580) 860584 NEW - Incorrect video wake up with Nvidia G72M (Geforce Go 7300) and nouveau after suspend 860611 NEW - Freezed and/or black Xorg output (with cursor) on G72M with nouveau on 2nd display configuration (multihead) 860624 NEW - Gnome-shall hangs when requested activities 860628 NEW - [abrt] mesa-demos-7.10-8.20101028.fc18: exit: Process /usr/lib64/mesa/fbo_firecube was killed by signal 6 (SIGABRT) 860637 NEW - Gnome-shell is slow after suspend-resume 860660 NEW - Transparency rendering problems on PyMOL with HD4890 and radeon driver 860956 NEW - Resume from suspend fails on GeForce 7300 GT 861106 NEW - X Test Day: Video breaks when secondary monitor is connected 861280 NEW - KDE Crashes or Hangs After Moving Konqueror Window Quickly 861624 NEW - NV43 (Geforce 6600GT) suspend artifact 861625 NEW - NV43 (Geforce 6600GT) visual artefacts when running pymol 861628 NEW - NV43 (Geforce 6600GT) tux racer doesn't restore default screen resolution when exiting 861646 NEW - Sporadic GPU Lockups when Fast User Switching 861656 NEW - Rendering / performance issues with polymol on GeForce 6150SE 861746 NEW - GDM login screen artefacts and corruption with Geforce 210 861939 NEW - [Nouveau] Missing some textures in fbo_firecube, teapot and reflect mesa demos 861946 NEW - Slow rendering in OpenArena 861951 NEW - Resume from suspend slowly freezes input. 861960 NEW - Disabling primary display when secondary is enabled blanks both of them 861969 NEW - Rendering glitch after resume from suspend 862026 NEW - Rendering glitch after opening kickoff application launcher 857267 ON_QA - SELinux is preventing /usr/bin/totem-video-thumbnailer from 'create' accesses on the directory gstreamer-1.0. 806491 ON_QA - systemd-logind not tracking startx sessions 862363 CLOSED UPSTREAM - OpenGL rendering on Quadro 310 10de:107d is not supported 857662 CLOSED DUPLICATE - [mesa] libGL error: failed to open drm device Radeon ------ 790365 NEW - Can't resume from suspend to RAM: Black screen and laptop frozen 847818 NEW - Radeon RS690M/X1200 screen corruption beyond kernel-3.4.6-2.fc17.i686 848488 NEW - [abrt] extremetuxracer-0.4-9.fc17: size: Process /usr/bin/etracer was killed by signal 11 (SIGSEGV) 849347 NEW - radeon driver cannot handle restarting xorg server sometimes (fails with: *ERROR* Failed to parse relocation -35!) 860451 NEW - Artifact between apply and rotated screen 860628 NEW - [abrt] mesa-demos-7.10-8.20101028.fc18: exit: Process /usr/lib64/mesa/fbo_firecube was killed by signal 6 (SIGABRT) 860660 NEW - Transparency rendering problems on PyMOL with HD4890 and radeon driver 860668 NEW - artifacts on xrestart with rv710 860779 NEW - Radeon HD6570 hangs on VT switching 860783 NEW - Radeon HD6570 GLX test failed (radeon test day) 860789 NEW - Multihead displays shows artifacts 860794 NEW - openarena failed to start on Radeon HD6570 (radeon test day) 860798 NEW - ExtremeTuxRacer crashes on HD6570 with a free driver 860843 NEW - [abrt] extremetuxracer-0.4-10.fc18: radeon_bo_destroy: Process /usr/bin/etracer was killed by signal 11 (SIGSEGV) 861755 NEW - X fails to start or display after reboot on ATI FirePro V4800 861772 NEW - Xv video playback in Totem lags audio by several seconds on ATI FirePro V4800 861831 NEW - GNOME 3 Activities panel takes several minutes to appear after login on ATI FirePro V4800 861946 NEW - Slow rendering in OpenArena 862372 NEW - etracer killed with SIGABRT on quit 860712 ASSIGNED - [6550D SUMO] problems with second monitor on VGA, causing GPU lockups Intel ----- 847479 NEW - Kernel 3.5.0-2 and 3.5.1-1 nouveau driver causes Xorg to abrt after resume from RAM 848488 NEW - [abrt] extremetuxracer-0.4-9.fc17: size: Process /usr/bin/etracer was killed by signal 11 (SIGSEGV) 860551 NEW - black screen fedora 20120924-test_days-x86_64 861144 NEW - Can't log in with fast user switch after use gstreamer 861219 NEW - Switching to discret GPU render system unusable 862190 NEW - X Test Week: warning: Failed to initialize plugin: '/usr/lib64/grilo-0.2/libgrlflickr.so' 861166 CLOSED DUPLICATE - [abrt] python-2.7.3-13.fc18: CShaderPrg_Enable: Process /usr/bin/python2.7 was killed by signal 11 (SIGSEGV) -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net _______________________________________________ test-announce mailing list test-announce@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/test-announce -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
