Re: Requiring package test instructions

Przemek Klosowski <przemek.klosowski@xxxxxxxx> · Wed, 13 Jul 2016 13:23:04 -0400



    On 07/13/2016 01:38 AM, Siddhesh
      Poyarekar wrote:

    
      On Tue, Jul 12, 2016 at 10:26:20PM -0700, Adam Williamson wrote:

      
        It would not be 'a lot of work', it would be a gigantic, totally
unsustainable burden. I honestly think you're shooting *way* too high
here. Even with all the recent volunteers, we have like a couple dozen

      
      I agree it is a massive task, which is why it hasn't gotten off the
ground for glibc over the last year.  However I remain optimistic that
someone someday will do at least a fraction of the automation :)

    
    There are many levels of functionality before you get to the
    coverage you're talking about.

    I have bugzilla cases against distributed packages that 

    
      failed to correctly update during routine system updates (last
        one yesterday, 1355916)
      failed to start (crashed before getting to first
        prompt/window)

      
      started the application but crashed on a simplest function
    
    I think the functionality you're talking about (checking
      correctness of bug fixes, etc) should be left to the original bug
      reporters. After all, they raised the issue so they are invested
      in the result.
    Automation of those checks is tricky: bugzilla does ask for
      'steps to reproduce' and actual vs. expected results but we'd have
      to really come up with a better technology to translate that into
      automated test cases. It's something that looks desirable and even
      doable, but not quite easy.

    
--
devel mailing list
devel@xxxxxxxxxxxxxxxxxxxxxxx
https://lists.fedoraproject.org/admin/lists/devel@xxxxxxxxxxxxxxxxxxxxxxx