On Wed, Jan 18, 2017 at 11:55:38AM -0700, Kevin Fenzi wrote: > On Wed, 18 Jan 2017 07:22:49 -0500 (EST) > Charalampos Stratakis <cstratak@xxxxxxxxxx> wrote: > > > Python 3 started failing on i686, x86_64 and arm only. > > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=17313924 > > > > The failures come from the test_socket.py [0], test_aead_aes_gcm [1] > > and more specifically this line [2] with the message: OSError: [Errno > > 22] Invalid argument > > > > This test is checking Python's interface for the kernel crypto API > > (added in 3.6 [3]). > > > > [0] > > https://hg.python.org/cpython/file/3.6/Lib/test/test_socket.py#l5473 > > [1] > > https://hg.python.org/cpython/file/3.6/Lib/test/test_socket.py#l5472 > > [2] > > https://hg.python.org/cpython/file/3.6/Lib/test/test_socket.py#l5497 > > [3] http://bugs.python.org/issue27744 > > Thats nothing to do with the buildsystem. Or at least if it is, it's > not any of the known issues I was working on. > > As far as I know now everything should be back to working/normal. > > There were 3 issues: > > 1. The buildvm's were updated and started crashing under load. They > would spew oopses and finally reboot. This would result in builds on > them getting restarted or dying in strange ways. This seems to be > something particular to their hardware/setup that changed in later > 4.8.x and early 4.9.x kernels. We are currently running 4.10rc4 on them > and it's been stable (knock on wood). > > 2. Sometimes, very rarely dnf would fail downloading packages for the > build root. We have worked around this by passing dnf via koji several > mirrors where it can download from, so if it fails on one it should > fall back to the next and so on. > https://pagure.io/fedora-infrastructure/issue/5689 > > 3. In some configurations (where there was more than I host to download > from) koji would fail to download the src.rpm correctly and error out. > We are working around this now by just pointing koji at one place for > these downloads for now until we can get things fixed. > https://pagure.io/fedora-infrastructure/issue/5694 > > Sorry for all the instability the last few days. > > If you see any further issues like the above, let me know. I'm still seeing quite alot of problems today - randomly the arm7 or ppc task will just sit there in "open" status doing nothing for hours. If I cancel & rerun the build it works fine, presumably because it hit a different build instance. Now I've just seen some builds which succeeded in the build phase get marked as failed because mock throws an exception handling the results https://kojipkgs.fedoraproject.org//work/tasks/1011/17331011/mock_output.log Finish: run ERROR: list index out of range Traceback (most recent call last): File "/usr/libexec/mock/mock", line 885, in <module> raise File "/usr/lib/python3.5/site-packages/mockbuild/trace_decorator.py", line 89, in trace result = func(*args, **kw) File "/usr/libexec/mock/mock", line 704, in main File "/usr/lib/python3.5/site-packages/mockbuild/trace_decorator.py", line 89, in trace result = func(*args, **kw) File "/usr/lib/python3.5/site-packages/mockbuild/buildroot.py", line 545, in finalize self._unlock_buildroot() File "/usr/lib/python3.5/site-packages/mockbuild/trace_decorator.py", line 89, in trace result = func(*args, **kw) File "/usr/lib/python3.5/site-packages/mockbuild/mounts.py", line 162, in umountall File "/usr/lib/python3.5/site-packages/mockbuild/trace_decorator.py", line 81, in trace frame = inspect.getouterframes(inspect.currentframe())[1][0] File "/usr/lib/python3.5/inspect.py", line 1441, in getouterframes frameinfo = (frame,) + getframeinfo(frame, context) File "/usr/lib/python3.5/inspect.py", line 1414, in getframeinfo lines, lnum = findsource(frame) File "/usr/lib/python3.5/inspect.py", line 804, in findsource if pat.match(lines[lnum]): break IndexError: list index out of range Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :| _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
