Postmortem for RPM build failures

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

Some of you may have noticed that we've been particularly plagued with RPM
build failures for the past couple of days. The failure looks like a Java error
from Jenkins[1]. I first noticed the error on Tuesday, 31st January. I narrowed
down the issue with a problem with permissions. Jenkins cleans the workspace at
the start of every run and Jenkins was unable to delete the RPM folder for some
reason. My best guess is that something changed in the rpmbuild or mock package
in the last couple of days, which lead to this. We have not deployed any change
from the infra side which should have caused this.

I immediately ran an Ansible job that deleted all the RPM workspaces so that
fresh ones would work and I added this line at the end of every Jenkins job:

    sudo chown -R jenkins:jenkins ${WORKSPACE}/RPMS

This seemed to work for the moment and I'd moved onto other things. Yesterday,
we saw quite a resurgence of the errors. Atin filed a bug this morning and I've
figured out what I did wrong. The permission change runs only when the job is
successful. Not exactly my best idea :)

I've now fixed it up by making it a post-job script[2] that runs the same command
irrespective of whether the job failed or worked. That should fix up any
problems in the future. Apologies for the inconvenience. As always, please file
a bug when you notice any unexpected Jenkins behavior.

[1]: https://build.gluster.org/job/strfmt_errors/2789/console
[2]: http://git.gluster.org/cgit/build-jobs.git/commit/?id=3b77dbdac288bf21f802be41019e2cd4d4dc3e3c

--
nigelb

Attachment: signature.asc
Description: PGP signature

_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux