All, As you may or may not know Alfredo and I have been working on designing a new system to replace the gitbuilders. I wanted to send out an outline of what we're thinking to gather review and feedback. We're hoping this new system will be able to produce a test build much faster than the current solution and scale easily as the ceph core team grows. Components of the new gitbuilder system. - Jenkins - responsible for building ceph and uploading the binaries to chacra - provides an interface to initiate builds - provides authentication and authorization on who can initiate builds - Chacra - responsible for storage of the binaries and repo creation - can be scaled horizontally by creating new nodes and registering them with Shaman - Shaman - responsible for coordination between the chacra nodes. - responsible for acting as a load balancer for all the chacra nodes. This is so jenkins can know which node to upload binaries to and to allow for the ability for chacra nodes to scale horizontally. - responsible for answering questions about the availability of ceph repos. (i.e. What is the latest sha1 I can test for the master branch on xenial, trusty and centos7? Is sha1 X built for xenial?) - provides metadata about ceph repositories, - Mita - responsible for creating new jenkins slaves. - responsible for tearing down unused jenkins slaves. - Prado - responsible for provisioning of new jenkins slaves Basic workflow of a "gitbuilder like" ceph build, from initiation to repo ready for use: - Jenkins build initiated - By sha1 or branch - upstream release or dev release (gitbuilder build) - provide list of distros - Jenkins performs the build - Jenkins asks Shaman which chacra instance to upload binaries to - Shaman will know all the chacra nodes and will provide them for use in a round robin fashion by use of an api endpoint. - Jenkins uploads binaries to chacra instance given by shaman - When repo creation is queued, chacra will ping shaman to notify it that a repo has been requested. - When repo creation starts, chacra will ping shaman with that status. - If repo creation fails, chacra will notify shaman. - If repo creation succeeds, chacra will notify shaman. - Shaman can be queried for status of the repos by ref, sha1 or distro. - Shaman can answer questions like, "What is the latest sha1 I can use for the master branch on trusty, xenial and centos7?" Best, Andrew -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
