If we are talking about generic install method then compromise has to be made, our cluster configs as is works only with ceph task, it has to be made generic enough so that any higher level tasks using it will work and not fail, this needs to be better described in a document and what various upper layer install would be looking for. I am +1 too for generic install method but it is definitely more work, as I see it is mostly defining things properly in config(cluster + task config) and calling the appropriate functions that handle it. better work in document would result in easier migration. On Wed, Jun 6, 2018 at 3:55 AM, Jan Fajerski <jfajerski@xxxxxxxx> wrote: > On Mon, Jun 04, 2018 at 01:05:21PM -0700, Gregory Farnum wrote: >> >> This is definitely something I'd like to see us do long-term (although >> it would be good to rely, not on ceph-ansible directly, but on some >> abstracted "ceph installer" so that it works with DeepSea, Rook, etc). >> But I'm not sure this list covers a lot of the issues, although maybe >> that's on purpose since you talk about getting up to 50%. > > +1 for this. Fwiw we at suse have already hacked on teuthology to use > salt/DeepSea to run a few things. Something generic to hook into would be > great! > >> >> Specific issues I remember/can think of >> * installing Ceph and other packages. ceph-ansible does install for >> Ceph as well, right? But many tests rely on non-Ceph pieces getting >> installed, including many/most of the FS and rbd suites. >> * daemon helper — as you say, some stuff can be fixed around it, but >> we've got a lot of raw manipulation going on in eg the thrasher tasks. >> We can work out fixes for those if we have a better abstraction layer >> to work with, though! >> >> Okay, that's actually all I've got off the top of my head. Just, don't >> assume that if the smoke suite passes we can switch most of the other >> suites as well. :) >> -Greg >> >> On Fri, Jun 1, 2018 at 9:44 AM, Vasu Kulkarni <vakulkar@xxxxxxxxxx> wrote: >>> >>> Hi Folks, >>> >>> In the qa suites, We have been relying on Ceph task for long time and >>> quite frankly it has done excellent job of setting up cluster for >>> various upper layer tests which don't care how the cluster >>> is setup. >>> >>> I am now proposing where possible we start taking out Ceph Task and >>> replace this with Ceph-Ansible Task and fix the roles to properly work >>> with ceph-ansible task, this will help >>> us test more like customer scenario's, make it easy to consume new >>> features(eg: ceph-volume, vdo etc) there are few things to fix in >>> role's for each cluster that is setup using ceph-ansible >>> >>> a) multiple mon's or multiple mds cannot exist in same node and should >>> be put on different node), also look into how active/standy mds are setup >>> using ceph-ansible and use them appropriately in config. >>> >>> b) Daemon helper was another thing that was holding use of >>> Ceph-Ansible task, but I have tested a fix in my personal branch and >>> it seems to be working as long as we fix the roles >>> >>> c) for Dual Cluster setup we can continue to use old style Ceph Task, >>> we recommend not to use different cluster names in ceph-deploy or >>> ceph-ansible due to systemd issues. >>> >>> d) upgrades can also use the ceph-ansible task with cluster role fix. >>> ( the inital cluster is setup using ceph-ansible but then later on we >>> can still rely on old style apt-get update/yum update to >>> update individual nodes, behind the scenes systemd will handle the >>> daemons) >>> >>> I will update the Smoke suite to use ceph-ansible by default for all >>> tests and see how the trial runs work, Ideally we could move 50 to 60% >>> of the tests that can use ceph-ansible and rest can still rely on old >>> style so that we have good mix of overall deployment tests. >>> >>> Feedback welcome >>> >>> Thanks >>> Vasu >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > -- > Jan Fajerski > Engineer Enterprise Storage > SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, > HRB 21284 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
