On Fri, Jun 1, 2018 at 10:25 AM, Tamilarasi Muthamizhan <tmuthami@xxxxxxxxxx> wrote: > Hi Vasu, > > > In the qa suites, We have been relying on Ceph task for long time and > quite frankly it has done excellent job of setting up cluster for > various upper layer tests which don't care how the cluster > is setup. > > I am now proposing where possible we start taking out Ceph Task and > replace this with Ceph-Ansible Task and fix the roles to properly work > with ceph-ansible task, this will help > us test more like customer scenario's, make it easy to consume new > features(eg: ceph-volume, vdo etc) there are few things to fix in > role's for each cluster that is setup using ceph-ansible > > --- Great Idea! > > This is the very reason we have "ceph-ansible" suite in teuthology > [http://pulpito.ceph.com/teuthology-2018-05-23_04:15:02-ceph-ansible-jewel-distro-basic-ovh/] > that uses "ceph-ansible" task instead of "ceph". The purpose of the > "ceph-ansible" suite is to solely ensure the replacement of "ceph" task with > "ceph-ansible" DOES NOT AFFECT anything else. > > I'd recommend continue to using "ceph-ansible" suite to TEST all workunits > [rados/rbd/rgw/fs] by adding more workunits in there and and make sure > everything works before we go ahead and just replace "ceph" task with > "ceph-ansible" in all the QA suites. We have been using this for quite some time and I think more others use it we will have better shot at cleaning up and fixing any issues that pop up rather than one small suite. > > Last time we did this, I remember we hit issues with keyring files that > prevented RGW and FS workunits from passing. > [http://tracker.ceph.com/issues/18528] and a whole bunch of ceph-ansible > installation issues. It is just a keyring issue and for some compatibility the task already creates admin keyring, for other clients it could be just renaming the same file to match client names like client.0 etc. > > > > Regards > Tamil > > On Fri, Jun 1, 2018 at 9:44 AM, Vasu Kulkarni <vakulkar@xxxxxxxxxx> wrote: >> >> Hi Folks, >> >> In the qa suites, We have been relying on Ceph task for long time and >> quite frankly it has done excellent job of setting up cluster for >> various upper layer tests which don't care how the cluster >> is setup. >> >> I am now proposing where possible we start taking out Ceph Task and >> replace this with Ceph-Ansible Task and fix the roles to properly work >> with ceph-ansible task, this will help >> us test more like customer scenario's, make it easy to consume new >> features(eg: ceph-volume, vdo etc) there are few things to fix in >> role's for each cluster that is setup using ceph-ansible >> >> a) multiple mon's or multiple mds cannot exist in same node and should >> be put on different node), also look into how active/standy mds are setup >> using ceph-ansible and use them appropriately in config. >> >> b) Daemon helper was another thing that was holding use of >> Ceph-Ansible task, but I have tested a fix in my personal branch and >> it seems to be working as long as we fix the roles >> >> c) for Dual Cluster setup we can continue to use old style Ceph Task, >> we recommend not to use different cluster names in ceph-deploy or >> ceph-ansible due to systemd issues. >> >> d) upgrades can also use the ceph-ansible task with cluster role fix. >> ( the inital cluster is setup using ceph-ansible but then later on we >> can still rely on old style apt-get update/yum update to >> update individual nodes, behind the scenes systemd will handle the >> daemons) >> >> I will update the Smoke suite to use ceph-ansible by default for all >> tests and see how the trial runs work, Ideally we could move 50 to 60% >> of the tests that can use ceph-ansible and rest can still rely on old >> style so that we have good mix of overall deployment tests. >> >> Feedback welcome >> >> Thanks >> Vasu >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
