On Mon, Aug 29, 2016 at 8:42 AM, Niels de Vos <ndevos@xxxxxxxxxx> wrote: > On Mon, Aug 29, 2016 at 05:55:00PM +0530, Nigel Babu wrote: >> On Mon, Aug 29, 2016 at 01:49:52PM +0200, Niels de Vos wrote: >> > On Mon, Aug 29, 2016 at 05:01:18PM +0530, Nigel Babu wrote: >> > > Hello folks, >> > > >> > > I've had chats with Manoj and Ambarish about performance testing and what we >> > > can do upstream. Niels today solved half my problem by pointing out that we can >> > > get physical nodes on CentOS CI. The general idea is to run iozone[1] and >> > > smallfile[2] on a fixed frequency for master (to begin with). >> > > >> > > Does this sound like a good idea? If so, read on. >> > > >> > > For this to happens a few things need to happen: >> > > * I'll need some help from a few people who can read the reports and coordinate >> > > fixes. That is, someone needs to "own" performance for upstream. >> > > * I need some help in generating the right reports so we can figure out if our >> > > performance went up or down. >> > >> > The provisioning in the CentOS CI does not allow us to select certain >> > systems (yet). So you would get different performance results, depending >> > on the hardware that the reservation request returns: >> > https://wiki.centos.org/QaWiki/PubHardware >> > >> > Also, these physical machines do not have additional disks. The single >> > SSD that these systems have, is completely used by the installation, no >> > free space to partition to our liking, no additional disks available. >> > >> > I welcome any additional testing that we can run regulary, but to call >> > it 'performance testing' might be a little pre-mature. At least the >> > performance results should be marked as 'unoptimized' or similar. >> > >> > HTH, >> > Niels >> > >> >> The goal of this testing, to begin with, wouldn't be to get absolute numbers >> but to try and catch decrease in performance, if that makes sense. In essence, >> it's regression testing but for performance. > > Ah, ok, changing the subject to reflect that. > >> Thank you for raising the fact that it may be inconsistent, I'll talk to the >> Centos CI folks and see what's the best way forward for us before we get here. >> But let's work with the assumption that I'll sort out the infra side of things. > > There has been a request from others already to be able to select the > blade-chassis during reserving machines. This would come a long way to > get comparable results. Otherwise we can compare the results based on > sub-domain (per chassis) where the tests were running (more difficult > when multiple machines/chassis are involved). > +1 to running performance regression tests on identical hardware. I would also recommend running perf-test.sh [1] for regression. Thanks, Vijay [1] https://github.com/avati/perf-test/blob/master/perf-test.sh _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
