On Mon, Jun 29, 2020 at 11:38 AM Patrick Donnelly <pdonnell@xxxxxxxxxx> wrote: > > On Mon, Jun 29, 2020 at 10:35 AM Yuri Weinstein <yweinste@xxxxxxxxxx> wrote: > > fs - approved Patrick? > > There's a new upgrade test failure: > https://pulpito.ceph.com/yuriw-2020-06-25_22:31:00-fs-octopus-distro-basic-smithi/5180149 > > 2020-06-26T07:13:13.444 INFO:teuthology.orchestra.run.smithi087:> sudo > adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage > timeout 120 ceph --cluster ceph --log-early osd dump --format=json > ... > 2020-06-26T07:13:13.709 > INFO:teuthology.orchestra.run.smithi087.stderr:Error EINVAL: invalid > command > > I vaguely recall seeing this before but can't find the tracker ticket. > The last successful run was on the 24th (the day before you scheduled > tests!!): > > https://pulpito.ceph.com/teuthology-2020-06-24_04:10:03-fs-octopus-distro-basic-smithi/5174869/ > > The difference in commits > 5c9eee5a7ad1bb8b161f33b4560d5dfc4a000b6d..upstream/heads/octopus does > not shine light on why this would happen. > > Any ideas? Looks like this is caused by an older version of Mimic, v13.2.8. The successful run from June 24th: 2020-06-25T06:10:56.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=mimic 2020-06-25T06:10:56.938 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/mimic/8e9675a830b7e7a7164659a2a8f8ad08b8f61358/ubuntu/bionic/flavors/default/ 2020-06-25T06:10:56.938 INFO:teuthology.task.install.deb:Package version is 13.2.10-15-g8e9675a830-1bionic vs. the failure on June 25th: 2020-06-26T07:11:31.659 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=mimic 2020-06-26T07:11:31.820 INFO:teuthology.task.install.deb:Pulling from https://chacra.ceph.com/r/ceph/mimic/5579a94fafbc1f9cc913a0f5d362953a5d9c3ae0/ubuntu/bionic/flavors/default/ 2020-06-26T07:11:31.821 INFO:teuthology.task.install.deb:Package version is 13.2.8-1bionic This is probably caused by the LRC data loss. Looks like we need to get new builds of Mimic. -- Patrick Donnelly, Ph.D. He / Him / His Senior Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
