Re: Continous failing osd-scrub-repair.sh

[Willem Jan Withagen <wjw@xxxxxxxxxxx>]

On 01/06/2017 10:16, David Zafman wrote:
> Willem,
>
>     Yes, I was wrong this wasn't caused by 3ca750d4.   That change 
> change only affected the test that did EC on bluestore with EC 
> overwrites.
>
> If you have 44736aac74493b5532c9e3574a37a3642766f212 revert it and see 
> if that fixes your problem.
Thanx,
I'll put this on my work queue...

--WjW

> David
>
>
> On 6/1/17 1:11 AM, Willem Jan Withagen wrote:
> > On 01/06/2017 03:11, David Zafman wrote:
> > > The change 3ca750d4 from Josh broke this.  I didn't see before that 
> > > the test itself was broken.
> > Well then again, this is on FreeBSD, where your millage can (and 
> > will) vary.
> > For one, we do not do BlueStore atm.
> >
> > So what is my best option:
> >   - See if it is possible if we can fix this?
> >   - Disable the test on basis of FreeBSD
> >   - Get a different diff-file to compare with?
> >
> > --WjW
> >
> > > David
> > >
> > >
> > > On 5/31/17 4:14 PM, Willem Jan Withagen wrote:
> > > > On 31/05/2017 23:38, David Zafman wrote:
> > > > > All the failures relate to not have the omap digest available in 
> > > > > the object info.  That use to be present before so
> > > > >
> > > > > that the error omap_digest_mismatch_oi would be detected during 
> > > > > this test.
> > > > So, do you have any idea how this is caused?
> > > > What has happened to not have omap digests any longer?
> > > >
> > > > --WjW
> > > >
> > > > > David
> > > > >
> > > > >
> > > > > On 5/31/17 1:58 PM, Willem Jan Withagen wrote:
> > > > > > Since a few days I have a continuous failing test: 
> > > > > > osd-scrub-repair.sh
> > > > > >
> > > > > > There is a diff failing, where the data looks almost correct. :(
> > > > > > Looks there is something wrong with omap_digest.
> > > > > >
> > > > > > Could this be due to bluestore being enable, whilst FreeBSD does not
> > > > > > have that?
> > > > > >
> > > > > > Anyways, all suggestions are welcome
> > > > > >
> > > > > > --WjW
> > > > > >
> > > > > > /home/jenkins/workspace/ceph-master/src/test/osd/osd-scrub-repair.sh:1569: 
> > > > > >
> > > > > > TEST_corrupt_scrub_replicated:  diff td/osd-scrub-repair/checkcsjson
> > > > > > td/osd-scrub-repair/csjson
> > > > > > 322c322
> > > > > > <     "selected_object_info":
> > > > > > "2:f4981d31:::ROBJ4:head(dirty|omap|data_digest|omap_digest s 7 
> > > > > > uv 12 dd
> > > > > > 2ddbf8f5 od d7178dfe alloc_hint [0 0 0])",
> > > > > > ---
> > > > > > >      "selected_object_info":
> > > > > > "2:f4981d31:::ROBJ4:head(dirty|omap|data_digest s 7 uv 12 dd 
> > > > > > 2ddbf8f5
> > > > > > alloc_hint [0 0 0])",
> > > > > > 333,335c333
> > > > > > <         "errors": [
> > > > > > <           "omap_digest_mismatch_oi"
> > > > > > <         ],
> > > > > > ---
> > > > > > >          "errors": [],
> > > > > > 341,343c339
> > > > > > <     "union_shard_errors": [
> > > > > > <       "omap_digest_mismatch_oi"
> > > > > > <     ]
> > > > > > ---
> > > > > > >      "union_shard_errors": []
> > > > > > 356c352
> > > > > > <     "selected_object_info":
> > > > > > "2:f4bfd4d1:::ROBJ5:head(dirty|omap|data_digest|omap_digest s 7 
> > > > > > uv 15 dd
> > > > > > 2ddbf8f5 od 1a862a41 alloc_hint [0 0 0])",
> > > > > > ---
> > > > > > >      "selected_object_info":
> > > > > > "2:f4bfd4d1:::ROBJ5:head(dirty|omap|data_digest s 7 uv 15 dd 
> > > > > > 2ddbf8f5
> > > > > > alloc_hint [0 0 0])",
> > > > > > 367,369c363
> > > > > > <         "errors": [
> > > > > > <           "omap_digest_mismatch_oi"
> > > > > > <         ],
> > > > > > ---
> > > > > > >          "errors": [],
> > > > > > 375,377c369
> > > > > > <     "union_shard_errors": [
> > > > > > <       "omap_digest_mismatch_oi"
> > > > > > <     ]
> > > > > > ---
> > > > > > >      "union_shard_errors": []
> > > > > > 390c382
> > > > > > <     "selected_object_info":
> > > > > > "2:a53c12e8:::ROBJ6:head(dirty|omap|data_digest|omap_digest s 7 
> > > > > > uv 18 dd
> > > > > > 2ddbf8f5 od 689ee887 alloc_hint [0 0 0])",
> > > > > > ---
> > > > > > >      "selected_object_info":
> > > > > > "2:a53c12e8:::ROBJ6:head(dirty|omap|data_digest s 7 uv 18 dd 
> > > > > > 2ddbf8f5
> > > > > > alloc_hint [0 0 0])",
> > > > > > 401,403c393
> > > > > > <         "errors": [
> > > > > > <           "omap_digest_mismatch_oi"
> > > > > > <         ],
> > > > > > ---
> > > > > > >          "errors": [],
> > > > > > 409,411c399
> > > > > > <     "union_shard_errors": [
> > > > > > <       "omap_digest_mismatch_oi"
> > > > > > <     ]
> > > > > > ---
> > > > > > >      "union_shard_errors": []
> > > > > > 424c412
> > > > > > <     "selected_object_info":
> > > > > > "2:8b55fa4b:::ROBJ7:head(dirty|omap|data_digest|omap_digest s 7 
> > > > > > uv 21 dd
> > > > > > 2ddbf8f5 od efced57a alloc_hint [0 0 0])",
> > > > > > ---
> > > > > > >      "selected_object_info":
> > > > > > "2:8b55fa4b:::ROBJ7:head(dirty|omap|data_digest s 7 uv 21 dd 
> > > > > > 2ddbf8f5
> > > > > > alloc_hint [0 0 0])",
> > > > > > 435,437c423
> > > > > > <         "errors": [
> > > > > > <           "omap_digest_mismatch_oi"
> > > > > > <         ],
> > > > > > ---
> > > > > > >          "errors": [],
> > > > > > 443,445c429
> > > > > > <     "union_shard_errors": [
> > > > > > <       "omap_digest_mismatch_oi"
> > > > > > <     ]
> > > > > > ---
> > > > > > >      "union_shard_errors": []
> > > > > > 459c443
> > > > > > <     "selected_object_info":
> > > > > > "2:86586531:::ROBJ8:head(dirty|omap|data_digest|omap_digest s 7 
> > > > > > uv 26 dd
> > > > > > 2ddbf8f5 od d6be81dc alloc_hint [0 0 0])",
> > > > > > ---
> > > > > > >      "selected_object_info":
> > > > > > "2:86586531:::ROBJ8:head(dirty|omap|data_digest s 7 uv 26 dd 
> > > > > > 2ddbf8f5
> > > > > > alloc_hint [0 0 0])",
> > > > > > 532c516
> > > > > > <       "version": 56
> > > > > > ---
> > > > > > >        "version": 50
> > > > > > 534c518
> > > > > > <     "selected_object_info":
> > > > > > "2:ffdb2004:::ROBJ9:head(dirty|omap|data_digest|omap_digest s 3 
> > > > > > uv 56 dd
> > > > > > 1f26fb26 od 2eecc539 alloc_hint [0 0 0])",
> > > > > > ---
> > > > > > >      "selected_object_info":
> > > > > > "2:ffdb2004:::ROBJ9:head(dirty|omap|data_digest s 3 uv 50 dd 
> > > > > > 1f26fb26
> > > > > > alloc_hint [0 0 0])",
> > > > > > 571c555
> > > > > > <         "object_info":
> > > > > > "2:ffdb2004:::ROBJ9:head(dirty|omap|data_digest|omap_digest s 3 
> > > > > > uv 56 dd
> > > > > > 1f26fb26 od 2eecc539 alloc_hint [0 0 0])",
> > > > > > ---
> > > > > > >          "object_info": 
> > > > > > > "2:ffdb2004:::ROBJ9:head(dirty|omap|data_digest
> > > > > > s 3 uv 50 dd 1f26fb26 alloc_hint [0 0 0])",
> > > > > > /home/jenkins/workspace/ceph-master/src/test/osd/osd-scrub-repair.sh:1569: 
> > > > > >
> > > > > > TEST_corrupt_scrub_replicated:  test no = yes
> > > > > > /home/jenkins/workspace/ceph-master/src/test/osd/osd-scrub-repair.sh:1569: 
> > > > > >
> > > > > > --
> > > > > > To unsubscribe from this list: send the line "unsubscribe 
> > > > > > ceph-devel" in
> > > > > > the body of a message to majordomo@xxxxxxxxxxxxxxx
> > > > > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > > >
> > > > --
> > > > To unsubscribe from this list: send the line "unsubscribe 
> > > > ceph-devel" in
> > > > the body of a message to majordomo@xxxxxxxxxxxxxxx
> > > > More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html