Re: inconsistent PG -> unfound objects on an erasure coded system

Jeffrey McDonald <jmcdonal@xxxxxxx> · Tue, 8 Mar 2016 15:04:25 -0600

The filesystem check came back clean:xfs_repair -o ag_stride=8 -t 5 -n /dev/disk/by-id/md-uuid-07f46e30:e2734cbd:ae2f95b3:71ec3067

The other tree sets:  

./70.3bs3_head/DIR_B/DIR_3/DIR_0/DIR_D/DIR_F/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long
./70.3bs3_head/DIR_B/DIR_3/DIR_0/DIR_D/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long

# find . -name "*873db704e96860fddf29*" -exec ls -ltr {} +
-rw-r--r-- 1 root root       0 Jan 23 22:01 ./DIR_B/DIR_3/DIR_0/DIR_D/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long
-rw-r--r-- 1 root root 1048576 Jan 23 22:01 ./DIR_B/DIR_3/DIR_0/DIR_D/DIR_F/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long

# find . -name "*873db704e96860fddf29*" -exec xattr -l  {} +
./DIR_B/DIR_3/DIR_0/DIR_D/DIR_F/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.cephos.lfn3: default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1.5\u220__head_0B4FD03B__46_ffffffffffffffff_3
./DIR_B/DIR_3/DIR_0/DIR_D/DIR_F/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.cephos.spill_out:
0000   30 00                                              0.

./DIR_B/DIR_3/DIR_0/DIR_D/DIR_F/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.ceph._:
0000   0F 08 C0 01 00 00 04 03 FC 00 00 00 00 00 00 00    ................
0010   DB 00 00 00 64 65 66 61 75 6C 74 2E 37 32 34 37    ....default.7247
0020   33 33 2E 31 37 5F 5F 73 68 61 64 6F 77 5F 70 72    33.17__shadow_pr
0030   6F 73 74 61 74 65 2F 72 6E 61 73 65 71 2F 31 37    ostate/rnaseq/17
0040   62 38 30 34 39 64 2D 64 66 33 62 2D 34 38 39 31    b8049d-df3b-4891
0050   2D 38 37 35 63 2D 62 32 61 30 37 37 66 32 61 66    -875c-b2a077f2af
0060   37 61 2F 55 4E 43 49 44 5F 32 32 35 36 35 34 30    7a/UNCID_2256540
0070   2E 62 33 38 33 35 30 64 62 2D 62 31 30 32 2D 34    .b38350db-b102-4
0080   61 33 38 2D 39 65 64 62 2D 30 30 38 39 63 61 39    a38-9edb-0089ca9
0090   35 38 34 30 62 2E 31 33 30 37 32 33 5F 55 4E 43    5840b.130723_UNC
00A0   39 2D 53 4E 32 39 36 5F 30 33 38 36 5F 42 43 32    9-SN296_0386_BC2
00B0   45 34 57 41 43 58 58 5F 31 5F 47 41 54 43 41 47    E4WACXX_1_GATCAG
00C0   2E 74 61 72 2E 67 7A 2E 32 7E 78 50 35 55 6E 37    .tar.gz.2~xP5Un7
00D0   72 68 5F 50 66 6E 74 51 54 67 30 46 49 44 71 78    rh_PfntQTg0FIDqx
00E0   53 49 4C 56 36 31 6E 6B 31 2E 35 5F 32 32 30 FE    SILV61nk1.5_220.
00F0   FF FF FF FF FF FF FF 3B D0 4F 0B 00 00 00 00 00    .......;.O......
0100   46 00 00 00 00 00 00 00 06 03 1C 00 00 00 46 00    F.............F.
0110   00 00 00 00 00 00 FF FF FF FF 00 00 00 00 00 00    ................
0120   00 00 FF FF FF FF FF FF FF FF 00 00 00 00 D0 0E    ................
0130   03 00 00 00 00 00 EC FD 01 00 CF 0E 03 00 00 00    ................
0140   00 00 EC FD 01 00 02 02 15 00 00 00 08 04 DF 0A    ................
0150   00 00 00 00 00 EE A7 B5 01 00 00 00 00 00 00 00    ................
0160   00 00 00 40 00 00 00 00 00 84 4C A4 56 83 F8 5E    ...@......L.V..^
0170   2B 02 02 15 00 00 00 00 00 00 00 00 00 00 00 00    +...............
0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
01A0   00 AA 47 2E 00 00 00 00 00 00 00 00 00 00 00 00    ..G.............
01B0   00 00 24 00 00 00 84 4C A4 56 A5 A7 5A 2B FF FF    ..$....L.V..Z+..
01C0   FF FF FF FF FF FF                                  ......

./DIR_B/DIR_3/DIR_0/DIR_D/DIR_F/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.ceph.snapset:
0000   02 02 19 00 00 00 00 00 00 00 00 00 00 00 01 00    ................
0010   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00       ...............

./DIR_B/DIR_3/DIR_0/DIR_D/DIR_F/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.ceph.hinfo_key:
0000   01 01 24 00 00 00 00 00 10 00 00 00 00 00 06 00    ..$.............
0010   00 00 24 E9 5B 97 2E 88 76 2E 42 AD CE 91 CC 05    ..$.[...v.B.....
0020   89 66 69 BA BC A5 3C F6 E8 80                      .fi...<...

./DIR_B/DIR_3/DIR_0/DIR_D/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.cephos.lfn3: default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1.5\u220__head_0B4FD03B__46_30ec9_3
./DIR_B/DIR_3/DIR_0/DIR_D/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.cephos.spill_out:
0000   30 00                                              0.

./DIR_B/DIR_3/DIR_0/DIR_D/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.ceph._:
0000   0F 08 C0 01 00 00 04 03 FC 00 00 00 00 00 00 00    ................
0010   DB 00 00 00 64 65 66 61 75 6C 74 2E 37 32 34 37    ....default.7247
0020   33 33 2E 31 37 5F 5F 73 68 61 64 6F 77 5F 70 72    33.17__shadow_pr
0030   6F 73 74 61 74 65 2F 72 6E 61 73 65 71 2F 31 37    ostate/rnaseq/17
0040   62 38 30 34 39 64 2D 64 66 33 62 2D 34 38 39 31    b8049d-df3b-4891
0050   2D 38 37 35 63 2D 62 32 61 30 37 37 66 32 61 66    -875c-b2a077f2af
0060   37 61 2F 55 4E 43 49 44 5F 32 32 35 36 35 34 30    7a/UNCID_2256540
0070   2E 62 33 38 33 35 30 64 62 2D 62 31 30 32 2D 34    .b38350db-b102-4
0080   61 33 38 2D 39 65 64 62 2D 30 30 38 39 63 61 39    a38-9edb-0089ca9
0090   35 38 34 30 62 2E 31 33 30 37 32 33 5F 55 4E 43    5840b.130723_UNC
00A0   39 2D 53 4E 32 39 36 5F 30 33 38 36 5F 42 43 32    9-SN296_0386_BC2
00B0   45 34 57 41 43 58 58 5F 31 5F 47 41 54 43 41 47    E4WACXX_1_GATCAG
00C0   2E 74 61 72 2E 67 7A 2E 32 7E 78 50 35 55 6E 37    .tar.gz.2~xP5Un7
00D0   72 68 5F 50 66 6E 74 51 54 67 30 46 49 44 71 78    rh_PfntQTg0FIDqx
00E0   53 49 4C 56 36 31 6E 6B 31 2E 35 5F 32 32 30 FE    SILV61nk1.5_220.
00F0   FF FF FF FF FF FF FF 3B D0 4F 0B 00 00 00 00 00    .......;.O......
0100   46 00 00 00 00 00 00 00 06 03 1C 00 00 00 46 00    F.............F.
0110   00 00 00 00 00 00 FF FF FF FF 00 00 00 00 00 00    ................
0120   00 00 FF FF FF FF FF FF FF FF 00 00 00 00 C8 0E    ................
0130   03 00 00 00 00 00 EC FD 01 00 00 00 00 00 00 00    ................
0140   00 00 00 00 00 00 02 02 15 00 00 00 08 04 DF 0A    ................
0150   00 00 00 00 00 CA A7 B5 01 00 00 00 00 00 00 00    ................
0160   00 00 00 00 00 00 00 00 00 84 4C A4 56 55 BA 13    ..........L.VU..
0170   22 02 02 15 00 00 00 00 00 00 00 00 00 00 00 00    "...............
0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
01A0   00 A2 47 2E 00 00 00 00 00 00 00 00 00 00 00 00    ..G.............
01B0   00 00 34 00 00 00 84 4C A4 56 40 0B 2A 22 FF FF    ..4....L.V@.*"..
01C0   FF FF FF FF FF FF                                  ......

./DIR_B/DIR_3/DIR_0/DIR_D/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.ceph.snapset:
0000   02 02 19 00 00 00 00 00 00 00 00 00 00 00 01 00    ................
0010   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00       ...............

./DIR_B/DIR_3/DIR_0/DIR_D/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.ceph.hinfo_key:
0000   01 01 24 00 00 00 00 00 00 00 00 00 00 00 06 00    ..$.............
0010   00 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF    ................
0020   FF FF FF FF FF FF FF FF FF FF                      ..........

./DIR_B/DIR_3/DIR_0/DIR_D/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long: user.cephos.seq:
0000   01 01 10 00 00 00 2D 43 65 00 00 00 00 00 00 00    ......-Ce.......
0010   00 00 03 00 00 00 01                               .......

./70.ebs1_head/DIR_B/DIR_E/DIR_0/DIR_6/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long
./70.ebs1_head/DIR_B/DIR_E/DIR_0/DIR_6/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long

./70.3bs3_head/DIR_B/DIR_3/DIR_0/DIR_9/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long
./70.3bs3_head/DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long

# find . -name "*f23131e55994b3cd542a*" -exec ls -ltr  {} +
-rw-r--r-- 1 root root       0 Jan 23 22:00 ./DIR_B/DIR_E/DIR_0/DIR_6/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long
-rw-r--r-- 1 root root 1048576 Jan 23 22:00 ./DIR_B/DIR_E/DIR_0/DIR_6/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long

# find . -name "*f23131e55994b3cd542a*" -exec xattr -l  {} +
./DIR_B/DIR_E/DIR_0/DIR_6/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.cephos.lfn3: default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1.5\u24__head_1D7C60EB__46_ffffffffffffffff_1
./DIR_B/DIR_E/DIR_0/DIR_6/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.cephos.spill_out:
0000   30 00                                              0.

./DIR_B/DIR_E/DIR_0/DIR_6/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.ceph._:
0000   0F 08 BF 01 00 00 04 03 FB 00 00 00 00 00 00 00    ................
0010   DA 00 00 00 64 65 66 61 75 6C 74 2E 37 32 34 37    ....default.7247
0020   33 33 2E 31 37 5F 5F 73 68 61 64 6F 77 5F 70 72    33.17__shadow_pr
0030   6F 73 74 61 74 65 2F 72 6E 61 73 65 71 2F 31 37    ostate/rnaseq/17
0040   62 38 30 34 39 64 2D 64 66 33 62 2D 34 38 39 31    b8049d-df3b-4891
0050   2D 38 37 35 63 2D 62 32 61 30 37 37 66 32 61 66    -875c-b2a077f2af
0060   37 61 2F 55 4E 43 49 44 5F 32 32 35 36 35 34 30    7a/UNCID_2256540
0070   2E 62 33 38 33 35 30 64 62 2D 62 31 30 32 2D 34    .b38350db-b102-4
0080   61 33 38 2D 39 65 64 62 2D 30 30 38 39 63 61 39    a38-9edb-0089ca9
0090   35 38 34 30 62 2E 31 33 30 37 32 33 5F 55 4E 43    5840b.130723_UNC
00A0   39 2D 53 4E 32 39 36 5F 30 33 38 36 5F 42 43 32    9-SN296_0386_BC2
00B0   45 34 57 41 43 58 58 5F 31 5F 47 41 54 43 41 47    E4WACXX_1_GATCAG
00C0   2E 74 61 72 2E 67 7A 2E 32 7E 78 50 35 55 6E 37    .tar.gz.2~xP5Un7
00D0   72 68 5F 50 66 6E 74 51 54 67 30 46 49 44 71 78    rh_PfntQTg0FIDqx
00E0   53 49 4C 56 36 31 6E 6B 31 2E 35 5F 32 34 FE FF    SILV61nk1.5_24..
00F0   FF FF FF FF FF FF EB 60 7C 1D 00 00 00 00 00 46    .......`|......F
0100   00 00 00 00 00 00 00 06 03 1C 00 00 00 46 00 00    .............F..
0110   00 00 00 00 00 FF FF FF FF 00 00 00 00 00 00 00    ................
0120   00 FF FF FF FF FF FF FF FF 00 00 00 00 56 10 04    .............V..
0130   00 00 00 00 00 EC FD 01 00 55 10 04 00 00 00 00    .........U......
0140   00 EC FD 01 00 02 02 15 00 00 00 08 04 DF 0A 00    ................
0150   00 00 00 00 EF 83 B5 01 00 00 00 00 00 00 00 00    ................
0160   00 00 40 00 00 00 00 00 50 4C A4 56 82 68 A6 0A    ..@.....PL.V.h..
0170   02 02 15 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
01A0   BA 46 2E 00 00 00 00 00 00 00 00 00 00 00 00 00    .F..............
01B0   00 24 00 00 00 50 4C A4 56 3A 5B B7 0A FF FF FF    .$...PL.V:[.....
01C0   FF FF FF FF FF                                     .....

./DIR_B/DIR_E/DIR_0/DIR_6/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.ceph.snapset:
0000   02 02 19 00 00 00 00 00 00 00 00 00 00 00 01 00    ................
0010   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00       ...............

./DIR_B/DIR_E/DIR_0/DIR_6/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.ceph.hinfo_key:
0000   01 01 24 00 00 00 00 00 10 00 00 00 00 00 06 00    ..$.............
0010   00 00 EF BD AC B3 7C 77 68 4C 0C 79 F6 2A DF 51    ......|whL.y.*.Q
0020   F3 5A AD 91 17 64 E2 9B 23 97                      .Z...d..#.

./DIR_B/DIR_E/DIR_0/DIR_6/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.cephos.lfn3: default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1.5\u24__head_1D7C60EB__46_4104f_1
./DIR_B/DIR_E/DIR_0/DIR_6/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.cephos.spill_out:
0000   30 00                                              0.

./DIR_B/DIR_E/DIR_0/DIR_6/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.ceph._:
0000   0F 08 BF 01 00 00 04 03 FB 00 00 00 00 00 00 00    ................
0010   DA 00 00 00 64 65 66 61 75 6C 74 2E 37 32 34 37    ....default.7247
0020   33 33 2E 31 37 5F 5F 73 68 61 64 6F 77 5F 70 72    33.17__shadow_pr
0030   6F 73 74 61 74 65 2F 72 6E 61 73 65 71 2F 31 37    ostate/rnaseq/17
0040   62 38 30 34 39 64 2D 64 66 33 62 2D 34 38 39 31    b8049d-df3b-4891
0050   2D 38 37 35 63 2D 62 32 61 30 37 37 66 32 61 66    -875c-b2a077f2af
0060   37 61 2F 55 4E 43 49 44 5F 32 32 35 36 35 34 30    7a/UNCID_2256540
0070   2E 62 33 38 33 35 30 64 62 2D 62 31 30 32 2D 34    .b38350db-b102-4
0080   61 33 38 2D 39 65 64 62 2D 30 30 38 39 63 61 39    a38-9edb-0089ca9
0090   35 38 34 30 62 2E 31 33 30 37 32 33 5F 55 4E 43    5840b.130723_UNC
00A0   39 2D 53 4E 32 39 36 5F 30 33 38 36 5F 42 43 32    9-SN296_0386_BC2
00B0   45 34 57 41 43 58 58 5F 31 5F 47 41 54 43 41 47    E4WACXX_1_GATCAG
00C0   2E 74 61 72 2E 67 7A 2E 32 7E 78 50 35 55 6E 37    .tar.gz.2~xP5Un7
00D0   72 68 5F 50 66 6E 74 51 54 67 30 46 49 44 71 78    rh_PfntQTg0FIDqx
00E0   53 49 4C 56 36 31 6E 6B 31 2E 35 5F 32 34 FE FF    SILV61nk1.5_24..
00F0   FF FF FF FF FF FF EB 60 7C 1D 00 00 00 00 00 46    .......`|......F
0100   00 00 00 00 00 00 00 06 03 1C 00 00 00 46 00 00    .............F..
0110   00 00 00 00 00 FF FF FF FF 00 00 00 00 00 00 00    ................
0120   00 FF FF FF FF FF FF FF FF 00 00 00 00 4E 10 04    .............N..
0130   00 00 00 00 00 EC FD 01 00 00 00 00 00 00 00 00    ................
0140   00 00 00 00 00 02 02 15 00 00 00 08 04 DF 0A 00    ................
0150   00 00 00 00 4A 83 B5 01 00 00 00 00 00 00 00 00    ....J...........
0160   00 00 00 00 00 00 00 00 4F 4C A4 56 A8 1E A2 21    ........OL.V...!
0170   02 02 15 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
01A0   B2 46 2E 00 00 00 00 00 00 00 00 00 00 00 00 00    .F..............
01B0   00 34 00 00 00 4F 4C A4 56 39 2D 4A 23 FF FF FF    .4...OL.V9-J#...
01C0   FF FF FF FF FF                                     .....

./DIR_B/DIR_E/DIR_0/DIR_6/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.ceph.snapset:
0000   02 02 19 00 00 00 00 00 00 00 00 00 00 00 01 00    ................
0010   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00       ...............

./DIR_B/DIR_E/DIR_0/DIR_6/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.ceph.hinfo_key:
0000   01 01 24 00 00 00 00 00 00 00 00 00 00 00 06 00    ..$.............
0010   00 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF    ................
0020   FF FF FF FF FF FF FF FF FF FF                      ..........

./DIR_B/DIR_E/DIR_0/DIR_6/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long: user.cephos.seq:
0000   01 01 10 00 00 00 04 41 65 00 00 00 00 00 00 00    .......Ae.......
0010   00 00 03 00 00 00 01                               .......

# find . -name "*215ce1442b16dc173b77*" -exec ls  -ltr  {} +
-rw-r--r-- 1 root root       0 Jan 23 21:28 ./DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long
-rw-r--r-- 1 root root 1048576 Jan 23 21:28 ./DIR_B/DIR_3/DIR_0/DIR_9/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long
# find . -name "*215ce1442b16dc173b77*" -exec ls  -ltr  {} +
-rw-r--r-- 1 root root       0 Jan 23 21:28 ./DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long
-rw-r--r-- 1 root root 1048576 Jan 23 21:28 ./DIR_B/DIR_3/DIR_0/DIR_9/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long

root@ceph03:/var/lib/ceph/osd/ceph-307/current/70.3bs3_head# find . -name "*215ce1442b16dc173b77*" -exec xattr -l  {} +
./DIR_B/DIR_3/DIR_0/DIR_9/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.cephos.lfn3: default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3.3\u249__head_4B3C903B__46_ffffffffffffffff_3
./DIR_B/DIR_3/DIR_0/DIR_9/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.cephos.spill_out:
0000   30 00                                              0.

./DIR_B/DIR_3/DIR_0/DIR_9/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.ceph._:
0000   0F 08 C0 01 00 00 04 03 FC 00 00 00 00 00 00 00    ................
0010   DB 00 00 00 64 65 66 61 75 6C 74 2E 37 32 34 37    ....default.7247
0020   33 33 2E 31 37 5F 5F 73 68 61 64 6F 77 5F 70 72    33.17__shadow_pr
0030   6F 73 74 61 74 65 2F 72 6E 61 73 65 71 2F 34 31    ostate/rnaseq/41
0040   66 64 65 37 38 36 2D 63 62 66 62 2D 34 63 31 31    fde786-cbfb-4c11
0050   2D 38 36 39 36 2D 66 65 32 30 66 39 30 66 30 36    -8696-fe20f90f06
0060   32 61 2F 55 4E 43 49 44 5F 32 32 35 36 34 38 30    2a/UNCID_2256480
0070   2E 37 39 39 32 32 32 37 36 2D 36 65 30 30 2D 34    .79922276-6e00-4
0080   66 31 37 2D 38 35 31 66 2D 35 35 34 66 35 32 66    f17-851f-554f52f
0090   37 31 35 32 30 2E 31 33 30 37 32 33 5F 55 4E 43    71520.130723_UNC
00A0   39 2D 53 4E 32 39 36 5F 30 33 38 36 5F 42 43 32    9-SN296_0386_BC2
00B0   45 34 57 41 43 58 58 5F 32 5F 41 43 54 54 47 41    E4WACXX_2_ACTTGA
00C0   2E 74 61 72 2E 67 7A 2E 32 7E 54 57 66 42 55 6C    .tar.gz.2~TWfBUl
00D0   45 6B 6B 34 45 50 48 34 75 5F 4E 6B 4D 6A 6D 7A    Ekk4EPH4u_NkMjmz
00E0   36 35 43 52 53 6B 4A 41 33 2E 33 5F 32 34 39 FE    65CRSkJA3.3_249.
00F0   FF FF FF FF FF FF FF 3B 90 3C 4B 00 00 00 00 00    .......;.<K.....
0100   46 00 00 00 00 00 00 00 06 03 1C 00 00 00 46 00    F.............F.
0110   00 00 00 00 00 00 FF FF FF FF 00 00 00 00 00 00    ................
0120   00 00 FF FF FF FF FF FF FF FF 00 00 00 00 C2 0D    ................
0130   03 00 00 00 00 00 EC FD 01 00 C1 0D 03 00 00 00    ................
0140   00 00 EC FD 01 00 02 02 15 00 00 00 08 04 DF 0A    ................
0150   00 00 00 00 00 62 1E B0 01 00 00 00 00 00 00 00    .....b..........
0160   00 00 00 40 00 00 00 00 00 CF 44 A4 56 22 EC CE    ...@......D.V"..
0170   00 02 02 15 00 00 00 00 00 00 00 00 00 00 00 00    ................
0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
01A0   00 9C 46 2E 00 00 00 00 00 00 00 00 00 00 00 00    ..F.............
01B0   00 00 24 00 00 00 CF 44 A4 56 F2 38 2A 01 FF FF    ..$....D.V.8*...
01C0   FF FF FF FF FF FF                                  ......

./DIR_B/DIR_3/DIR_0/DIR_9/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.ceph.snapset:
0000   02 02 19 00 00 00 00 00 00 00 00 00 00 00 01 00    ................
0010   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00       ...............

./DIR_B/DIR_3/DIR_0/DIR_9/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.ceph.hinfo_key:
0000   01 01 24 00 00 00 00 00 10 00 00 00 00 00 06 00    ..$.............
0010   00 00 93 6A A4 39 3D AC B6 2E 71 30 1D 1A 98 00    ...j.9=...q0....
0020   FB 23 AA 85 22 C5 03 B5 B1 0F                      .#..".....

./DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.cephos.lfn3: default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3.3\u249__head_4B3C903B__46_30dbb_3
./DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.cephos.spill_out:
0000   30 00                                              0.

./DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.ceph._:
0000   0F 08 C0 01 00 00 04 03 FC 00 00 00 00 00 00 00    ................
0010   DB 00 00 00 64 65 66 61 75 6C 74 2E 37 32 34 37    ....default.7247
0020   33 33 2E 31 37 5F 5F 73 68 61 64 6F 77 5F 70 72    33.17__shadow_pr
0030   6F 73 74 61 74 65 2F 72 6E 61 73 65 71 2F 34 31    ostate/rnaseq/41
0040   66 64 65 37 38 36 2D 63 62 66 62 2D 34 63 31 31    fde786-cbfb-4c11
0050   2D 38 36 39 36 2D 66 65 32 30 66 39 30 66 30 36    -8696-fe20f90f06
0060   32 61 2F 55 4E 43 49 44 5F 32 32 35 36 34 38 30    2a/UNCID_2256480
0070   2E 37 39 39 32 32 32 37 36 2D 36 65 30 30 2D 34    .79922276-6e00-4
0080   66 31 37 2D 38 35 31 66 2D 35 35 34 66 35 32 66    f17-851f-554f52f
0090   37 31 35 32 30 2E 31 33 30 37 32 33 5F 55 4E 43    71520.130723_UNC
00A0   39 2D 53 4E 32 39 36 5F 30 33 38 36 5F 42 43 32    9-SN296_0386_BC2
00B0   45 34 57 41 43 58 58 5F 32 5F 41 43 54 54 47 41    E4WACXX_2_ACTTGA
00C0   2E 74 61 72 2E 67 7A 2E 32 7E 54 57 66 42 55 6C    .tar.gz.2~TWfBUl
00D0   45 6B 6B 34 45 50 48 34 75 5F 4E 6B 4D 6A 6D 7A    Ekk4EPH4u_NkMjmz
00E0   36 35 43 52 53 6B 4A 41 33 2E 33 5F 32 34 39 FE    65CRSkJA3.3_249.
00F0   FF FF FF FF FF FF FF 3B 90 3C 4B 00 00 00 00 00    .......;.<K.....
0100   46 00 00 00 00 00 00 00 06 03 1C 00 00 00 46 00    F.............F.
0110   00 00 00 00 00 00 FF FF FF FF 00 00 00 00 00 00    ................
0120   00 00 FF FF FF FF FF FF FF FF 00 00 00 00 BA 0D    ................
0130   03 00 00 00 00 00 EC FD 01 00 00 00 00 00 00 00    ................
0140   00 00 00 00 00 00 02 02 15 00 00 00 08 04 DF 0A    ................
0150   00 00 00 00 00 52 1E B0 01 00 00 00 00 00 00 00    .....R..........
0160   00 00 00 00 00 00 00 00 00 CE 44 A4 56 96 5A CA    ..........D.V.Z.
0170   27 02 02 15 00 00 00 00 00 00 00 00 00 00 00 00    '...............
0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
01A0   00 94 46 2E 00 00 00 00 00 00 00 00 00 00 00 00    ..F.............
01B0   00 00 34 00 00 00 CE 44 A4 56 96 F4 45 2D FF FF    ..4....D.V..E-..
01C0   FF FF FF FF FF FF                                  ......

./DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.ceph.snapset:
0000   02 02 19 00 00 00 00 00 00 00 00 00 00 00 01 00    ................
0010   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00       ...............

./DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.ceph.hinfo_key:
0000   01 01 24 00 00 00 00 00 00 00 00 00 00 00 06 00    ..$.............
0010   00 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF    ................
0020   FF FF FF FF FF FF FF FF FF FF                      ..........

./DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long: user.cephos.seq:
0000   01 01 10 00 00 00 15 F5 64 00 00 00 00 00 00 00    ........d.......
0010   00 00 03 00 00 00 01                               .......

I put three new tar archives in this folder with names corresponding to the PG: 
https://drive.google.com/folderview?id=0Bzz8TrxFvfema2NQUmotd1BOTnM&usp=drive_web#list

Jeff

On Tue, Mar 8, 2016 at 2:08 PM, Jeffrey McDonald <jmcdonal@xxxxxxx> wrote:
Will do, I'm in the midst of the xfs filesystem check so it will be a bit before I have the filesystem back.....jeff

On Tue, Mar 8, 2016 at 2:05 PM, Samuel Just <sjust@xxxxxxxxxx> wrote:
There are 3 other example pairs on that osd.  Can you gather the same

information about these as well?

./70.3bs3_head/DIR_B/DIR_3/DIR_0/DIR_D/DIR_F/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long

./70.3bs3_head/DIR_B/DIR_3/DIR_0/DIR_D/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._873db704e96860fddf29_0_long

./70.ebs1_head/DIR_B/DIR_E/DIR_0/DIR_6/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long

./70.ebs1_head/DIR_B/DIR_E/DIR_0/DIR_6/default.724733.17\u\ushadow\uprostate\srnaseq\s17b8049d-df3b-4891-875c-b2a077f2af7a\sUNCID\u2256540.b38350db-b102-4a38-9edb-0089ca95840b.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u1\uGATCAG.tar.gz.2~xP5Un7rh\uPfntQTg0FIDqxSILV61nk1._f23131e55994b3cd542a_0_long

./70.3bs3_head/DIR_B/DIR_3/DIR_0/DIR_9/DIR_C/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long

./70.3bs3_head/DIR_B/DIR_3/DIR_0/DIR_9/default.724733.17\u\ushadow\uprostate\srnaseq\s41fde786-cbfb-4c11-8696-fe20f90f062a\sUNCID\u2256480.79922276-6e00-4f17-851f-554f52f71520.130723\uUNC9-SN296\u0386\uBC2E4WACXX\u2\uACTTGA.tar.gz.2~TWfBUlEkk4EPH4u\uNkMjmz65CRSkJA3._215ce1442b16dc173b77_0_long

On Tue, Mar 8, 2016 at 12:00 PM, Samuel Just <sjust@xxxxxxxxxx> wrote:

> Yeah, that procedure should have isolated any filesystem issues.  Are

> there still unfound objects?

> -sam

>

> On Tue, Mar 8, 2016 at 11:58 AM, Jeffrey McDonald <jmcdonal@xxxxxxx> wrote:

>> Yes, I used the crushmap, set them to 0, then let ceph migrate/remap them

>> over.   I controlled the tempo of the move by allowing fewer backfill

>> threads.

>>

>> Jeff

>>

-- 
Jeffrey McDonald, PhD
Assistant Director for HPC Operations
Minnesota Supercomputing Institute
University of Minnesota Twin Cities
599 Walter Library           email: jeffrey.mcdonald@xxxxxxxxxxx
117 Pleasant St SE           phone: +1 612 625-6905
Minneapolis, MN 55455        fax:   +1 612 624-8861

-- 
Jeffrey McDonald, PhD
Assistant Director for HPC Operations
Minnesota Supercomputing Institute
University of Minnesota Twin Cities
599 Walter Library           email: jeffrey.mcdonald@xxxxxxxxxxx
117 Pleasant St SE           phone: +1 612 625-6905
Minneapolis, MN 55455        fax:   +1 612 624-8861

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com