Am Montag, den 21.01.2019, 08:01 -0500 schrieb Brian Foster: [...] > > root@XXX:/mnt/metadump# xfs_repair /dev/XXX > > Phase 1 - find and verify superblock... > > - reporting progress in intervals of 15 minutes > > Phase 2 - using internal log > > - zero log... > > - scan filesystem freespace and inode maps... > > bad magic # 0x49414233 in inobt block 5/7831662 > > Hmm, so this looks like a very isolated corruption. It's complaining > about a magic number (internal filesystem value stamped to metadata > blocks for verification/sanity purposes) being wrong on a finobt block > and nothing else seems to be wrong in the fs. (I guess repair should > probably print out 'finobt' here instead of 'inobt,' but that's a > separate issue..). > > The finobt uses a magic value of XFS_FIBT_CRC_MAGIC (0x46494233, 'FIB3') > whereas this block has a magic value of 0x49414233. The latter is > 'IAB3' (XFS_IBT_CRC_MAGIC), which is the magic value for regular inode > btree blocks. > > > - 23:06:50: scanning filesystem freespace - 33 of 33 allocation groups done > > - found root inode chunk > > Phase 3 - for each AG... > > ... > > Phase 7 - verify and correct link counts... > > - 22:29:19: verify and correct link counts - 33 of 33 allocation groups done > > done > > > > > Would you be able to provide an xfs_metadump image > > > of this filesystem for closer inspection? > > > > This filesystem is really metadata heavy, so an xfs_metadump ended up > > being around 400GB of data. Not sure if this is something you would be > > willing to look into? > > > > Ok, it might be difficult to get ahold of that. Does the image happen to > compress well? I'll see how well it compresses, but this might take a while... > In the meantime, given that the corruption appears to be so isolated you > might be able to provide enough information from the metadump without > having to transfer it. The first thing is probably to take a look at the > block in question.. > > First, restore the metadump somewhere: > > xfs_mdrestore -g ./md.img <destination> > > You'll need somewhere with enough space for that 400G or so. Note that > you can restore to a file and mount/inspect that file as if it were the > original fs. I'd also mount/unmount the restored metadump and run an > 'xfs_repair -n' on it just to double check that the corruption was > captured properly and there are no other issues with the metadump. -n is > important here as otherwise repair will fix the metadump and remove the > corruption. > > Next, use xfs_db to dump the contents of the suspect block. Run 'xfs_db > <metadump image>' to open the fs and try the following sequence of > commands. > > - Convert to a global fsb: 'convert agno 5 agbno 7831662 fsb' > - Jump to the fsb: 'fsb <output of prev cmd>' > - Set the block type: 'type finobt' > - Print the block: 'print' > > ... and copy/paste the output. So for the moment, here's the output of the above sequence. xfs_db> convert agno 5 agbno 7831662 fsb 0x5077806e (1350008942) xfs_db> fsb 0x5077806e xfs_db> type finobt xfs_db> print magic = 0x49414233 level = 1 numrecs = 335 leftsib = 7810856 rightsib = null bno = 7387612016 lsn = 0x6671003d9700 uuid = 026711cc-25c7-44b9-89aa-0aac496edfec owner = 5 crc = 0xe12b19b2 (correct) keys[1-335] = [startino] 1:[958766208] 2:[960203904] 3:[961707648] 4:[963003264] 5:[964566528] 6:[965274432] 7:[966643456] 8:[967151808] 9:[967937920] 10:[968462976] 11:[969040768] 12:[970681856] 13:[971156992] 14:[972031232] 15:[972772608] 16:[973479296] 17:[974016512] 18:[974475776] 19:[974999552] 20:[975452160] 21:[975836160] 22:[976584576] 23:[977213824] 24:[982036096] 25:[987294592] 26:[988030720] 27:[988549888] 28:[989288000] 29:[990621440] 30:[991238016] 31:[991789824] 32:[992320128] 33:[993190848] 34:[994163712] 35:[995337856] 36:[997400448] 37:[997834496] 38:[998261504] 39:[998809472] 40:[1000180608] 41:[1002622976] 42:[1003701376] 43:[1004380288] 44:[1006392384] 45:[1007378048] 46:[1008899264] 47:[1010753920] 48:[1011690944] 49:[1012841856] 50:[1014790016] 51:[1018967104] 52:[1021546752] 53:[1022964608] 54:[1025270016] 55:[1026458240] 56:[1027689088] 57:[1029101632] 58:[1030116032] 59:[1030642176] 60:[1032072704] 61:[1032929664] 62:[1033577728] 63:[1034240256] 64:[1035221504] 65:[1036707584] 66:[1037945472] 67:[1039025408] 68:[1039866752] 69:[1040149888] 70:[1040465664] 71:[1041233024] 72:[1042081024] 73:[1042587776] 74:[1044256448] 75:[1044670336] 76:[1045996928] 77:[1047547904] 78:[1048474880] 79:[1049188224] 80:[1049874432] 81:[1049919104] 82:[1050624128] 83:[1051385088] 84:[1052172288] 85:[1052987968] 86:[1054098688] 87:[1054993728] 88:[1055481984] 89:[1055968320] 90:[1056760704] 91:[1057265408] 92:[1057744896] 93:[1058394752] 94:[1058656000] 95:[1059149824] 96:[1059732928] 97:[1061290624] 98:[1062338624] 99:[1062885248] 100:[1063539328] 101:[1063939200] 102:[1065515904] 103:[1067503232] 104:[1068582144] 105:[1069002880] 106:[1069577664] 107:[1069960128] 108:[1070688000] 109:[1071114368] 110:[1072531328] 111:[1073282944] 112:[1074234496] 113:[1075912320] 114:[1076997888] 115:[1077965440] 116:[1080531584] 117:[1081580672] 118:[1082154112] 119:[1085020288] 120:[1085956480] 121:[1087185664] 122:[1088890368] 123:[1088965760] 124:[1094695424] 125:[1098562560] 126:[1101387776] 127:[1101951488] 128:[1102746752] 129:[1103568128] 130:[1105154048] 131:[1105477632] 132:[1106437248] 133:[1108366720] 134:[1109182336] 135:[1110930880] 136:[1113504384] 137:[1115613056] 138:[1116307456] 139:[1118841984] 140:[1121370752] 141:[1122338816] 142:[1122727488] 143:[1123963712] 144:[1126139520] 145:[1126974208] 146:[1129443648] 147:[1130809408] 148:[1132503552] 149:[1135562240] 150:[1137630720] 151:[1139159872] 152:[1139976256] 153:[1140282880] 154:[1141660160] 155:[1144208320] 156:[1144265344] 157:[1144312192] 158:[1145091072] 159:[1147749760] 160:[1148044352] 161:[1148957952] 162:[1149682112] 163:[1150844800] 164:[1152461568] 165:[1152877824] 166:[1154310464] 167:[1162018240] 168:[1163022208] 169:[1178935936] 170:[1190226752] 171:[1191987584] 172:[1199930368] 173:[1201136576] 174:[1202296320] 175:[1202753792] 176:[1202950656] 177:[1203254016] 178:[1203438464] 179:[1203862016] 180:[1204373888] 181:[1205211648] 182:[1205716992] 183:[1206647424] 184:[1207043840] 185:[1207310976] 186:[1207687680] 187:[1208031744] 188:[1208310912] 189:[1208601280] 190:[1208916608] 191:[1209711104] 192:[1210376128] 193:[1211742528] 194:[1212674304] 195:[1212952896] 196:[1213629312] 197:[1214420480] 198:[1214658560] 199:[1214882368] 200:[1215331264] 201:[1215529856] 202:[1215810560] 203:[1216206592] 204:[1216486912] 205:[1216681344] 206:[1216914816] 207:[1217146880] 208:[1217465984] 209:[1217724544] 210:[1218293440] 211:[1218903424] 212:[1227517312] 213:[1228301056] 214:[1228486528] 215:[1249407040] 216:[1249770752] 217:[1250043968] 218:[1250634048] 219:[1250931392] 220:[1251296128] 221:[1251928448] 222:[1252114816] 223:[1252309888] 224:[1253285504] 225:[1253446080] 226:[1254226432] 227:[1254858624] 228:[1255042112] 229:[1255288064] 230:[1255453120] 231:[1256473984] 232:[1256852480] 233:[1257881088] 234:[1259645184] 235:[1260438912] 236:[1261217280] 237:[1261577536] 238:[1262093696] 239:[1262966688] 240:[1263311744] 241:[1264222016] 242:[1265448192] 243:[1266591360] 244:[1268133376] 245:[1268988928] 246:[1269833024] 247:[1271247296] 248:[1272420224] 249:[1273266752] 250:[1274076224] 251:[1275203072] 252:[1277416320] 253:[1278511232] 254:[1281360640] 255:[1302046720] 256:[1309009024] 257:[1325006208] 258:[1330860928] 259:[1332440704] 260:[1332907520] 261:[1333996288] 262:[1334853248] 263:[1335306240] 264:[1336120640] 265:[1336960192] 266:[1338423040] 267:[1339761216] 268:[1341126208] 269:[1342171904] 270:[1343213568] 271:[1350140672] 272:[1350661184] 273:[1351507264] 274:[1352978240] 275:[1381651968] 276:[1384242688] 277:[1385959936] 278:[1386452096] 279:[1388349312] 280:[1389233664] 281:[1393009344] 282:[1396065408] 283:[1396865152] 284:[1397867520] 285:[1398936704] 286:[1400073472] 287:[1402718912] 288:[1406069440] 289:[1416367744] 290:[1422916608] 291:[1424121600] 292:[1424982976] 293:[1426681920] 294:[1428046592] 295:[1428927040] 296:[1430388736] 297:[1430967456] 298:[1431724224] 299:[1434083328] 300:[1434882048] 301:[1435227904] 302:[1435628160] 303:[1436132224] 304:[1436509568] 305:[1436820160] 306:[1437206080] 307:[1437950720] 308:[1438352896] 309:[1438715008] 310:[1439823552] 311:[1440158976] 312:[1440351616] 313:[1440654528] 314:[1440928384] 315:[1441604736] 316:[1441856576] 317:[1442484736] 318:[1442851584] 319:[1445870976] 320:[1447073152] 321:[1448021696] 322:[1448938112] 323:[1449341824] 324:[1449566464] 325:[1449835520] 326:[1450147584] 327:[1453411072] 328:[1456621952] 329:[1457883264] 330:[1460340352] 331:[1461539840] 332:[1462206336] 333:[1463179136] 334:[1463538944] 335:[1463832064] ptrs[1-335] = 1:7831434 2:7831454 3:7831469 4:7831515 5:7831581 6:7831612 7:7831624 8:7831628 9:7831655 10:7831685 11:7831752 12:7831761 13:7831791 14:7831812 15:7831840 16:7831850 17:7831863 18:7831880 19:7831888 20:7831914 21:7831918 22:7831938 23:7831948 24:7832184 25:7832191 26:7832205 27:7832217 28:7832236 29:7832259 30:7832269 31:7832319 32:7832328 33:7832394 34:7832401 35:7832411 36:7832479 37:7832489 38:7832495 39:7832616 40:7832649 41:7832674 42:7832687 43:7832697 44:7832729 45:7832744 46:7832885 47:7832920 48:7832939 49:7832946 50:7833037 51:7833279 52:7833312 53:7833388 54:7833399 55:7833424 56:7833560 57:7833566 58:7833742 59:7833788 60:7833791 61:7833822 62:7834001 63:7834032 64:7834039 65:7834056 66:7834084 67:7834230 68:7834298 69:7834679 70:7834690 71:7834756 72:7834914 73:7834959 74:7835107 75:7835225 76:7835243 77:7835388 78:7835497 79:7835508 80:7835737 81:7835759 82:7835769 83:7835775 84:7836319 85:7836472 86:7836555 87:7836557 88:7836607 89:7836628 90:7836702 91:7836714 92:7838820 93:7839025 94:7839030 95:7839034 96:7839159 97:7839210 98:7839220 99:7839243 100:7839317 101:7839967 102:7839970 103:7840045 104:7840232 105:7840262 106:7840340 107:7840363 108:7840427 109:7840492 110:7840527 111:7840540 112:7840584 113:7840651 114:7840698 115:7840806 116:7840835 117:7841438 118:7841500 119:7841631 120:7841695 121:7841704 122:7841864 123:7841882 124:7842286 125:7842367 126:7842385 127:7842408 128:7842456 129:7842519 130:7842555 131:7842587 132:7842590 133:7842607 134:7842620 135:7842632 136:7842700 137:7842725 138:7844503 139:7844559 140:7844598 141:7844794 142:7844814 143:7844872 144:7845233 145:7845237 146:7845388 147:7845404 148:7845407 149:7845502 150:7845512 151:7845514 152:7845516 153:7845518 154:7845678 155:7845832 156:7845841 157:7846015 158:7846121 159:7846205 160:7846344 161:7846346 162:7846447 163:7846560 164:7846571 165:7846718 166:7846744 167:7846776 168:7846815 169:7846830 170:7846879 171:7846888 172:7846893 173:7846904 174:7846942 175:7846958 176:7846973 177:7847002 178:7847006 179:7847017 180:7847045 181:7847077 182:7847085 183:7847097 184:7847117 185:7847133 186:7847178 187:7847203 188:7847241 189:7847245 190:7847267 191:7847304 192:7847373 193:7847401 194:7847576 195:7847588 196:7847592 197:7847663 198:7847679 199:7847689 200:7847716 201:7847729 202:7847836 203:7847859 204:7847879 205:7847883 206:7847904 207:7847908 208:7847920 209:7847937 210:7847967 211:7847980 212:7847995 213:7848019 214:7848029 215:7848044 216:7848079 217:7848088 218:7848107 219:7848111 220:7848195 221:7848202 222:7848209 223:7848238 224:7848259 225:7848267 226:7848274 227:7848376 228:7848381 229:7848398 230:7848436 231:7848449 232:7848488 233:7848520 234:7848601 235:7848621 236:7848682 237:7848684 238:7849805 239:7849864 240:7849882 241:7849959 242:7850079 243:7850189 244:7850201 245:7850469 246:7850817 247:7850832 248:7850838 249:7850856 250:7850961 251:7851311 252:7851329 253:7851438 254:7851839 255:7851848 256:7852670 257:7853114 258:7853150 259:7853182 260:7853210 261:7853308 262:7853312 263:7853405 264:7853412 265:7853418 266:7853422 267:7853600 268:7853602 269:7853652 270:7853673 271:7853805 272:7854192 273:7854207 274:7854320 275:7854358 276:7854404 277:7854432 278:7854456 279:7854487 280:7854542 281:7854578 282:7854623 283:7854654 284:7854673 285:7854702 286:7854708 287:7854713 288:7854722 289:7854775 290:7854779 291:7854793 292:7854799 293:7854816 294:7854818 295:7854829 296:7854865 297:7854942 298:7854952 299:7854979 300:7855040 301:7855251 302:7855257 303:7855262 304:7855362 305:7855656 306:7855759 307:7856074 308:7856095 309:7856111 310:7856131 311:7856153 312:7856157 313:7856181 314:7856252 315:7856255 316:7856297 317:7856467 318:7856519 319:7856762 320:7856767 321:7856785 322:7856787 323:7856800 324:7856827 325:7856833 326:7856891 327:7857097 328:7857180 329:7857343 330:7857530 331:7857787 332:7857791 333:7857856 334:7857860 335:7857900
