fs/xfs/xfs_vnodeops.c. Return address = 0xffffffffa02ee20a
The return address always 0xffffffffa02ee20a?2013/4/12 符永涛 <yongtaofu@xxxxxxxxx>
It happens tonight again on one of our servers, how to debug the root cause? Thank you.xfs_repair -n
Apr 12 02:32:10 cqdx kernel: XFS (sdb): xfs_iunlink_remove: xfs_inotobp() returned error 22.
Apr 12 02:32:10 cqdx kernel: XFS (sdb): xfs_inactive: xfs_ifree returned error 22
Apr 12 02:32:10 cqdx kernel: XFS (sdb): xfs_do_force_shutdown(0x1) called from line 1184 of file fs/xfs/xfs_vnodeops.c. Return address = 0xffffffffa02ee20a
Apr 12 02:32:10 cqdx kernel: XFS (sdb): I/O Error Detected. Shutting down filesystem
Apr 12 02:32:10 cqdx kernel: XFS (sdb): Please umount the filesystem and rectify the problem(s)
Apr 12 02:32:19 cqdx kernel: XFS (sdb): xfs_log_force: error 5 returned.
Apr 12 02:32:49 cqdx kernel: XFS (sdb): xfs_log_force: error 5 returned.
Apr 12 02:33:19 cqdx kernel: XFS (sdb): xfs_log_force: error 5 returned.
Apr 12 02:33:49 cqdx kernel: XFS (sdb): xfs_log_force: error 5 returned.
Phase 7 - verify link counts...
would have reset inode 20021 nlinks from 0 to 1
would have reset inode 20789 nlinks from 0 to 1
would have reset inode 35125 nlinks from 0 to 1
would have reset inode 35637 nlinks from 0 to 1
would have reset inode 36149 nlinks from 0 to 1
would have reset inode 38197 nlinks from 0 to 1
would have reset inode 39477 nlinks from 0 to 1
would have reset inode 54069 nlinks from 0 to 1
would have reset inode 62261 nlinks from 0 to 1
would have reset inode 63029 nlinks from 0 to 1
would have reset inode 72501 nlinks from 0 to 1
would have reset inode 79925 nlinks from 0 to 1
would have reset inode 81205 nlinks from 0 to 1
would have reset inode 84789 nlinks from 0 to 1
would have reset inode 87861 nlinks from 0 to 1
would have reset inode 90663 nlinks from 0 to 1
would have reset inode 91189 nlinks from 0 to 1
would have reset inode 95541 nlinks from 0 to 1
would have reset inode 98101 nlinks from 0 to 1
would have reset inode 101173 nlinks from 0 to 1
would have reset inode 113205 nlinks from 0 to 1
would have reset inode 114741 nlinks from 0 to 1
would have reset inode 126261 nlinks from 0 to 1
would have reset inode 140597 nlinks from 0 to 1
would have reset inode 144693 nlinks from 0 to 1
would have reset inode 147765 nlinks from 0 to 1
would have reset inode 152885 nlinks from 0 to 1
would have reset inode 161333 nlinks from 0 to 1
would have reset inode 161845 nlinks from 0 to 1
would have reset inode 167477 nlinks from 0 to 1
would have reset inode 172341 nlinks from 0 to 1
would have reset inode 191797 nlinks from 0 to 1
would have reset inode 204853 nlinks from 0 to 1
would have reset inode 205365 nlinks from 0 to 1
would have reset inode 215349 nlinks from 0 to 1
would have reset inode 215861 nlinks from 0 to 1
would have reset inode 216373 nlinks from 0 to 1
would have reset inode 217397 nlinks from 0 to 1
would have reset inode 224309 nlinks from 0 to 1
would have reset inode 225589 nlinks from 0 to 1
would have reset inode 234549 nlinks from 0 to 1
would have reset inode 234805 nlinks from 0 to 1
would have reset inode 249653 nlinks from 0 to 1
would have reset inode 250677 nlinks from 0 to 1
would have reset inode 252469 nlinks from 0 to 1
would have reset inode 261429 nlinks from 0 to 1
would have reset inode 265013 nlinks from 0 to 1
would have reset inode 266805 nlinks from 0 to 1
would have reset inode 267317 nlinks from 0 to 1
would have reset inode 268853 nlinks from 0 to 1
would have reset inode 272437 nlinks from 0 to 1
would have reset inode 273205 nlinks from 0 to 1
would have reset inode 274229 nlinks from 0 to 1
would have reset inode 278325 nlinks from 0 to 1
would have reset inode 278837 nlinks from 0 to 1
would have reset inode 281397 nlinks from 0 to 1
would have reset inode 292661 nlinks from 0 to 1
would have reset inode 300853 nlinks from 0 to 1
would have reset inode 302901 nlinks from 0 to 1
would have reset inode 305205 nlinks from 0 to 1
would have reset inode 314165 nlinks from 0 to 1
would have reset inode 315189 nlinks from 0 to 1
would have reset inode 320309 nlinks from 0 to 1
would have reset inode 324917 nlinks from 0 to 1
would have reset inode 328245 nlinks from 0 to 1
would have reset inode 335925 nlinks from 0 to 1
would have reset inode 339253 nlinks from 0 to 1
would have reset inode 339765 nlinks from 0 to 1
would have reset inode 348213 nlinks from 0 to 1
would have reset inode 360501 nlinks from 0 to 1
would have reset inode 362037 nlinks from 0 to 1
would have reset inode 366389 nlinks from 0 to 1
would have reset inode 385845 nlinks from 0 to 1
would have reset inode 390709 nlinks from 0 to 1
would have reset inode 409141 nlinks from 0 to 1
would have reset inode 413237 nlinks from 0 to 1
would have reset inode 414773 nlinks from 0 to 1
would have reset inode 417845 nlinks from 0 to 1
would have reset inode 436021 nlinks from 0 to 1
would have reset inode 439349 nlinks from 0 to 1
would have reset inode 447029 nlinks from 0 to 1
would have reset inode 491317 nlinks from 0 to 1
would have reset inode 494133 nlinks from 0 to 1
would have reset inode 495413 nlinks from 0 to 1
would have reset inode 501301 nlinks from 0 to 1
would have reset inode 506421 nlinks from 0 to 1
would have reset inode 508469 nlinks from 0 to 1
would have reset inode 508981 nlinks from 0 to 1
would have reset inode 511797 nlinks from 0 to 1
would have reset inode 513077 nlinks from 0 to 1
would have reset inode 517941 nlinks from 0 to 1
would have reset inode 521013 nlinks from 0 to 1
would have reset inode 522805 nlinks from 0 to 1
would have reset inode 523317 nlinks from 0 to 1
would have reset inode 525621 nlinks from 0 to 1
would have reset inode 527925 nlinks from 0 to 1
would have reset inode 535605 nlinks from 0 to 1
would have reset inode 541749 nlinks from 0 to 1
would have reset inode 573493 nlinks from 0 to 1
would have reset inode 578613 nlinks from 0 to 1
would have reset inode 583029 nlinks from 0 to 1
would have reset inode 585525 nlinks from 0 to 1
would have reset inode 586293 nlinks from 0 to 1
would have reset inode 586805 nlinks from 0 to 1
would have reset inode 591413 nlinks from 0 to 1
would have reset inode 594485 nlinks from 0 to 1
would have reset inode 596277 nlinks from 0 to 1
would have reset inode 603189 nlinks from 0 to 1
would have reset inode 613429 nlinks from 0 to 1
would have reset inode 617781 nlinks from 0 to 1
would have reset inode 621877 nlinks from 0 to 1
would have reset inode 623925 nlinks from 0 to 1
would have reset inode 625205 nlinks from 0 to 1
would have reset inode 626741 nlinks from 0 to 1
would have reset inode 639541 nlinks from 0 to 1
would have reset inode 640053 nlinks from 0 to 1
would have reset inode 640565 nlinks from 0 to 1
would have reset inode 645173 nlinks from 0 to 1
would have reset inode 652853 nlinks from 0 to 1
would have reset inode 656181 nlinks from 0 to 1
would have reset inode 659253 nlinks from 0 to 1
would have reset inode 663605 nlinks from 0 to 1
would have reset inode 667445 nlinks from 0 to 1
would have reset inode 680757 nlinks from 0 to 1
would have reset inode 691253 nlinks from 0 to 1
would have reset inode 691765 nlinks from 0 to 1
would have reset inode 697653 nlinks from 0 to 1
would have reset inode 700469 nlinks from 0 to 1
would have reset inode 707893 nlinks from 0 to 1
would have reset inode 716853 nlinks from 0 to 1
would have reset inode 722229 nlinks from 0 to 1
would have reset inode 722741 nlinks from 0 to 1
would have reset inode 723765 nlinks from 0 to 1
would have reset inode 731957 nlinks from 0 to 1
would have reset inode 742965 nlinks from 0 to 1
would have reset inode 743477 nlinks from 0 to 1
would have reset inode 745781 nlinks from 0 to 1
would have reset inode 746293 nlinks from 0 to 1
would have reset inode 774453 nlinks from 0 to 1
would have reset inode 778805 nlinks from 0 to 1
would have reset inode 785013 nlinks from 0 to 1
would have reset inode 785973 nlinks from 0 to 1
would have reset inode 791349 nlinks from 0 to 1
would have reset inode 796981 nlinks from 0 to 1
would have reset inode 803381 nlinks from 0 to 1
would have reset inode 806965 nlinks from 0 to 1
would have reset inode 811798 nlinks from 0 to 1
would have reset inode 812310 nlinks from 0 to 1
would have reset inode 813078 nlinks from 0 to 1
would have reset inode 813607 nlinks from 0 to 1
would have reset inode 814183 nlinks from 0 to 1
would have reset inode 822069 nlinks from 0 to 1
would have reset inode 828469 nlinks from 0 to 1
would have reset inode 830005 nlinks from 0 to 1
would have reset inode 832053 nlinks from 0 to 1
would have reset inode 832565 nlinks from 0 to 1
would have reset inode 836661 nlinks from 0 to 1
would have reset inode 841013 nlinks from 0 to 1
would have reset inode 841525 nlinks from 0 to 1
would have reset inode 845365 nlinks from 0 to 1
would have reset inode 846133 nlinks from 0 to 1
would have reset inode 847157 nlinks from 0 to 1
would have reset inode 852533 nlinks from 0 to 1
would have reset inode 857141 nlinks from 0 to 1
would have reset inode 863271 nlinks from 0 to 1
would have reset inode 866855 nlinks from 0 to 1
would have reset inode 887861 nlinks from 0 to 1
would have reset inode 891701 nlinks from 0 to 1
would have reset inode 894773 nlinks from 0 to 1
would have reset inode 900149 nlinks from 0 to 1
would have reset inode 902197 nlinks from 0 to 1
would have reset inode 906293 nlinks from 0 to 1
would have reset inode 906805 nlinks from 0 to 1
would have reset inode 909877 nlinks from 0 to 1
would have reset inode 925493 nlinks from 0 to 1
would have reset inode 949543 nlinks from 0 to 1
would have reset inode 955175 nlinks from 0 to 1
would have reset inode 963623 nlinks from 0 to 1
would have reset inode 967733 nlinks from 0 to 1
would have reset inode 968231 nlinks from 0 to 1
would have reset inode 982069 nlinks from 0 to 1
would have reset inode 1007413 nlinks from 0 to 1
would have reset inode 1011509 nlinks from 0 to 1
would have reset inode 1014069 nlinks from 0 to 1
would have reset inode 1014581 nlinks from 0 to 1
would have reset inode 1022005 nlinks from 0 to 1
would have reset inode 1022517 nlinks from 0 to 1
would have reset inode 1023029 nlinks from 0 to 1
would have reset inode 1025333 nlinks from 0 to 1
would have reset inode 1043765 nlinks from 0 to 1
would have reset inode 1044789 nlinks from 0 to 1
would have reset inode 1049397 nlinks from 0 to 1
would have reset inode 1050933 nlinks from 0 to 1
would have reset inode 1051445 nlinks from 0 to 1
would have reset inode 1054261 nlinks from 0 to 1
would have reset inode 1060917 nlinks from 0 to 1
would have reset inode 1063477 nlinks from 0 to 1
would have reset inode 1076021 nlinks from 0 to 1
would have reset inode 1081141 nlinks from 0 to 1
would have reset inode 1086261 nlinks from 0 to 1
would have reset inode 1097269 nlinks from 0 to 1
would have reset inode 1099829 nlinks from 0 to 1
would have reset inode 1100853 nlinks from 0 to 1
would have reset inode 1101877 nlinks from 0 to 1
would have reset inode 1126709 nlinks from 0 to 1
would have reset inode 1134389 nlinks from 0 to 1
would have reset inode 1141045 nlinks from 0 to 1
would have reset inode 1141557 nlinks from 0 to 1
would have reset inode 1142581 nlinks from 0 to 1
would have reset inode 1148469 nlinks from 0 to 1
would have reset inode 1153333 nlinks from 0 to 1
would have reset inode 1181749 nlinks from 0 to 1
would have reset inode 1192245 nlinks from 0 to 1
would have reset inode 1198133 nlinks from 0 to 1
would have reset inode 1203765 nlinks from 0 to 1
would have reset inode 1221429 nlinks from 0 to 1
would have reset inode 1223989 nlinks from 0 to 1
would have reset inode 1235509 nlinks from 0 to 1
would have reset inode 1239349 nlinks from 0 to 1
would have reset inode 1240885 nlinks from 0 to 1
would have reset inode 1241397 nlinks from 0 to 1
would have reset inode 1241909 nlinks from 0 to 1
would have reset inode 1242421 nlinks from 0 to 1
would have reset inode 1244981 nlinks from 0 to 1
would have reset inode 1246517 nlinks from 0 to 1
would have reset inode 1253429 nlinks from 0 to 1
would have reset inode 1271861 nlinks from 0 to 1
would have reset inode 1274677 nlinks from 0 to 1
would have reset inode 1277749 nlinks from 0 to 1
would have reset inode 1278773 nlinks from 0 to 1
would have reset inode 1286709 nlinks from 0 to 1
would have reset inode 1288245 nlinks from 0 to 1
would have reset inode 1299765 nlinks from 0 to 1
would have reset inode 1302325 nlinks from 0 to 1
would have reset inode 1304885 nlinks from 0 to 1
would have reset inode 1305397 nlinks from 0 to 1
would have reset inode 1307509 nlinks from 0 to 1
would have reset inode 1309493 nlinks from 0 to 1
would have reset inode 1310517 nlinks from 0 to 1
would have reset inode 1311029 nlinks from 0 to 1
would have reset inode 1312053 nlinks from 0 to 1
would have reset inode 1316917 nlinks from 0 to 1
would have reset inode 1317941 nlinks from 0 to 1
would have reset inode 1320821 nlinks from 0 to 1
would have reset inode 1322805 nlinks from 0 to 1
would have reset inode 1332789 nlinks from 0 to 1
would have reset inode 1336373 nlinks from 0 to 1
would have reset inode 1345653 nlinks from 0 to 1
would have reset inode 1354549 nlinks from 0 to 1
would have reset inode 1361973 nlinks from 0 to 1
would have reset inode 1369909 nlinks from 0 to 1
would have reset inode 1372981 nlinks from 0 to 1
would have reset inode 1388853 nlinks from 0 to 1
would have reset inode 1402933 nlinks from 0 to 1
would have reset inode 1403445 nlinks from 0 to 1
would have reset inode 1420085 nlinks from 0 to 1
would have reset inode 1452853 nlinks from 0 to 1
would have reset inode 1456437 nlinks from 0 to 1
would have reset inode 1457973 nlinks from 0 to 1
would have reset inode 1459253 nlinks from 0 to 1
would have reset inode 1467957 nlinks from 0 to 1
would have reset inode 1471541 nlinks from 0 to 1
would have reset inode 1476661 nlinks from 0 to 1
would have reset inode 1479733 nlinks from 0 to 1
would have reset inode 1483061 nlinks from 0 to 1
would have reset inode 1484085 nlinks from 0 to 1
would have reset inode 1486133 nlinks from 0 to 1
would have reset inode 1489461 nlinks from 0 to 1
would have reset inode 1490037 nlinks from 0 to 1
would have reset inode 1492021 nlinks from 0 to 1
would have reset inode 1493557 nlinks from 0 to 1
would have reset inode 1494069 nlinks from 0 to 1
would have reset inode 1496885 nlinks from 0 to 1
would have reset inode 1498421 nlinks from 0 to 1
would have reset inode 1498933 nlinks from 0 to 1
would have reset inode 1499957 nlinks from 0 to 1
would have reset inode 1506101 nlinks from 0 to 1
would have reset inode 1507637 nlinks from 0 to 1
would have reset inode 1510453 nlinks from 0 to 1
would have reset inode 1514293 nlinks from 0 to 1
would have reset inode 1517365 nlinks from 0 to 1
would have reset inode 1520693 nlinks from 0 to 1
would have reset inode 1521973 nlinks from 0 to 1
would have reset inode 1530421 nlinks from 0 to 1
would have reset inode 1530933 nlinks from 0 to 1
would have reset inode 1537333 nlinks from 0 to 1
would have reset inode 1538357 nlinks from 0 to 1
would have reset inode 1548853 nlinks from 0 to 1
would have reset inode 1553973 nlinks from 0 to 1
would have reset inode 1557301 nlinks from 0 to 1
would have reset inode 1564213 nlinks from 0 to 1
would have reset inode 1564725 nlinks from 0 to 1
would have reset inode 1576501 nlinks from 0 to 1
would have reset inode 1580597 nlinks from 0 to 1
would have reset inode 1584693 nlinks from 0 to 1
would have reset inode 1586485 nlinks from 0 to 1
would have reset inode 1589301 nlinks from 0 to 1
would have reset inode 1589813 nlinks from 0 to 1
would have reset inode 1592629 nlinks from 0 to 1
would have reset inode 1595701 nlinks from 0 to 1
would have reset inode 1601077 nlinks from 0 to 1
would have reset inode 1623861 nlinks from 0 to 1
would have reset inode 1626677 nlinks from 0 to 1
would have reset inode 1627701 nlinks from 0 to 1
would have reset inode 1633333 nlinks from 0 to 1
would have reset inode 1639221 nlinks from 0 to 1
would have reset inode 1649205 nlinks from 0 to 1
would have reset inode 1686325 nlinks from 0 to 1
would have reset inode 1690677 nlinks from 0 to 1
would have reset inode 1693749 nlinks from 0 to 1
would have reset inode 1704757 nlinks from 0 to 1
would have reset inode 1707061 nlinks from 0 to 1
would have reset inode 1709109 nlinks from 0 to 1
would have reset inode 1719349 nlinks from 0 to 1
would have reset inode 1737013 nlinks from 0 to 1
would have reset inode 1741365 nlinks from 0 to 1
would have reset inode 1747509 nlinks from 0 to 1
would have reset inode 1770805 nlinks from 0 to 1
would have reset inode 1780789 nlinks from 0 to 1
would have reset inode 1793589 nlinks from 0 to 1
would have reset inode 1795125 nlinks from 0 to 1
would have reset inode 1800757 nlinks from 0 to 1
would have reset inode 1801269 nlinks from 0 to 1
would have reset inode 1802549 nlinks from 0 to 1
would have reset inode 1804085 nlinks from 0 to 1
would have reset inode 1817141 nlinks from 0 to 1
would have reset inode 1821749 nlinks from 0 to 1
would have reset inode 1832757 nlinks from 0 to 1
would have reset inode 1836341 nlinks from 0 to 1
would have reset inode 1856309 nlinks from 0 to 1
would have reset inode 1900597 nlinks from 0 to 1
would have reset inode 1902901 nlinks from 0 to 1
would have reset inode 1912373 nlinks from 0 to 1
would have reset inode 1943093 nlinks from 0 to 1
would have reset inode 1944373 nlinks from 0 to 1
would have reset inode 1954101 nlinks from 0 to 1
would have reset inode 1955893 nlinks from 0 to 1
would have reset inode 1961781 nlinks from 0 to 1
would have reset inode 1974325 nlinks from 0 to 1
would have reset inode 1978677 nlinks from 0 to 1
would have reset inode 1981237 nlinks from 0 to 1
would have reset inode 1992245 nlinks from 0 to 1
would have reset inode 2000949 nlinks from 0 to 1
would have reset inode 2002229 nlinks from 0 to 1
would have reset inode 2004789 nlinks from 0 to 1
would have reset inode 2005301 nlinks from 0 to 1
would have reset inode 2011189 nlinks from 0 to 1
would have reset inode 2012981 nlinks from 0 to 1
would have reset inode 2015285 nlinks from 0 to 1
would have reset inode 2018869 nlinks from 0 to 1
would have reset inode 2028341 nlinks from 0 to 1
would have reset inode 2028853 nlinks from 0 to 1
would have reset inode 2030901 nlinks from 0 to 1
would have reset inode 2032181 nlinks from 0 to 1
would have reset inode 2032693 nlinks from 0 to 1
would have reset inode 2040117 nlinks from 0 to 1
would have reset inode 2053685 nlinks from 0 to 1
would have reset inode 2083893 nlinks from 0 to 1
would have reset inode 2087221 nlinks from 0 to 1
would have reset inode 2095925 nlinks from 0 to 1
would have reset inode 2098741 nlinks from 0 to 1
would have reset inode 2100533 nlinks from 0 to 1
would have reset inode 2101301 nlinks from 0 to 1
would have reset inode 2123573 nlinks from 0 to 1
would have reset inode 2132789 nlinks from 0 to 1
would have reset inode 2133813 nlinks from 0 to 1--2013/4/10 符永涛 <yongtaofu@xxxxxxxxx>
The storage info is as following:RAID-6
SATA HDD
Controller: PERC H710P Mini (Embedded)Disk /dev/sdb: 30000.3 GB, 30000346562560 bytes255 heads, 63 sectors/track, 3647334 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000sd 0:2:1:0: [sdb] 58594426880 512-byte logical blocks: (30.0 TB/27.2 TiB)sd 0:2:1:0: [sdb] Write Protect is offsd 0:2:1:0: [sdb] Mode Sense: 1f 00 00 08sd 0:2:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUAsd 0:2:1:0: [sdb] Attached SCSI disk*-storagedescription: RAID bus controllerproduct: MegaRAID SAS 2208 [Thunderbolt]vendor: LSI Logic / Symbios Logicphysical id: 0bus info: pci@0000:02:00.0logical name: scsi0version: 01width: 64 bitsclock: 33MHzcapabilities: storage pm pciexpress vpd msi msix bus_master cap_list romconfiguration: driver=megaraid_sas latency=0resources: irq:42 ioport:fc00(size=256) memory:dd7fc000-dd7fffff memory:dd780000-dd7bffff memory:dc800000-dc81ffff(prefetchable)*-disk:0description: SCSI Diskproduct: PERC H710Pvendor: DELLphysical id: 2.0.0bus info: scsi@0:2.0.0logical name: /dev/sdaversion: 3.13serial: 0049d6ce1d9f2035180096fde490f648size: 558GiB (599GB)capabilities: partitioned partitioned:dosconfiguration: ansiversion=5 signature=000aa336*-disk:1description: SCSI Diskproduct: PERC H710Pvendor: DELLphysical id: 2.1.0bus info: scsi@0:2.1.0logical name: /dev/sdblogical name: /mnt/xfsdversion: 3.13serial: 003366f71da22035180096fde490f648size: 27TiB (30TB)configuration: ansiversion=5 mount.fstype=xfs mount.options=rw,relatime,attr2,delaylog,logbsize=64k,sunit=128,swidth=1280,noquota state=mountedThank you.--2013/4/10 Emmanuel Florac <eflorac@xxxxxxxxxxxxxx>
Le Tue, 9 Apr 2013 23:10:03 +0800
符永涛 <yongtaofu@xxxxxxxxx> écrivait:
This. I/O error detected. That means that at some point the underlying
> > Apr 9 11:01:30 cqdx kernel: XFS (sdb): I/O Error Detected.
> > Shutting down filesystem
device (disk, RAID array, SAN volume) couldn't be reached. So this
could very well be a case of a flakey drive, array, cable or SCSI
driver.
What's the storage setup here?
--
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <eflorac@xxxxxxxxxxxxxx>
| +33 1 78 94 84 02
------------------------------------------------------------------------
符永涛
符永涛
--
符永涛
_______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
