On Thu, Mar 8, 2018 at 7:33 PM, 赵赵贺东 <zhaohedong@xxxxxxxxx> wrote: > Hi Brad, > > Thank you for your attention. > >> 在 2018年3月8日,下午4:47,Brad Hubbard <bhubbard@xxxxxxxxxx> 写道: >> >> On Thu, Mar 8, 2018 at 5:01 PM, 赵贺东 <zhaohedong@xxxxxxxxx> wrote: >>> Hi All, >>> >>> Every time after we activate osd, we got “Structure needs cleaning” in /var/lib/ceph/osd/ceph-xxx/current/meta. >>> >>> >>> /var/lib/ceph/osd/ceph-xxx/current/meta >>> # ls -l >>> ls: reading directory .: Structure needs cleaning >>> total 0 >>> >>> Could Anyone say something about this error? >> >> It's an indication of possible corruption on the filesystem containing "meta". >> >> Can you unmount it and run a filesystem check on it? > I did some xfs_repair operation, but no effect.Structure needs cleaning” still exist. > > > >> >> At the time the filesystem first detected the corruption it would have >> logged it to dmesg and possibly syslog which may give you a clue. Did >> you lose power or have a kernel panic or something? > We did not lose power. > You are right, we get a metadata corruption in dmesg&syslog every time just following the osd activating operation. > > [ 399.513525] XFS (sda1): Metadata corruption detected at xfs_dir3_data_read_verify+0x58/0xd0, xfs_dir3_data block 0x48b9ff80 > [ 399.524709] XFS (sda1): Unmount and run xfs_repair > [ 399.529511] XFS (sda1): First 64 bytes of corrupted metadata buffer: > [ 399.535917] dd8f2000: 58 46 53 42 00 00 10 00 00 00 00 00 91 73 fe fb XFSB.........s.. > [ 399.543959] dd8f2010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 399.551983] dd8f2020: e5 30 40 22 51 8f 4f 1c 80 73 56 9b 71 aa 92 24 .0@"Q.O..sV.q..$ > [ 399.560037] dd8f2030: 00 00 00 00 80 00 00 07 ff ff ff ff ff ff ff ff ................ > [ 399.568118] XFS (sda1): metadata I/O error: block 0x48b9ff80 ("xfs_trans_read_buf_map") error 117 numblks 8 > [ 399.583179] XFS (sda1): Metadata corruption detected at xfs_dir3_data_read_verify+0x58/0xd0, xfs_dir3_data block 0x48b9ff80 > [ 399.594378] XFS (sda1): Unmount and run xfs_repair > [ 399.599182] XFS (sda1): First 64 bytes of corrupted metadata buffer: > [ 399.605575] e47db000: 58 46 53 42 00 00 10 00 00 00 00 00 91 73 fe fb XFSB.........s.. > [ 399.613613] e47db010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 399.621637] e47db020: e5 30 40 22 51 8f 4f 1c 80 73 56 9b 71 aa 92 24 .0@"Q.O..sV.q..$ > [ 399.629679] e47db030: 00 00 00 00 80 00 00 07 ff ff ff ff ff ff ff ff ................ > [ 399.637856] XFS (sda1): metadata I/O error: block 0x48b9ff80 ("xfs_trans_read_buf_map") error 117 numblks 8 > [ 399.648165] XFS (sda1): Metadata corruption detected at xfs_dir3_data_read_verify+0x58/0xd0, xfs_dir3_data block 0x48b9ff80 > [ 399.659378] XFS (sda1): Unmount and run xfs_repair > [ 399.664196] XFS (sda1): First 64 bytes of corrupted metadata buffer: > [ 399.670570] e47db000: 58 46 53 42 00 00 10 00 00 00 00 00 91 73 fe fb XFSB.........s.. > [ 399.678610] e47db010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 399.686643] e47db020: e5 30 40 22 51 8f 4f 1c 80 73 56 9b 71 aa 92 24 .0@"Q.O..sV.q..$ > [ 399.694681] e47db030: 00 00 00 00 80 00 00 07 ff ff ff ff ff ff ff ff ................ > [ 399.702794] XFS (sda1): metadata I/O error: block 0x48b9ff80 ("xfs_trans_read_buf_map") error 117 numblks 8 I'd suggest the next step is to look for a matching XFS bug in your distro and, if possible, try a different distro and see if you get the same result. > > > Thank you ! > > >> >>> >>> Thank you! >>> >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@xxxxxxxxxxxxxx >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >> >> >> -- >> Cheers, >> Brad > -- Cheers, Brad _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com