CentOS 6.4 kernel 2.6.32-358.2.1.el6.x86_64 i`ll try to make a script that can be used out-of-house. several requests may have failed (un)conveniently, so we have passed through setting XLOG_IO_ERROR in xfs_log_force_unmount() and eventually skipped xlog_wait altogether: ----------------- <1>XFS (<device>): metadata I/O error: block 0xa03e1d ("xlog_iodone") error 5 buf count 1024 <5>XFS (<device>): xfs_do_force_shutdown(0x2) called from line 1052 of file fs/xfs/xfs_log.c. Return address = 0xffffffffa0559e51 <1>XFS (<device>): Log I/O Error Detected. Shutting down filesystem <1>XFS (<device>): Please umount the filesystem and rectify the problem(s) <4>XFS (<device>): xfs_log_force: error 5 returned. <5>XFS (<device>): xfs_do_force_shutdown(0x1) called from line 1063 of file fs/xfs/linux-2.6/xfs_buf.c. Return address = 0xffffffffa0575 <4>XFS (<device>): xfs_log_force: error 5 returned. <4>XFS (<device>): xfs_log_force: error 5 returned. ... <1>XFS (mtab~): metadata I/O error: block 0xa03e1b ("xlog_iodone") error 5 buf count 1024 --------------- Thanks. On 15 June 2013 05:03, Dave Chinner <david@xxxxxxxxxxxxx> wrote: > On Fri, Jun 14, 2013 at 09:15:04PM +0400, Vlad Bespalov wrote: >> i`m running an xfs filesystem over device going offline/online and >> sometimes offline may be done in parallel with unmounting >> >> at some point i got several crashes with null pointer panic in >> xlog_iodone: xlog_t structure taken from input buffer is null >> >> i wonder if the following call path combined with disk online/offline >> handling could have led to this crash: >> >> -------------- >> xfs_unmountfs() >> xfs_log_unmount_write(mp) >> xlog_state_release_iclog(log) >> xlog_sync(log, iclog = log->l_iclog) >> (bp=iclog->ic_bp) >> xlog_bdstrat(bp) >> (iclog->ic_state != XLOG_STATE_ERROR ? ) >> xfs_buf_iorequest(bp) >> xfs_buf_ioend (called with scheduling (*) ) >> (queues : bp->b_iodone_work, >> callback: xlog_iodone) > > Which is followed by: > > if (!(iclog->ic_state == XLOG_STATE_ACTIVE || > iclog->ic_state == XLOG_STATE_DIRTY)) { > if (!XLOG_FORCED_SHUTDOWN(log)) { > xlog_wait(&iclog->ic_force_wait, > &log->l_icloglock) > > Which is supposed to wait for the log IO to complete and hence > xlog_iodone() is supposed to have been run by the time this code > completes. > > What kernel are you tesing on? Do you have a script that reproduces > it? > > Cheers, > > Dave. > -- > Dave Chinner > david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs