I am hitting what seems to be a bug in one of my Linux storage layers, this is
my system:
distro: Arch Linux x86_64
cpu: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz (4 cores)
skuda@skuda ~/Desktop $ uname -a
Linux skuda 3.7.1-426-bfs #1 SMP PREEMPT Wed Dec 26 09:56:18 CET 2012 x86_64
GNU/Linux
skuda@skuda ~/Desktop $ xfs_repair -V
xfs_repair version 3.1.8
skuda@skuda ~/Desktop $ cat /proc/mounts
rootfs / rootfs rw 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sys /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
dev /dev devtmpfs rw,nosuid,relatime,size=8200368k,nr_inodes=2050092,mode=755 0 0
run /run tmpfs rw,nosuid,nodev,relatime,mode=755 0 0
/dev/sda3 / ext4 rw,relatime,discard,stripe=128,data=ordered 0 0
devpts /dev/pts devpts rw,relatime,mode=600,ptmxmode=000 0 0
shm /dev/shm tmpfs rw,nosuid,nodev,relatime 0 0
/dev/sda1 /boot ext4 rw,relatime,stripe=128,data=ordered 0 0
/dev/md1 /mnt/datos xfs
rw,relatime,attr2,nobarrier,inode64,logbsize=256k,sunit=512,swidth=1024,noquota 0 0
tmpfs /tmp tmpfs rw,nosuid,nodev,relatime 0 0
binfmt /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0
skuda@skuda ~/Desktop $ mdadm --detail /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Sat Apr 2 15:14:11 2011
Raid Level : raid5
Array Size : 1953522688 (1863.02 GiB 2000.41 GB)
Used Dev Size : 976761344 (931.51 GiB 1000.20 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Mon Jan 7 17:44:38 2013
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 256K
Name : sysresccd:1
UUID : 47dddb83:9fdb7858:1d71bb15:9b35a1e0
Events : 12873
Number Major Minor RaidDevice State
0 8 32 0 active sync /dev/sdc
1 8 48 1 active sync /dev/sdd
3 8 16 2 active sync /dev/sdb
skuda@skuda ~/Desktop $ mdadm -V
mdadm - v3.2.6 - 25th October 2012
The three hard drives in raid are Western Digital RE4 WDC WD1003FBYX with write
cache enabled (yes i know i have write barries disabled in fstab).
I use this raid to backup my machine with rsnapshot than uses internally rsync,
it seems that, given sufficient pressure, i can almost always crash xfs
filesystem and my machine with it, i have attached what it's logged in
/var/log/messages.
Regards,
Miguel Angel.
Jan 6 21:53:01 skuda kernel: [74266.804938] ------------[ cut here ]------------
Jan 6 21:53:01 skuda kernel: [74266.804948] WARNING: at kernel/workqueue.c:1550 worker_enter_idle+0xd5/0x130()
Jan 6 21:53:01 skuda kernel: [74266.804950] Hardware name: System Product Name
Jan 6 21:53:01 skuda kernel: [74266.804951] Modules linked in: tun pci_stub vboxpci(O) vboxnetflt(O) vboxnetadp(O) vboxdrv(O) ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip_tables x_tables xfs snd_hda_codec_hdmi raid456 async_raid6_recov async_memcpy async_pq iTCO_wdt iTCO_vendor_support snd_hda_codec_realtek raid6_pq async_xor kvm_intel kvm crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 ablk_helper cryptd xor xts async_tx lrw gf128mul uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_core snd_usb_audio videodev snd_usbmidi_lib snd_rawmidi eeepc_wmi asus_wmi snd_seq_device sparse_keymap pci_hotplug md_mod media mxm_wmi evdev joydev btusb bluetooth nvidia(PO) rfkill psmouse microcode serio_raw pcspkr lpc_ich snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_page_alloc snd_timer snd i2c_core e1000e soundcore video thermal fan msr wmi mei acpi_cpufreq mperf button processor coretemp pppoe pppox ppp_generic slhc nf_nat_snmp_basic nf_conntrack_snmp nf_conntrack_broadcast
nf_nat_proto_sctp crc32c libcrc32c nf_nat_proto_dccp nf_nat_proto_udplite nf_nat_amanda ts_kmp nf_conntrack_amanda nf_nat_irc nf_conntrack_irc nf_nat_sip nf_conntrack_sip nf_nat_tftp nf_conntrack_tftp nf_nat_h323 nf_conntrack_h323 nf_nat_pptp nf_nat_proto_gre nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_ftp nf_conntrack_ftp nf_nat nf_conntrack fuse loop ext4 crc16 jbd2 mbcache usb_storage hid_generic usbhid hid sr_mod cdrom sd_mod ehci_hcd ahci libahci libata xhci_hcd scsi_mod usbcore usb_common
Jan 6 21:53:01 skuda kernel: [74266.805058] Pid: 27771, comm: kworker/0:2 Tainted: P O 3.7.1-426-bfs #1
Jan 6 21:53:01 skuda kernel: [74266.805060] Call Trace:
Jan 6 21:53:01 skuda kernel: [74266.805066] [<ffffffff8105740f>] warn_slowpath_common+0x7f/0xc0
Jan 6 21:53:01 skuda kernel: [74266.805070] [<ffffffff810728d0>] ? cwq_activate_delayed_work+0xc0/0xc0
Jan 6 21:53:01 skuda kernel: [74266.805073] [<ffffffff8105746a>] warn_slowpath_null+0x1a/0x20
Jan 6 21:53:01 skuda kernel: [74266.805076] [<ffffffff810729c5>] worker_enter_idle+0xd5/0x130
Jan 6 21:53:01 skuda kernel: [74266.805080] [<ffffffff81076481>] worker_thread+0x1f1/0x400
Jan 6 21:53:01 skuda kernel: [74266.805085] [<ffffffff81492fd9>] ? preempt_schedule+0x49/0x70
Jan 6 21:53:01 skuda kernel: [74266.805088] [<ffffffff81076290>] ? rescuer_thread+0x240/0x240
Jan 6 21:53:01 skuda kernel: [74266.805093] [<ffffffff8107aec0>] kthread+0xc0/0xd0
Jan 6 21:53:01 skuda kernel: [74266.805097] [<ffffffff81010000>] ? perf_trace_xen_mmu_set_pte_at+0xb0/0x100
Jan 6 21:53:01 skuda kernel: [74266.805102] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
Jan 6 21:53:01 skuda kernel: [74266.805106] [<ffffffff8149b76c>] ret_from_fork+0x7c/0xb0
Jan 6 21:53:01 skuda kernel: [74266.805110] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
Jan 6 21:53:01 skuda kernel: [74266.805113] ---[ end trace b0d33c4a02723ea3 ]---
Jan 6 21:58:03 skuda kernel: [74568.833540] xfsaild/md1 D 0000000000000001 0 500 2 0x00000000
Jan 6 21:58:03 skuda kernel: [74568.833545] ffff8803fd7fdaf8 0000000000000046 00000002a0e26770 ffff8803fd7fdfd8
Jan 6 21:58:03 skuda kernel: [74568.833549] ffff8804061d1800 ffff8803fd7fdfd8 ffff8803fd7fdfd8 ffff8803fd7fdfd8
Jan 6 21:58:03 skuda kernel: [74568.833553] ffffffff81814420 ffff880404835100 0000000000000001 ffff880406fcd400
Jan 6 21:58:03 skuda kernel: [74568.833557] Call Trace:
Jan 6 21:58:03 skuda kernel: [74568.833569] [<ffffffffa0e266e5>] ? release_stripe_plug+0x25/0x80 [raid456]
Jan 6 21:58:03 skuda kernel: [74568.833574] [<ffffffffa0e2cf77>] ? make_request+0x2b7/0x6c0 [raid456]
Jan 6 21:58:03 skuda kernel: [74568.833581] [<ffffffff8107bbb0>] ? abort_exclusive_wait+0xb0/0xb0
Jan 6 21:58:03 skuda kernel: [74568.833586] [<ffffffff81490254>] schedule_timeout+0x1f4/0x260
Jan 6 21:58:03 skuda kernel: [74568.833591] [<ffffffff81113ab5>] ? mempool_alloc_slab+0x15/0x20
Jan 6 21:58:03 skuda kernel: [74568.833595] [<ffffffff81491b3b>] wait_for_common+0xcb/0x160
Jan 6 21:58:03 skuda kernel: [74568.833600] [<ffffffff8108b050>] ? try_to_wake_up+0x1f0/0x1f0
Jan 6 21:58:03 skuda kernel: [74568.833604] [<ffffffff81491bed>] wait_for_completion+0x1d/0x20
Jan 6 21:58:03 skuda kernel: [74568.833607] [<ffffffff8107460b>] flush_work+0xeb/0x170
Jan 6 21:58:03 skuda kernel: [74568.833610] [<ffffffff810728d0>] ? cwq_activate_delayed_work+0xc0/0xc0
Jan 6 21:58:03 skuda kernel: [74568.833625] [<ffffffffa0e9b9a9>] xlog_cil_force_lsn+0x39/0x160 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833629] [<ffffffff81067ff8>] ? lock_timer_base.isra.37+0x38/0x70
Jan 6 21:58:03 skuda kernel: [74568.833632] [<ffffffff8106807f>] ? try_to_del_timer_sync+0x4f/0x70
Jan 6 21:58:03 skuda kernel: [74568.833645] [<ffffffffa0e99ce9>] _xfs_log_force+0x69/0x280 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833648] [<ffffffff81490187>] ? schedule_timeout+0x127/0x260
Jan 6 21:58:03 skuda kernel: [74568.833658] [<ffffffffa0e422c0>] ? xfs_buf_delwri_submit_nowait+0x20/0x30 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833669] [<ffffffffa0e99f2a>] xfs_log_force+0x2a/0xc0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833680] [<ffffffffa0e9e814>] xfsaild+0x144/0x700 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833685] [<ffffffff810846ff>] ? __wake_up_common+0x4f/0x80
Jan 6 21:58:03 skuda kernel: [74568.833696] [<ffffffffa0e9e6d0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833700] [<ffffffff8107aec0>] kthread+0xc0/0xd0
Jan 6 21:58:03 skuda kernel: [74568.833704] [<ffffffff81010000>] ? perf_trace_xen_mmu_set_pte_at+0xb0/0x100
Jan 6 21:58:03 skuda kernel: [74568.833709] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
Jan 6 21:58:03 skuda kernel: [74568.833713] [<ffffffff8149b76c>] ret_from_fork+0x7c/0xb0
Jan 6 21:58:03 skuda kernel: [74568.833717] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
Jan 6 21:58:03 skuda kernel: [74568.833761] rsync D 0000000000000001 0 20348 2830 0x00000000
Jan 6 21:58:03 skuda kernel: [74568.833765] ffff8801753afcb8 0000000000000086 ffff8800000000c8 ffff8801753affd8
Jan 6 21:58:03 skuda kernel: [74568.833768] 00000000340643e8 ffff8801753affd8 ffff8801753affd8 ffff8801753affd8
Jan 6 21:58:03 skuda kernel: [74568.833772] ffffffff81814420 ffff8802ad26de80 ffff8801753afce8 ffffffff81497614
Jan 6 21:58:03 skuda kernel: [74568.833775] Call Trace:
Jan 6 21:58:03 skuda kernel: [74568.833779] [<ffffffff81497614>] ? __do_page_fault+0x2d4/0x590
Jan 6 21:58:03 skuda kernel: [74568.833788] [<ffffffffa0e40c35>] ? xfs_buf_get_map+0x35/0x1d0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833797] [<ffffffffa0e41c23>] ? xfs_buf_read_map+0x33/0x140 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833808] [<ffffffffa0e9fc5d>] ? xfs_trans_read_buf_map+0x29d/0x4f0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833819] [<ffffffffa0e976ff>] xlog_grant_head_wait+0xaf/0x210 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833829] [<ffffffffa0e978ee>] xlog_grant_head_check+0x8e/0xf0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833839] [<ffffffffa0e9a795>] xfs_log_reserve+0xc5/0x1f0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833851] [<ffffffffa0e587f7>] ? kmem_zone_alloc+0x67/0xe0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833862] [<ffffffffa0e95c73>] xfs_trans_reserve+0x223/0x230 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833873] [<ffffffffa0e958b7>] ? _xfs_trans_alloc+0x37/0xa0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833883] [<ffffffffa0e4d7eb>] xfs_vn_update_time+0x5b/0x1c0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833887] [<ffffffff81186720>] ? sys_ioctl+0xb0/0xb0
Jan 6 21:58:03 skuda kernel: [74568.833892] [<ffffffff8118e135>] update_time+0x25/0xc0
Jan 6 21:58:03 skuda kernel: [74568.833895] [<ffffffff8118e3b2>] touch_atime+0xf2/0x140
Jan 6 21:58:03 skuda kernel: [74568.833899] [<ffffffff81186ada>] vfs_readdir+0xca/0xe0
Jan 6 21:58:03 skuda kernel: [74568.833902] [<ffffffff81186c0f>] sys_getdents+0x8f/0x110
Jan 6 21:58:03 skuda kernel: [74568.833906] [<ffffffff8149b81d>] system_call_fastpath+0x1a/0x1f
Jan 6 21:58:03 skuda kernel: [74568.833912] rsync D 0000000000000001 0 20349 20348 0x00000000
Jan 6 21:58:03 skuda kernel: [74568.833915] ffff8801c0d71c38 0000000000000086 00000000648bdf17 ffff8801c0d71fd8
Jan 6 21:58:03 skuda kernel: [74568.833918] ffff8800493cec80 ffff8801c0d71fd8 ffff8801c0d71fd8 ffff8801c0d71fd8
Jan 6 21:58:03 skuda kernel: [74568.833922] ffffffff81814420 ffff8802ad269b00 0000000000000000 0000000000000000
Jan 6 21:58:03 skuda kernel: [74568.833925] Call Trace:
Jan 6 21:58:03 skuda kernel: [74568.833935] [<ffffffffa0e48cfc>] ? xfs_iunlock+0x10c/0x130 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833945] [<ffffffffa0e48d2e>] ? xfs_iunlock_map_shared+0xe/0x10 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833949] [<ffffffff8118998a>] ? __d_instantiate+0x8a/0x100
Jan 6 21:58:03 skuda kernel: [74568.833953] [<ffffffff81083991>] ? in_group_p+0x31/0x40
Jan 6 21:58:03 skuda kernel: [74568.833956] [<ffffffff8117fa26>] ? generic_permission+0x176/0x280
Jan 6 21:58:03 skuda kernel: [74568.833967] [<ffffffffa0e976ff>] xlog_grant_head_wait+0xaf/0x210 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833977] [<ffffffffa0e97949>] xlog_grant_head_check+0xe9/0xf0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833987] [<ffffffffa0e9a795>] xfs_log_reserve+0xc5/0x1f0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.833997] [<ffffffffa0e95c73>] xfs_trans_reserve+0x223/0x230 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.834007] [<ffffffffa0e958b7>] ? _xfs_trans_alloc+0x37/0xa0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.834019] [<ffffffffa0e5691b>] xfs_create+0x1eb/0x690 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.834022] [<ffffffff8117edab>] ? lookup_dcache+0x3b/0xd0
Jan 6 21:58:03 skuda kernel: [74568.834032] [<ffffffffa0e4d386>] xfs_vn_mknod+0xa6/0x1b0 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.834042] [<ffffffffa0e4d4a6>] xfs_vn_mkdir+0x16/0x20 [xfs]
Jan 6 21:58:03 skuda kernel: [74568.834045] [<ffffffff81180c72>] vfs_mkdir+0xc2/0x140
Jan 6 21:58:03 skuda kernel: [74568.834048] [<ffffffff81184793>] sys_mkdirat+0xa3/0xb0
Jan 6 21:58:03 skuda kernel: [74568.834051] [<ffffffff811847b9>] sys_mkdir+0x19/0x20
Jan 6 21:58:03 skuda kernel: [74568.834055] [<ffffffff8149b81d>] system_call_fastpath+0x1a/0x1f
Jan 6 22:00:03 skuda kernel: [74688.718560] xfsaild/md1 D 0000000000000001 0 500 2 0x00000000
Jan 6 22:00:03 skuda kernel: [74688.718565] ffff8803fd7fdaf8 0000000000000046 00000002a0e26770 ffff8803fd7fdfd8
Jan 6 22:00:03 skuda kernel: [74688.718569] ffff8804061d1800 ffff8803fd7fdfd8 ffff8803fd7fdfd8 ffff8803fd7fdfd8
Jan 6 22:00:03 skuda kernel: [74688.718573] ffffffff81814420 ffff880404835100 0000000000000001 ffff880406fcd400
Jan 6 22:00:03 skuda kernel: [74688.718576] Call Trace:
Jan 6 22:00:03 skuda kernel: [74688.718588] [<ffffffffa0e266e5>] ? release_stripe_plug+0x25/0x80 [raid456]
Jan 6 22:00:03 skuda kernel: [74688.718593] [<ffffffffa0e2cf77>] ? make_request+0x2b7/0x6c0 [raid456]
Jan 6 22:00:03 skuda kernel: [74688.718600] [<ffffffff8107bbb0>] ? abort_exclusive_wait+0xb0/0xb0
Jan 6 22:00:03 skuda kernel: [74688.718605] [<ffffffff81490254>] schedule_timeout+0x1f4/0x260
Jan 6 22:00:03 skuda kernel: [74688.718610] [<ffffffff81113ab5>] ? mempool_alloc_slab+0x15/0x20
Jan 6 22:00:03 skuda kernel: [74688.718614] [<ffffffff81491b3b>] wait_for_common+0xcb/0x160
Jan 6 22:00:03 skuda kernel: [74688.718618] [<ffffffff8108b050>] ? try_to_wake_up+0x1f0/0x1f0
Jan 6 22:00:03 skuda kernel: [74688.718622] [<ffffffff81491bed>] wait_for_completion+0x1d/0x20
Jan 6 22:00:03 skuda kernel: [74688.718626] [<ffffffff8107460b>] flush_work+0xeb/0x170
Jan 6 22:00:03 skuda kernel: [74688.718629] [<ffffffff810728d0>] ? cwq_activate_delayed_work+0xc0/0xc0
Jan 6 22:00:03 skuda kernel: [74688.718644] [<ffffffffa0e9b9a9>] xlog_cil_force_lsn+0x39/0x160 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718648] [<ffffffff81067ff8>] ? lock_timer_base.isra.37+0x38/0x70
Jan 6 22:00:03 skuda kernel: [74688.718651] [<ffffffff8106807f>] ? try_to_del_timer_sync+0x4f/0x70
Jan 6 22:00:03 skuda kernel: [74688.718663] [<ffffffffa0e99ce9>] _xfs_log_force+0x69/0x280 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718667] [<ffffffff81490187>] ? schedule_timeout+0x127/0x260
Jan 6 22:00:03 skuda kernel: [74688.718676] [<ffffffffa0e422c0>] ? xfs_buf_delwri_submit_nowait+0x20/0x30 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718687] [<ffffffffa0e99f2a>] xfs_log_force+0x2a/0xc0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718698] [<ffffffffa0e9e814>] xfsaild+0x144/0x700 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718703] [<ffffffff810846ff>] ? __wake_up_common+0x4f/0x80
Jan 6 22:00:03 skuda kernel: [74688.718713] [<ffffffffa0e9e6d0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718717] [<ffffffff8107aec0>] kthread+0xc0/0xd0
Jan 6 22:00:03 skuda kernel: [74688.718722] [<ffffffff81010000>] ? perf_trace_xen_mmu_set_pte_at+0xb0/0x100
Jan 6 22:00:03 skuda kernel: [74688.718726] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
Jan 6 22:00:03 skuda kernel: [74688.718731] [<ffffffff8149b76c>] ret_from_fork+0x7c/0xb0
Jan 6 22:00:03 skuda kernel: [74688.718735] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
Jan 6 22:00:03 skuda kernel: [74688.718779] rsync D 0000000000000001 0 20348 2830 0x00000000
Jan 6 22:00:03 skuda kernel: [74688.718783] ffff8801753afcb8 0000000000000086 ffff8800000000c8 ffff8801753affd8
Jan 6 22:00:03 skuda kernel: [74688.718786] 00000000340643e8 ffff8801753affd8 ffff8801753affd8 ffff8801753affd8
Jan 6 22:00:03 skuda kernel: [74688.718790] ffffffff81814420 ffff8802ad26de80 ffff8801753afce8 ffffffff81497614
Jan 6 22:00:03 skuda kernel: [74688.718793] Call Trace:
Jan 6 22:00:03 skuda kernel: [74688.718797] [<ffffffff81497614>] ? __do_page_fault+0x2d4/0x590
Jan 6 22:00:03 skuda kernel: [74688.718806] [<ffffffffa0e40c35>] ? xfs_buf_get_map+0x35/0x1d0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718815] [<ffffffffa0e41c23>] ? xfs_buf_read_map+0x33/0x140 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718826] [<ffffffffa0e9fc5d>] ? xfs_trans_read_buf_map+0x29d/0x4f0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718837] [<ffffffffa0e976ff>] xlog_grant_head_wait+0xaf/0x210 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718847] [<ffffffffa0e978ee>] xlog_grant_head_check+0x8e/0xf0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718857] [<ffffffffa0e9a795>] xfs_log_reserve+0xc5/0x1f0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718870] [<ffffffffa0e587f7>] ? kmem_zone_alloc+0x67/0xe0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718881] [<ffffffffa0e95c73>] xfs_trans_reserve+0x223/0x230 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718891] [<ffffffffa0e958b7>] ? _xfs_trans_alloc+0x37/0xa0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718901] [<ffffffffa0e4d7eb>] xfs_vn_update_time+0x5b/0x1c0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718905] [<ffffffff81186720>] ? sys_ioctl+0xb0/0xb0
Jan 6 22:00:03 skuda kernel: [74688.718910] [<ffffffff8118e135>] update_time+0x25/0xc0
Jan 6 22:00:03 skuda kernel: [74688.718914] [<ffffffff8118e3b2>] touch_atime+0xf2/0x140
Jan 6 22:00:03 skuda kernel: [74688.718917] [<ffffffff81186ada>] vfs_readdir+0xca/0xe0
Jan 6 22:00:03 skuda kernel: [74688.718920] [<ffffffff81186c0f>] sys_getdents+0x8f/0x110
Jan 6 22:00:03 skuda kernel: [74688.718924] [<ffffffff8149b81d>] system_call_fastpath+0x1a/0x1f
Jan 6 22:00:03 skuda kernel: [74688.718929] rsync D 0000000000000001 0 20349 20348 0x00000000
Jan 6 22:00:03 skuda kernel: [74688.718932] ffff8801c0d71c38 0000000000000086 00000000648bdf17 ffff8801c0d71fd8
Jan 6 22:00:03 skuda kernel: [74688.718936] ffff8800493cec80 ffff8801c0d71fd8 ffff8801c0d71fd8 ffff8801c0d71fd8
Jan 6 22:00:03 skuda kernel: [74688.718939] ffffffff81814420 ffff8802ad269b00 0000000000000000 0000000000000000
Jan 6 22:00:03 skuda kernel: [74688.718943] Call Trace:
Jan 6 22:00:03 skuda kernel: [74688.718953] [<ffffffffa0e48cfc>] ? xfs_iunlock+0x10c/0x130 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718963] [<ffffffffa0e48d2e>] ? xfs_iunlock_map_shared+0xe/0x10 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718967] [<ffffffff8118998a>] ? __d_instantiate+0x8a/0x100
Jan 6 22:00:03 skuda kernel: [74688.718971] [<ffffffff81083991>] ? in_group_p+0x31/0x40
Jan 6 22:00:03 skuda kernel: [74688.718974] [<ffffffff8117fa26>] ? generic_permission+0x176/0x280
Jan 6 22:00:03 skuda kernel: [74688.718985] [<ffffffffa0e976ff>] xlog_grant_head_wait+0xaf/0x210 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.718995] [<ffffffffa0e97949>] xlog_grant_head_check+0xe9/0xf0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.719005] [<ffffffffa0e9a795>] xfs_log_reserve+0xc5/0x1f0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.719015] [<ffffffffa0e95c73>] xfs_trans_reserve+0x223/0x230 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.719025] [<ffffffffa0e958b7>] ? _xfs_trans_alloc+0x37/0xa0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.719037] [<ffffffffa0e5691b>] xfs_create+0x1eb/0x690 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.719040] [<ffffffff8117edab>] ? lookup_dcache+0x3b/0xd0
Jan 6 22:00:03 skuda kernel: [74688.719050] [<ffffffffa0e4d386>] xfs_vn_mknod+0xa6/0x1b0 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.719060] [<ffffffffa0e4d4a6>] xfs_vn_mkdir+0x16/0x20 [xfs]
Jan 6 22:00:03 skuda kernel: [74688.719063] [<ffffffff81180c72>] vfs_mkdir+0xc2/0x140
Jan 6 22:00:03 skuda kernel: [74688.719066] [<ffffffff81184793>] sys_mkdirat+0xa3/0xb0
Jan 6 22:00:03 skuda kernel: [74688.719069] [<ffffffff811847b9>] sys_mkdir+0x19/0x20
Jan 6 22:00:03 skuda kernel: [74688.719073] [<ffffffff8149b81d>] system_call_fastpath+0x1a/0x1f
Jan 6 22:02:03 skuda kernel: [74808.603584] xfsaild/md1 D 0000000000000001 0 500 2 0x00000000
Jan 6 22:02:03 skuda kernel: [74808.603589] ffff8803fd7fdaf8 0000000000000046 00000002a0e26770 ffff8803fd7fdfd8
Jan 6 22:02:03 skuda kernel: [74808.603593] ffff8804061d1800 ffff8803fd7fdfd8 ffff8803fd7fdfd8 ffff8803fd7fdfd8
Jan 6 22:02:03 skuda kernel: [74808.603597] ffffffff81814420 ffff880404835100 0000000000000001 ffff880406fcd400
Jan 6 22:02:03 skuda kernel: [74808.603601] Call Trace:
Jan 6 22:02:03 skuda kernel: [74808.603613] [<ffffffffa0e266e5>] ? release_stripe_plug+0x25/0x80 [raid456]
Jan 6 22:02:03 skuda kernel: [74808.603617] [<ffffffffa0e2cf77>] ? make_request+0x2b7/0x6c0 [raid456]
Jan 6 22:02:03 skuda kernel: [74808.603624] [<ffffffff8107bbb0>] ? abort_exclusive_wait+0xb0/0xb0
Jan 6 22:02:03 skuda kernel: [74808.603630] [<ffffffff81490254>] schedule_timeout+0x1f4/0x260
Jan 6 22:02:03 skuda kernel: [74808.603635] [<ffffffff81113ab5>] ? mempool_alloc_slab+0x15/0x20
Jan 6 22:02:03 skuda kernel: [74808.603639] [<ffffffff81491b3b>] wait_for_common+0xcb/0x160
Jan 6 22:02:03 skuda kernel: [74808.603643] [<ffffffff8108b050>] ? try_to_wake_up+0x1f0/0x1f0
Jan 6 22:02:03 skuda kernel: [74808.603647] [<ffffffff81491bed>] wait_for_completion+0x1d/0x20
Jan 6 22:02:03 skuda kernel: [74808.603651] [<ffffffff8107460b>] flush_work+0xeb/0x170
Jan 6 22:02:03 skuda kernel: [74808.603654] [<ffffffff810728d0>] ? cwq_activate_delayed_work+0xc0/0xc0
Jan 6 22:02:03 skuda kernel: [74808.603668] [<ffffffffa0e9b9a9>] xlog_cil_force_lsn+0x39/0x160 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603672] [<ffffffff81067ff8>] ? lock_timer_base.isra.37+0x38/0x70
Jan 6 22:02:03 skuda kernel: [74808.603676] [<ffffffff8106807f>] ? try_to_del_timer_sync+0x4f/0x70
Jan 6 22:02:03 skuda kernel: [74808.603687] [<ffffffffa0e99ce9>] _xfs_log_force+0x69/0x280 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603691] [<ffffffff81490187>] ? schedule_timeout+0x127/0x260
Jan 6 22:02:03 skuda kernel: [74808.603701] [<ffffffffa0e422c0>] ? xfs_buf_delwri_submit_nowait+0x20/0x30 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603711] [<ffffffffa0e99f2a>] xfs_log_force+0x2a/0xc0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603722] [<ffffffffa0e9e814>] xfsaild+0x144/0x700 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603727] [<ffffffff810846ff>] ? __wake_up_common+0x4f/0x80
Jan 6 22:02:03 skuda kernel: [74808.603738] [<ffffffffa0e9e6d0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603742] [<ffffffff8107aec0>] kthread+0xc0/0xd0
Jan 6 22:02:03 skuda kernel: [74808.603747] [<ffffffff81010000>] ? perf_trace_xen_mmu_set_pte_at+0xb0/0x100
Jan 6 22:02:03 skuda kernel: [74808.603751] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
Jan 6 22:02:03 skuda kernel: [74808.603756] [<ffffffff8149b76c>] ret_from_fork+0x7c/0xb0
Jan 6 22:02:03 skuda kernel: [74808.603760] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
Jan 6 22:02:03 skuda kernel: [74808.603804] rsync D 0000000000000001 0 20348 2830 0x00000000
Jan 6 22:02:03 skuda kernel: [74808.603808] ffff8801753afcb8 0000000000000086 ffff8800000000c8 ffff8801753affd8
Jan 6 22:02:03 skuda kernel: [74808.603811] 00000000340643e8 ffff8801753affd8 ffff8801753affd8 ffff8801753affd8
Jan 6 22:02:03 skuda kernel: [74808.603815] ffffffff81814420 ffff8802ad26de80 ffff8801753afce8 ffffffff81497614
Jan 6 22:02:03 skuda kernel: [74808.603818] Call Trace:
Jan 6 22:02:03 skuda kernel: [74808.603822] [<ffffffff81497614>] ? __do_page_fault+0x2d4/0x590
Jan 6 22:02:03 skuda kernel: [74808.603831] [<ffffffffa0e40c35>] ? xfs_buf_get_map+0x35/0x1d0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603840] [<ffffffffa0e41c23>] ? xfs_buf_read_map+0x33/0x140 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603851] [<ffffffffa0e9fc5d>] ? xfs_trans_read_buf_map+0x29d/0x4f0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603862] [<ffffffffa0e976ff>] xlog_grant_head_wait+0xaf/0x210 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603872] [<ffffffffa0e978ee>] xlog_grant_head_check+0x8e/0xf0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603883] [<ffffffffa0e9a795>] xfs_log_reserve+0xc5/0x1f0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603895] [<ffffffffa0e587f7>] ? kmem_zone_alloc+0x67/0xe0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603906] [<ffffffffa0e95c73>] xfs_trans_reserve+0x223/0x230 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603916] [<ffffffffa0e958b7>] ? _xfs_trans_alloc+0x37/0xa0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603927] [<ffffffffa0e4d7eb>] xfs_vn_update_time+0x5b/0x1c0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603931] [<ffffffff81186720>] ? sys_ioctl+0xb0/0xb0
Jan 6 22:02:03 skuda kernel: [74808.603935] [<ffffffff8118e135>] update_time+0x25/0xc0
Jan 6 22:02:03 skuda kernel: [74808.603939] [<ffffffff8118e3b2>] touch_atime+0xf2/0x140
Jan 6 22:02:03 skuda kernel: [74808.603943] [<ffffffff81186ada>] vfs_readdir+0xca/0xe0
Jan 6 22:02:03 skuda kernel: [74808.603946] [<ffffffff81186c0f>] sys_getdents+0x8f/0x110
Jan 6 22:02:03 skuda kernel: [74808.603950] [<ffffffff8149b81d>] system_call_fastpath+0x1a/0x1f
Jan 6 22:02:03 skuda kernel: [74808.603955] rsync D 0000000000000001 0 20349 20348 0x00000000
Jan 6 22:02:03 skuda kernel: [74808.603958] ffff8801c0d71c38 0000000000000086 00000000648bdf17 ffff8801c0d71fd8
Jan 6 22:02:03 skuda kernel: [74808.603962] ffff8800493cec80 ffff8801c0d71fd8 ffff8801c0d71fd8 ffff8801c0d71fd8
Jan 6 22:02:03 skuda kernel: [74808.603965] ffffffff81814420 ffff8802ad269b00 0000000000000000 0000000000000000
Jan 6 22:02:03 skuda kernel: [74808.603969] Call Trace:
Jan 6 22:02:03 skuda kernel: [74808.603979] [<ffffffffa0e48cfc>] ? xfs_iunlock+0x10c/0x130 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603989] [<ffffffffa0e48d2e>] ? xfs_iunlock_map_shared+0xe/0x10 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.603993] [<ffffffff8118998a>] ? __d_instantiate+0x8a/0x100
Jan 6 22:02:03 skuda kernel: [74808.603997] [<ffffffff81083991>] ? in_group_p+0x31/0x40
Jan 6 22:02:03 skuda kernel: [74808.604000] [<ffffffff8117fa26>] ? generic_permission+0x176/0x280
Jan 6 22:02:03 skuda kernel: [74808.604011] [<ffffffffa0e976ff>] xlog_grant_head_wait+0xaf/0x210 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.604020] [<ffffffffa0e97949>] xlog_grant_head_check+0xe9/0xf0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.604031] [<ffffffffa0e9a795>] xfs_log_reserve+0xc5/0x1f0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.604041] [<ffffffffa0e95c73>] xfs_trans_reserve+0x223/0x230 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.604050] [<ffffffffa0e958b7>] ? _xfs_trans_alloc+0x37/0xa0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.604062] [<ffffffffa0e5691b>] xfs_create+0x1eb/0x690 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.604065] [<ffffffff8117edab>] ? lookup_dcache+0x3b/0xd0
Jan 6 22:02:03 skuda kernel: [74808.604075] [<ffffffffa0e4d386>] xfs_vn_mknod+0xa6/0x1b0 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.604085] [<ffffffffa0e4d4a6>] xfs_vn_mkdir+0x16/0x20 [xfs]
Jan 6 22:02:03 skuda kernel: [74808.604088] [<ffffffff81180c72>] vfs_mkdir+0xc2/0x140
Jan 6 22:02:03 skuda kernel: [74808.604091] [<ffffffff81184793>] sys_mkdirat+0xa3/0xb0
Jan 6 22:02:03 skuda kernel: [74808.604094] [<ffffffff811847b9>] sys_mkdir+0x19/0x20
Jan 6 22:02:03 skuda kernel: [74808.604098] [<ffffffff8149b81d>] system_call_fastpath+0x1a/0x1f
Jan 6 22:04:03 skuda kernel: [74928.488601] xfsaild/md1 D 0000000000000001 0 500 2 0x00000000
Jan 6 22:04:03 skuda kernel: [74928.488606] ffff8803fd7fdaf8 0000000000000046 00000002a0e26770 ffff8803fd7fdfd8
Jan 6 22:04:03 skuda kernel: [74928.488611] ffff8804061d1800 ffff8803fd7fdfd8 ffff8803fd7fdfd8 ffff8803fd7fdfd8
Jan 6 22:04:03 skuda kernel: [74928.488614] ffffffff81814420 ffff880404835100 0000000000000001 ffff880406fcd400
Jan 6 22:04:03 skuda kernel: [74928.488618] Call Trace:
Jan 6 22:04:03 skuda kernel: [74928.488630] [<ffffffffa0e266e5>] ? release_stripe_plug+0x25/0x80 [raid456]
Jan 6 22:04:03 skuda kernel: [74928.488635] [<ffffffffa0e2cf77>] ? make_request+0x2b7/0x6c0 [raid456]
Jan 6 22:04:03 skuda kernel: [74928.488642] [<ffffffff8107bbb0>] ? abort_exclusive_wait+0xb0/0xb0
Jan 6 22:04:03 skuda kernel: [74928.488647] [<ffffffff81490254>] schedule_timeout+0x1f4/0x260
Jan 6 22:04:03 skuda kernel: [74928.488652] [<ffffffff81113ab5>] ? mempool_alloc_slab+0x15/0x20
Jan 6 22:04:03 skuda kernel: [74928.488656] [<ffffffff81491b3b>] wait_for_common+0xcb/0x160
Jan 6 22:04:03 skuda kernel: [74928.488661] [<ffffffff8108b050>] ? try_to_wake_up+0x1f0/0x1f0
Jan 6 22:04:03 skuda kernel: [74928.488665] [<ffffffff81491bed>] wait_for_completion+0x1d/0x20
Jan 6 22:04:03 skuda kernel: [74928.488668] [<ffffffff8107460b>] flush_work+0xeb/0x170
Jan 6 22:04:03 skuda kernel: [74928.488671] [<ffffffff810728d0>] ? cwq_activate_delayed_work+0xc0/0xc0
Jan 6 22:04:03 skuda kernel: [74928.488686] [<ffffffffa0e9b9a9>] xlog_cil_force_lsn+0x39/0x160 [xfs]
Jan 6 22:04:03 skuda kernel: [74928.488690] [<ffffffff81067ff8>] ? lock_timer_base.isra.37+0x38/0x70
Jan 6 22:04:03 skuda kernel: [74928.488693] [<ffffffff8106807f>] ? try_to_del_timer_sync+0x4f/0x70
Jan 6 22:04:03 skuda kernel: [74928.488705] [<ffffffffa0e99ce9>] _xfs_log_force+0x69/0x280 [xfs]
Jan 6 22:04:03 skuda kernel: [74928.488709] [<ffffffff81490187>] ? schedule_timeout+0x127/0x260
Jan 6 22:04:03 skuda kernel: [74928.488719] [<ffffffffa0e422c0>] ? xfs_buf_delwri_submit_nowait+0x20/0x30 [xfs]
Jan 6 22:04:03 skuda kernel: [74928.488729] [<ffffffffa0e99f2a>] xfs_log_force+0x2a/0xc0 [xfs]
Jan 6 22:04:03 skuda kernel: [74928.488740] [<ffffffffa0e9e814>] xfsaild+0x144/0x700 [xfs]
Jan 6 22:04:03 skuda kernel: [74928.488745] [<ffffffff810846ff>] ? __wake_up_common+0x4f/0x80
Jan 6 22:04:03 skuda kernel: [74928.488756] [<ffffffffa0e9e6d0>] ? xfs_trans_ail_cursor_first+0x90/0x90 [xfs]
Jan 6 22:04:03 skuda kernel: [74928.488760] [<ffffffff8107aec0>] kthread+0xc0/0xd0
Jan 6 22:04:03 skuda kernel: [74928.488764] [<ffffffff81010000>] ? perf_trace_xen_mmu_set_pte_at+0xb0/0x100
Jan 6 22:04:03 skuda kernel: [74928.488769] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
Jan 6 22:04:03 skuda kernel: [74928.488773] [<ffffffff8149b76c>] ret_from_fork+0x7c/0xb0
Jan 6 22:04:03 skuda kernel: [74928.488778] [<ffffffff8107ae00>] ? kthread_freezable_should_stop+0x70/0x70
_______________________________________________
xfs mailing list
xfs@xxxxxxxxxxx
http://oss.sgi.com/mailman/listinfo/xfs
