Process blocked for more than 120 seconds.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

I have been using rhev 3.1 and created a few VMs. I have provisioning system that boots machines via pxe with centos 6.3 images.

It creates the following:

/dev/vda1 on / type ext3 (rw,noatime,nodiratime)
/dev/vda6 on /local type ext3 (rw,noatime,nodiratime)
/dev/vda3 on /tmp type ext3 (rw,nosuid,nodev,noatime,nodiratime)
/dev/vda2 on /var type ext3 (rw,noatime,nodiratime)

With:  Kernel:  2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Thanks,

Andrew


Dec  7 12:22:01 cheese04 kernel: imklog 4.6.2, log source = /proc/kmsg started.
Dec  7 12:22:01 cheese04 rsyslogd: [origin software="rsyslogd" swVersion="4.6.2" x-pid="2790" x-info="http://www.rsyslog.com";] (re)start
Dec  7 12:29:56 cheese04 kernel: INFO: task kjournald:2520 blocked for more than 120 seconds.
Dec  7 12:29:56 cheese04 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec  7 12:29:56 cheese04 kernel: kjournald     D 0000000000000000     0  2520      2 0x00000080
Dec  7 12:29:56 cheese04 kernel: ffff880101005d50 0000000000000046 ffffffff81a8d020 ffffffff8160b400
Dec  7 12:29:56 cheese04 kernel: 0000000000000000 ffff88011fc23080 ffff880101005da0 ffffffff8105b483
Dec  7 12:29:56 cheese04 kernel: ffff880117f99af8 ffff880101005fd8 000000000000fb88 ffff880117f99af8
Dec  7 12:29:56 cheese04 kernel: Call Trace:
Dec  7 12:29:56 cheese04 kernel: [<ffffffff8105b483>] ? perf_event_task_sched_out+0x33/0x80
Dec  7 12:29:56 cheese04 kernel: [<ffffffff810923be>] ? prepare_to_wait+0x4e/0x80
Dec  7 12:29:56 cheese04 kernel: [<ffffffffa036f751>] journal_commit_transaction+0x161/0x1310 [jbd]
Dec  7 12:29:56 cheese04 kernel: [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
Dec  7 12:29:56 cheese04 kernel: [<ffffffff8107eabb>] ? try_to_del_timer_sync+0x7b/0xe0
Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0375bb8>] kjournald+0xe8/0x250 [jbd]
Dec  7 12:29:56 cheese04 kernel: [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0375ad0>] ? kjournald+0x0/0x250 [jbd]
Dec  7 12:29:56 cheese04 kernel: [<ffffffff81091d66>] kthread+0x96/0xa0
Dec  7 12:29:56 cheese04 kernel: [<ffffffff8100c14a>] child_rip+0xa/0x20
Dec  7 12:29:56 cheese04 kernel: [<ffffffff81091cd0>] ? kthread+0x0/0xa0
Dec  7 12:29:56 cheese04 kernel: [<ffffffff8100c140>] ? child_rip+0x0/0x20
Dec  7 12:29:56 cheese04 kernel: INFO: task master:3023 blocked for more than 120 seconds.
Dec  7 12:29:56 cheese04 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec  7 12:29:56 cheese04 kernel: master        D 0000000000000000     0  3023      1 0x00000084
Dec  7 12:29:56 cheese04 kernel: ffff8801013dd968 0000000000000082 0000000000000000 00000792d63945c4
Dec  7 12:29:56 cheese04 kernel: ffff8801013dd928 ffff880117e21e50 000000000007884e ffffffffaf3e39ec
Dec  7 12:29:56 cheese04 kernel: ffff8800c4282638 ffff8801013ddfd8 000000000000fb88 ffff8800c4282638
Dec  7 12:29:56 cheese04 kernel: Call Trace:
Dec  7 12:29:56 cheese04 kernel: [<ffffffff811adfc0>] ? sync_buffer+0x0/0x50
Dec  7 12:29:56 cheese04 kernel: [<ffffffff814fdfc3>] io_schedule+0x73/0xc0
Dec  7 12:29:56 cheese04 kernel: [<ffffffff811ae000>] sync_buffer+0x40/0x50
Dec  7 12:29:56 cheese04 kernel: [<ffffffff814fe82a>] __wait_on_bit_lock+0x5a/0xc0
Dec  7 12:29:56 cheese04 kernel: [<ffffffff811adfc0>] ? sync_buffer+0x0/0x50
Dec  7 12:29:56 cheese04 kernel: [<ffffffff814fe908>] out_of_line_wait_on_bit_lock+0x78/0x90
Dec  7 12:29:56 cheese04 kernel: [<ffffffff81092110>] ? wake_bit_function+0x0/0x50
Dec  7 12:29:56 cheese04 kernel: [<ffffffff81039678>] ? pvclock_clocksource_read+0x58/0xd0
Dec  7 12:29:56 cheese04 kernel: [<ffffffff811ae1a6>] __lock_buffer+0x36/0x40
Dec  7 12:29:56 cheese04 kernel: [<ffffffffa036f2b3>] do_get_write_access+0x483/0x500 [jbd]
Dec  7 12:29:56 cheese04 kernel: [<ffffffff811ad6bc>] ? __getblk+0x2c/0x2e0
Dec  7 12:29:56 cheese04 kernel: [<ffffffffa036f4c1>] journal_get_write_access+0x31/0x50 [jbd]
Dec  7 12:29:56 cheese04 kernel: [<ffffffffa03ac7dd>] __ext3_journal_get_write_access+0x2d/0x60 [ext3]
Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0392a5b>] ext3_reserve_inode_write+0x7b/0xa0 [ext3]
Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0392ac8>] ext3_mark_inode_dirty+0x48/0xa0 [ext3]
Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0392ca1>] ext3_dirty_inode+0x61/0xa0 [ext3]
Dec  7 12:29:56 cheese04 kernel: [<ffffffff811a49eb>] __mark_inode_dirty+0x3b/0x160
Dec  7 12:29:56 cheese04 kernel: [<ffffffff81194f32>] file_update_time+0xf2/0x170
Dec  7 12:29:56 cheese04 kernel: [<ffffffff81184cd2>] pipe_write+0x2d2/0x650
Dec  7 12:29:56 cheese04 kernel: [<ffffffff8117ad6a>] do_sync_write+0xfa/0x140
Dec  7 12:29:56 cheese04 kernel: [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
Dec  7 12:29:56 cheese04 kernel: [<ffffffff8121fd8b>] ? selinux_file_permission+0xfb/0x150
Dec  7 12:29:56 cheese04 kernel: [<ffffffff81213136>] ? security_file_permission+0x16/0x20
Dec  7 12:29:56 cheese04 kernel: [<ffffffff8117b068>] vfs_write+0xb8/0x1a0
Dec  7 12:29:56 cheese04 kernel: [<ffffffff8117ba81>] sys_write+0x51/0x90
Dec  7 12:29:56 cheese04 kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b

[root@cheese04 ~]# dumpe2fs /dev/vda1 
dumpe2fs 1.41.12 (17-May-2010)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          79efe567-75c2-4fd1-800c-429d0530b945
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags:         signed_directory_hash 
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              1250928
Block count:              5000000
Reserved block count:     250000
Free blocks:              4259450
Free inodes:              1167788
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1022
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8176
Inode blocks per group:   511
Filesystem created:       Fri Dec  7 11:34:58 2012
Last mount time:          Fri Dec  7 11:37:23 2012
Last write time:          Fri Dec  7 11:37:16 2012
Mount count:              2
Maximum mount count:      21
Last checked:             Fri Dec  7 11:34:58 2012
Check interval:           15552000 (6 months)
Next check after:         Wed Jun  5 12:34:58 2013
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	          256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      aa5c6c5f-f16d-46a9-9a62-c55d37053adb
Journal backup:           inode blocks
Journal features:         (none)
Journal size:             128M
Journal length:           32768
Journal sequence:         0x00000015
Journal start:            1


Group 0: (Blocks 0-32767)
  Primary superblock at 0, Group descriptors at 1-2
  Reserved GDT blocks at 3-1024
  Block bitmap at 1025 (+1025), Inode bitmap at 1026 (+1026)
  Inode table at 1027-1537 (+1027)
  31224 free blocks, 8163 free inodes, 2 directories
  Free blocks: 1544-32767
  Free inodes: 13, 15-8176


/usr/libexec/qemu-kvm-S-Mrhel6.3.0-cpuSandyBridge-enable-kvm-m4096-smp4                             
sockets=4                             
cores=1                             
threads=1-namecheese03-uuidd782ef0b-07c6-4950-873d-def2acde8d82-smbiostype=1                             
manufacturer=Red Hat                             
product=RHEV Hypervisor                             
version=6.3-20121121.0.el6_3                             
serial=32333536-3030-5A43-3232-32383046375A_E4:11:5B:62:F1:18                             
uuid=d782ef0b-07c6-4950-873d-def2acde8d82-nodefconfig-nodefaults-chardevsocket                             
id=charmonitor                             
path=/var/lib/libvirt/qemu/cheese03.monitor                             
server                            
nowait-monchardev=charmonitor                             
id=monitor                             
mode=control-rtcbase=2012-12-06T22:33:31                             
driftfix=slew-no-shutdown-devicepiix3-usb-uhci                             
id=usb                             
bus=pci.0                             
addr=0x1.0x2-devicevirtio-serial-pci                             
id=virtio-serial0                             
bus=pci.0                             
addr=0x4-driveif=none                             
media=cdrom                             
id=drive-ide0-1-0                             
readonly=on                             
format=raw                             
serial=-deviceide-drive                            
bus=ide.1                             
unit=0                             
drive=drive-ide0-1-0                             
id=ide0-1-0-drivefile=/rhev/data-center/3ecf6306-3fa6-11e2-b544-00215e253fcc/b2a3daf4-7315-4cd8-a076-4ab005db7410/images/8c3541ea-9b89-4837-b98a-ae97feae6765/c90ceaf1-9c3a-42e6-b1a2-939fc1403fcb
if=none                            
id=drive-virtio-disk0                             
format=raw                             
serial=8c3541ea-9b89-4837-b98a-ae97feae6765                             
cache=none                             
werror=stop                             
rerror=stop                             
aio=threads-devicevirtio-blk-pci                             
scsi=off                             
bus=pci.0                             
addr=0x5                             
drive=drive-virtio-disk0                             
id=virtio-disk0-netdevtap                            
fd=26                             
id=hostnet0                             
vhost=on                             
vhostfd=27-devicevirtio-net-pci                             
netdev=hostnet0                             
id=net0                             
mac=00:00:00:00:03:00                             
bus=pci.0                             
addr=0x3                             
bootindex=1-chardevsocket                             
id=charchannel0                             
path=/var/lib/libvirt/qemu/channels/cheese03.com.redhat.rhevm.vdsm                             
server                             
nowait-devicevirtserialport                             
bus=virtio-serial0.0                             
nr=1                             
chardev=charchannel0                             
id=channel0                             
name=com.redhat.rhevm.vdsm-chardevsocket                             
id=charchannel1                             
path=/var/lib/libvirt/qemu/channels/cheese03.org.qemu.guest_agent.0                             
server                             
nowait-devicevirtserialport                             
bus=virtio-serial0.0                             
nr=2                             
chardev=charchannel1                             
id=channel1                             
name=org.qemu.guest_agent.0-chardevspicevmc                             
id=charchannel2                             
name=vdagent-devicevirtserialport                             
bus=virtio-serial0.0                             
nr=3                             
chardev=charchannel2                             
id=channel2                             
name=com.redhat.spice.0-spiceport=5900                             
tls-port=5901                             
addr=0                             
x509-dir=/etc/pki/vdsm/libvirt-spice                             
tls-channel=main                             
tls-channel=display                             
tls-channel=inputs                             
tls-channel=cursor                             
tls-channel=playback                             
tls-channel=record                             
tls-channel=smartcard                             
tls-channel=usbredir-ken-us-vgaqxl-globalqxl-vga.vram_size=67108864-devicevirtio-balloon-pci                             
id=balloon0                             
bus=pci.0                             
addr=0x6





--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux