Re: Process blocked for more than 120 seconds.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



sorry. I forgot to mention that my images are being created on a Solaris based NFS / ZFS server.

Thanks,

Andrew


On Dec 7, 2012, at 1:09 PM, Andrew Holway wrote:

> Hello,
> 
> I have been using rhev 3.1 and created a few VMs. I have provisioning system that boots machines via pxe with centos 6.3 images.
> 
> It creates the following:
> 
> /dev/vda1 on / type ext3 (rw,noatime,nodiratime)
> /dev/vda6 on /local type ext3 (rw,noatime,nodiratime)
> /dev/vda3 on /tmp type ext3 (rw,nosuid,nodev,noatime,nodiratime)
> /dev/vda2 on /var type ext3 (rw,noatime,nodiratime)
> 
> With:  Kernel:  2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
> 
> Thanks,
> 
> Andrew
> 
> 
> Dec  7 12:22:01 cheese04 kernel: imklog 4.6.2, log source = /proc/kmsg started.
> Dec  7 12:22:01 cheese04 rsyslogd: [origin software="rsyslogd" swVersion="4.6.2" x-pid="2790" x-info="http://www.rsyslog.com";] (re)start
> Dec  7 12:29:56 cheese04 kernel: INFO: task kjournald:2520 blocked for more than 120 seconds.
> Dec  7 12:29:56 cheese04 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> Dec  7 12:29:56 cheese04 kernel: kjournald     D 0000000000000000     0  2520      2 0x00000080
> Dec  7 12:29:56 cheese04 kernel: ffff880101005d50 0000000000000046 ffffffff81a8d020 ffffffff8160b400
> Dec  7 12:29:56 cheese04 kernel: 0000000000000000 ffff88011fc23080 ffff880101005da0 ffffffff8105b483
> Dec  7 12:29:56 cheese04 kernel: ffff880117f99af8 ffff880101005fd8 000000000000fb88 ffff880117f99af8
> Dec  7 12:29:56 cheese04 kernel: Call Trace:
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff8105b483>] ? perf_event_task_sched_out+0x33/0x80
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff810923be>] ? prepare_to_wait+0x4e/0x80
> Dec  7 12:29:56 cheese04 kernel: [<ffffffffa036f751>] journal_commit_transaction+0x161/0x1310 [jbd]
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff8107eabb>] ? try_to_del_timer_sync+0x7b/0xe0
> Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0375bb8>] kjournald+0xe8/0x250 [jbd]
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
> Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0375ad0>] ? kjournald+0x0/0x250 [jbd]
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff81091d66>] kthread+0x96/0xa0
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff8100c14a>] child_rip+0xa/0x20
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff81091cd0>] ? kthread+0x0/0xa0
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff8100c140>] ? child_rip+0x0/0x20
> Dec  7 12:29:56 cheese04 kernel: INFO: task master:3023 blocked for more than 120 seconds.
> Dec  7 12:29:56 cheese04 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> Dec  7 12:29:56 cheese04 kernel: master        D 0000000000000000     0  3023      1 0x00000084
> Dec  7 12:29:56 cheese04 kernel: ffff8801013dd968 0000000000000082 0000000000000000 00000792d63945c4
> Dec  7 12:29:56 cheese04 kernel: ffff8801013dd928 ffff880117e21e50 000000000007884e ffffffffaf3e39ec
> Dec  7 12:29:56 cheese04 kernel: ffff8800c4282638 ffff8801013ddfd8 000000000000fb88 ffff8800c4282638
> Dec  7 12:29:56 cheese04 kernel: Call Trace:
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff811adfc0>] ? sync_buffer+0x0/0x50
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff814fdfc3>] io_schedule+0x73/0xc0
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff811ae000>] sync_buffer+0x40/0x50
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff814fe82a>] __wait_on_bit_lock+0x5a/0xc0
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff811adfc0>] ? sync_buffer+0x0/0x50
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff814fe908>] out_of_line_wait_on_bit_lock+0x78/0x90
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff81092110>] ? wake_bit_function+0x0/0x50
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff81039678>] ? pvclock_clocksource_read+0x58/0xd0
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff811ae1a6>] __lock_buffer+0x36/0x40
> Dec  7 12:29:56 cheese04 kernel: [<ffffffffa036f2b3>] do_get_write_access+0x483/0x500 [jbd]
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff811ad6bc>] ? __getblk+0x2c/0x2e0
> Dec  7 12:29:56 cheese04 kernel: [<ffffffffa036f4c1>] journal_get_write_access+0x31/0x50 [jbd]
> Dec  7 12:29:56 cheese04 kernel: [<ffffffffa03ac7dd>] __ext3_journal_get_write_access+0x2d/0x60 [ext3]
> Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0392a5b>] ext3_reserve_inode_write+0x7b/0xa0 [ext3]
> Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0392ac8>] ext3_mark_inode_dirty+0x48/0xa0 [ext3]
> Dec  7 12:29:56 cheese04 kernel: [<ffffffffa0392ca1>] ext3_dirty_inode+0x61/0xa0 [ext3]
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff811a49eb>] __mark_inode_dirty+0x3b/0x160
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff81194f32>] file_update_time+0xf2/0x170
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff81184cd2>] pipe_write+0x2d2/0x650
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff8117ad6a>] do_sync_write+0xfa/0x140
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff810920d0>] ? autoremove_wake_function+0x0/0x40
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff8121fd8b>] ? selinux_file_permission+0xfb/0x150
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff81213136>] ? security_file_permission+0x16/0x20
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff8117b068>] vfs_write+0xb8/0x1a0
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff8117ba81>] sys_write+0x51/0x90
> Dec  7 12:29:56 cheese04 kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
> 
> [root@cheese04 ~]# dumpe2fs /dev/vda1 
> dumpe2fs 1.41.12 (17-May-2010)
> Filesystem volume name:   <none>
> Last mounted on:          <not available>
> Filesystem UUID:          79efe567-75c2-4fd1-800c-429d0530b945
> Filesystem magic number:  0xEF53
> Filesystem revision #:    1 (dynamic)
> Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
> Filesystem flags:         signed_directory_hash 
> Default mount options:    (none)
> Filesystem state:         clean
> Errors behavior:          Continue
> Filesystem OS type:       Linux
> Inode count:              1250928
> Block count:              5000000
> Reserved block count:     250000
> Free blocks:              4259450
> Free inodes:              1167788
> First block:              0
> Block size:               4096
> Fragment size:            4096
> Reserved GDT blocks:      1022
> Blocks per group:         32768
> Fragments per group:      32768
> Inodes per group:         8176
> Inode blocks per group:   511
> Filesystem created:       Fri Dec  7 11:34:58 2012
> Last mount time:          Fri Dec  7 11:37:23 2012
> Last write time:          Fri Dec  7 11:37:16 2012
> Mount count:              2
> Maximum mount count:      21
> Last checked:             Fri Dec  7 11:34:58 2012
> Check interval:           15552000 (6 months)
> Next check after:         Wed Jun  5 12:34:58 2013
> Reserved blocks uid:      0 (user root)
> Reserved blocks gid:      0 (group root)
> First inode:              11
> Inode size:	          256
> Required extra isize:     28
> Desired extra isize:      28
> Journal inode:            8
> Default directory hash:   half_md4
> Directory Hash Seed:      aa5c6c5f-f16d-46a9-9a62-c55d37053adb
> Journal backup:           inode blocks
> Journal features:         (none)
> Journal size:             128M
> Journal length:           32768
> Journal sequence:         0x00000015
> Journal start:            1
> 
> 
> Group 0: (Blocks 0-32767)
>  Primary superblock at 0, Group descriptors at 1-2
>  Reserved GDT blocks at 3-1024
>  Block bitmap at 1025 (+1025), Inode bitmap at 1026 (+1026)
>  Inode table at 1027-1537 (+1027)
>  31224 free blocks, 8163 free inodes, 2 directories
>  Free blocks: 1544-32767
>  Free inodes: 13, 15-8176
> 
> 
> /usr/libexec/qemu-kvm-S-Mrhel6.3.0-cpuSandyBridge-enable-kvm-m4096-smp4                             
> sockets=4                             
> cores=1                             
> threads=1-namecheese03-uuidd782ef0b-07c6-4950-873d-def2acde8d82-smbiostype=1                             
> manufacturer=Red Hat                             
> product=RHEV Hypervisor                             
> version=6.3-20121121.0.el6_3                             
> serial=32333536-3030-5A43-3232-32383046375A_E4:11:5B:62:F1:18                             
> uuid=d782ef0b-07c6-4950-873d-def2acde8d82-nodefconfig-nodefaults-chardevsocket                             
> id=charmonitor                             
> path=/var/lib/libvirt/qemu/cheese03.monitor                             
> server                            
> nowait-monchardev=charmonitor                             
> id=monitor                             
> mode=control-rtcbase=2012-12-06T22:33:31                             
> driftfix=slew-no-shutdown-devicepiix3-usb-uhci                             
> id=usb                             
> bus=pci.0                             
> addr=0x1.0x2-devicevirtio-serial-pci                             
> id=virtio-serial0                             
> bus=pci.0                             
> addr=0x4-driveif=none                             
> media=cdrom                             
> id=drive-ide0-1-0                             
> readonly=on                             
> format=raw                             
> serial=-deviceide-drive                            
> bus=ide.1                             
> unit=0                             
> drive=drive-ide0-1-0                             
> id=ide0-1-0-drivefile=/rhev/data-center/3ecf6306-3fa6-11e2-b544-00215e253fcc/b2a3daf4-7315-4cd8-a076-4ab005db7410/images/8c3541ea-9b89-4837-b98a-ae97feae6765/c90ceaf1-9c3a-42e6-b1a2-939fc1403fcb
> if=none                            
> id=drive-virtio-disk0                             
> format=raw                             
> serial=8c3541ea-9b89-4837-b98a-ae97feae6765                             
> cache=none                             
> werror=stop                             
> rerror=stop                             
> aio=threads-devicevirtio-blk-pci                             
> scsi=off                             
> bus=pci.0                             
> addr=0x5                             
> drive=drive-virtio-disk0                             
> id=virtio-disk0-netdevtap                            
> fd=26                             
> id=hostnet0                             
> vhost=on                             
> vhostfd=27-devicevirtio-net-pci                             
> netdev=hostnet0                             
> id=net0                             
> mac=00:00:00:00:03:00                             
> bus=pci.0                             
> addr=0x3                             
> bootindex=1-chardevsocket                             
> id=charchannel0                             
> path=/var/lib/libvirt/qemu/channels/cheese03.com.redhat.rhevm.vdsm                             
> server                             
> nowait-devicevirtserialport                             
> bus=virtio-serial0.0                             
> nr=1                             
> chardev=charchannel0                             
> id=channel0                             
> name=com.redhat.rhevm.vdsm-chardevsocket                             
> id=charchannel1                             
> path=/var/lib/libvirt/qemu/channels/cheese03.org.qemu.guest_agent.0                             
> server                             
> nowait-devicevirtserialport                             
> bus=virtio-serial0.0                             
> nr=2                             
> chardev=charchannel1                             
> id=channel1                             
> name=org.qemu.guest_agent.0-chardevspicevmc                             
> id=charchannel2                             
> name=vdagent-devicevirtserialport                             
> bus=virtio-serial0.0                             
> nr=3                             
> chardev=charchannel2                             
> id=channel2                             
> name=com.redhat.spice.0-spiceport=5900                             
> tls-port=5901                             
> addr=0                             
> x509-dir=/etc/pki/vdsm/libvirt-spice                             
> tls-channel=main                             
> tls-channel=display                             
> tls-channel=inputs                             
> tls-channel=cursor                             
> tls-channel=playback                             
> tls-channel=record                             
> tls-channel=smartcard                             
> tls-channel=usbredir-ken-us-vgaqxl-globalqxl-vga.vram_size=67108864-devicevirtio-balloon-pci                             
> id=balloon0                             
> bus=pci.0                             
> addr=0x6
> 
> 
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux