Re: [REGRESSION][v6.8-rc1] virtio-pci: Introduce admin virtqueue

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 2024-05-08 a.m.7:18, Catherine Redfield wrote:
*External email: Use caution opening links or attachments*


On a VM with the GCP kernel (where we first identified the problem), I see:

1. The full kernel log from `journalctl --system > kernlog` attached. The specific suspend section is here:

May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal systemd[1]: Reached target sleep.target - Sleep. May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal systemd[1]: Starting systemd-suspend.service - System Suspend... May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal systemd-sleep[1413]: Performing sleep operation 'suspend'... May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: PM: suspend entry (deep) May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: Filesystems sync: 0.008 seconds May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: Freezing user space processes May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: Freezing user space processes completed (elapsed 0.001 seconds) May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: OOM killer disabled. May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: Freezing remaining freezable tasks May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: Freezing remaining freezable tasks completed (elapsed 0.000 seconds) May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: printk: Suspending console(s) (use no_console_suspend to debug) May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: port 00:03:0.0: PM: dpm_run_callback(): pm_runtime_force_suspend+0x0/0x130 returns -16 May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: port 00:03:0.0: PM: failed to suspend: error -16

Thanks Joesph and Catherine's help.

Hi,

I have alreay synced up with Cananical guys offline about this issue.

I can run "suspend/resume" sucessfully on my local server and VM.
And "PM: failed to suspend: error -16" looks like not cause by my previous virtio patch ( fd27ef6b44be ("virtio-pci: Introduce admin virtqueue")) which only modified "virtio_device_freeze" about "suspend" action.

So I have provide the my steps and debug patch to Joesph and Catherine. I will also sync up the information here, as follow:

I have read the qemu code and find a way to trigger "suspend/resume" on my setup, and add some debug message in the latest kerenel

My setps are:
1. QEMU cmdline add following
....
-global PIIX4_PM.disable_s3=0 \
-global PIIX4_PM.disable_s4=1 \
....
-netdev type=tap,ifname=tap0,id=hostnet0,script=no,downscript=no \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=$SSH_MAC,bus=pci.0,addr=0x3 \
......

2. In the VM, run "systemctl suspend" to PM suspend the VM into memory
3. In qemu hmp shell, run "system_wakeup" to resume the VM again

My VM configuration:
NIC:     1 virtio nic emulated by QEMU
OS:      Ubuntu 22.04.4 LTS
kernel: latest kernel, 6.9-rc7: ee5b455b0ada (kernel2/net-next-virito, kernel2/master, master) Merge tag 'slab-for-6.9-rc7-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab)


I add some debug message on the latest kernel, and do above steps to trigger "suspen/resume". Everything of VM is OK, VM could suspend/resume successfully.
Follwing is the kernel log:
----------------------------------------------------------------------------
........
May  6 15:59:52 feliu-vm kernel: [   43.446737] PM: suspend entry (deep)
May 6 16:00:04 feliu-vm kernel: [ 43.467640] Filesystems sync: 0.020 seconds May 6 16:00:04 feliu-vm kernel: [ 43.467923] Freezing user space processes May 6 16:00:04 feliu-vm kernel: [ 43.470294] Freezing user space processes completed (elapsed 0.002 seconds)
May  6 16:00:04 feliu-vm kernel: [   43.470299] OOM killer disabled.
May 6 16:00:04 feliu-vm kernel: [ 43.470301] Freezing remaining freezable tasks May 6 16:00:04 feliu-vm kernel: [ 43.471482] Freezing remaining freezable tasks completed (elapsed 0.001 seconds) May 6 16:00:04 feliu-vm kernel: [ 43.471495] printk: Suspending console(s) (use no_console_suspend to debug) May 6 16:00:04 feliu-vm kernel: [ 43.474034] virtio_net virtio0: godeng virtio device freeze May 6 16:00:04 feliu-vm kernel: [ 43.475714] virtio_net virtio0 ens3: godfeng virtnet_freeze done May 6 16:00:04 feliu-vm kernel: [ 43.475717] virtio_net virtio0: godfeng VIRTIO_F_ADMIN_VQ not enabled May 6 16:00:04 feliu-vm kernel: [ 43.475719] virtio_net virtio0: godeng virtio device freeze done
........
May 6 16:00:04 feliu-vm kernel: [ 43.535382] smpboot: CPU 1 is now offline May 6 16:00:04 feliu-vm kernel: [ 43.537283] IRQ fixup: irq 1 move in progress, old vector 32 May 6 16:00:04 feliu-vm kernel: [ 43.538504] smpboot: CPU 2 is now offline May 6 16:00:04 feliu-vm kernel: [ 43.541392] smpboot: CPU 3 is now offline

......

May 6 16:00:04 feliu-vm kernel: [ 54.973285] smpboot: Booting Node 0 Processor 15 APIC 0xf
May  6 16:00:04 feliu-vm kernel: [   54.975190] CPU15 is up
May 6 16:00:04 feliu-vm kernel: [ 54.976011] ACPI: PM: Waking up from system sleep state S3 May 6 16:00:04 feliu-vm kernel: [ 54.986071] virtio_net virtio0: godeng virtio device restore May 6 16:00:04 feliu-vm kernel: [ 54.987563] virtio_net virtio0 ens3: godfeng virtnet_restore done May 6 16:00:04 feliu-vm kernel: [ 54.987635] virtio_net virtio0: godfeng: virtio device restore done
......
May 6 16:00:04 feliu-vm kernel: [ 55.307221] ata8: SATA link down (SStatus 0 SControl 300)
May  6 16:00:04 feliu-vm kernel: [   55.442048] OOM killer enabled.
May  6 16:00:04 feliu-vm kernel: [   55.442051] Restarting tasks ... done.
May 6 16:00:04 feliu-vm kernel: [ 55.443576] random: crng reseeded on system resumption
May  6 16:00:04 feliu-vm kernel: [   55.443582] PM: suspend exit

----------------------------------------------------------------------------

Attachment is the full kernel log. I think maybe it is some configration error.


Thanks
Feng


May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: sd 0:0:1:0: [sda] Synchronizing SCSI cache May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: PM: Some devices failed to suspend, or early wake event detected May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: OOM killer enabled. May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: Restarting tasks ... done. May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: random: crng reseeded on system resumption May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: PM: suspend exit May 08 11:08:42 kernel-test-202405080702.c.ubuntu-catred.internal kernel: PM: suspend entry (s2idle)
-- Boot 61828bc938b44fc68a8aeedc16a23a9d --
May 08 11:09:03 localhost kernel: Linux version 6.8.0-1007-gcp (buildd@lcy02-amd64-079) (x86_64-linux-gnu-gcc-13 (Ubuntu 13.2.0-23ubuntu4) 13.2.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #7-Ubuntu SMP Sat Apr 20 00:58:31 UTC 2024 (Ubuntu 6.8.0-1007.7-gcp 6.8.1) May 08 11:09:03 localhost kernel: Command line: BOOT_IMAGE=/vmlinuz-6.8.0-1007-gcp root=PARTUUID=7a949935-6bf2-4cae-b404-803c95163572 ro console=ttyS0,115200 panic=-1

2. The features the devices has:

catred@kernel-test-202405080702:~$ cat /sys/bus/virtio/devices/virtio0/features
0110000000000000000000000000010000000000000000000000000000000000
catred@kernel-test-202405080702:~$ cat /sys/bus/virtio/devices/virtio1/features
1110010110011001110000100000010000000000000000000000000000000000
catred@kernel-test-202405080702:~$ cat /sys/bus/virtio/devices/virtio2/features
1110000000000000000000000000000000000000000000000000000000000000
catred@kernel-test-202405080702:~$ cat /sys/bus/virtio/devices/virtio3/features
0000000000000000000000000000000000000000000000000000000000000000

Catherine

On Tue, May 7, 2024 at 11:34 PM Jason Wang <jasowang@xxxxxxxxxx <mailto:jasowang@xxxxxxxxxx>> wrote:

    On Sat, May 4, 2024 at 2:10 AM Joseph Salisbury
    <joseph.salisbury@xxxxxxxxxxxxx
    <mailto:joseph.salisbury@xxxxxxxxxxxxx>> wrote:
     >
     > Hi Feng,
     >
     > During testing, a kernel bug was identified with the suspend/resume
     > functionality on instances running in a public cloud [0].  This
    bug is a
     > regression introduced in v6.8-rc1.  After a kernel bisect, the
    following
     > commit was identified as the cause of the regression:
     >
     >         fd27ef6b44be  ("virtio-pci: Introduce admin virtqueue")

    Have a quick glance at the patch it seems it should not damage the
    freeze/restore as it should behave as in the past.

    But I found something interesting:

    1) assumes 1 admin vq which is not what spec said
    2) special function for admin virtqueue during freeze/restore, but it
    doesn't do anything special than del_vq()
    3) lack real users but I guess e.g the destroy_avq() needs to be
    synchronized with the one that is using admin virtqueue

     >
     > I was hoping to get your feedback, since you are the patch author. Do
     > you think gathering any additional data will help diagnose this
    issue?

    Yes, please show us

    1) the kernel log here.
    2) the features that the device has like
    /sys/bus/virtio/devices/virtio0/features

     > This commit is depended upon by other virtio commits, so a revert
    test
     > is not really straight forward without reverting all the
    dependencies.
     > Any ideas you have would be greatly appreciated.

    Thanks

     >
     >
     > Thanks,
     >
     > Joe
     >
     > http://pad.lv/2063315 <http://pad.lv/2063315>
     >
May  6 15:59:52 feliu-vm kernel: [   43.446737] PM: suspend entry (deep)
May  6 16:00:04 feliu-vm kernel: [   43.467640] Filesystems sync: 0.020 seconds
May  6 16:00:04 feliu-vm kernel: [   43.467923] Freezing user space processes
May  6 16:00:04 feliu-vm kernel: [   43.470294] Freezing user space processes completed (elapsed 0.002 seconds)
May  6 16:00:04 feliu-vm kernel: [   43.470299] OOM killer disabled.
May  6 16:00:04 feliu-vm kernel: [   43.470301] Freezing remaining freezable tasks
May  6 16:00:04 feliu-vm kernel: [   43.471482] Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
May  6 16:00:04 feliu-vm kernel: [   43.471495] printk: Suspending console(s) (use no_console_suspend to debug)
May  6 16:00:04 feliu-vm kernel: [   43.474034] virtio_net virtio0: godeng virtio device freeze
May  6 16:00:04 feliu-vm kernel: [   43.475714] virtio_net virtio0 ens3: godfeng virtnet_freeze done
May  6 16:00:04 feliu-vm kernel: [   43.475717] virtio_net virtio0: godfeng VIRTIO_F_ADMIN_VQ not enabled
May  6 16:00:04 feliu-vm kernel: [   43.475719] virtio_net virtio0: godeng virtio device freeze done
May  6 16:00:04 feliu-vm kernel: [   43.528538] sd 3:0:0:0: [sdb] Synchronizing SCSI cache
May  6 16:00:04 feliu-vm kernel: [   43.528599] sd 2:0:0:0: [sda] Synchronizing SCSI cache
May  6 16:00:04 feliu-vm kernel: [   43.529288] ata4.00: Entering standby power mode
May  6 16:00:04 feliu-vm kernel: [   43.529364] ata3.00: Entering standby power mode
May  6 16:00:04 feliu-vm kernel: [   43.532432] ACPI: PM: Preparing to enter system sleep state S3
May  6 16:00:04 feliu-vm kernel: [   43.532534] ACPI: PM: Saving platform NVS memory
May  6 16:00:04 feliu-vm kernel: [   43.532570] Disabling non-boot CPUs ...
May  6 16:00:04 feliu-vm kernel: [   43.535382] smpboot: CPU 1 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.537283] IRQ fixup: irq 1 move in progress, old vector 32
May  6 16:00:04 feliu-vm kernel: [   43.538504] smpboot: CPU 2 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.541392] smpboot: CPU 3 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.544464] smpboot: CPU 4 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.547296] smpboot: CPU 5 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.550076] smpboot: CPU 6 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.552660] smpboot: CPU 7 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.555278] smpboot: CPU 8 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.557693] smpboot: CPU 9 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.560292] smpboot: CPU 10 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.562624] smpboot: CPU 11 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.565185] smpboot: CPU 12 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.567496] smpboot: CPU 13 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.570153] smpboot: CPU 14 is now offline
May  6 16:00:04 feliu-vm kernel: [   43.572965] smpboot: CPU 15 is now offline
May  6 16:00:04 feliu-vm kernel: [   54.947312] unchecked MSR access error: WRMSR to 0xc0011029 (tried to write 0x0000000000000000) at rIP: 0xffffffffac2b4c48 (native_write_msr+0x8/0x30)
May  6 16:00:04 feliu-vm kernel: [   54.947333] Call Trace:
May  6 16:00:04 feliu-vm kernel: [   54.947337]  <TASK>
May  6 16:00:04 feliu-vm kernel: [   54.947349]  ? show_stack_regs+0x26/0x30
May  6 16:00:04 feliu-vm kernel: [   54.947355]  ? ex_handler_msr+0x10f/0x180
May  6 16:00:04 feliu-vm kernel: [   54.947359]  ? search_extable+0x2b/0x40
May  6 16:00:04 feliu-vm kernel: [   54.947365]  ? fixup_exception+0x315/0x380
May  6 16:00:04 feliu-vm kernel: [   54.947369]  ? exc_general_protection+0x152/0x480
May  6 16:00:04 feliu-vm kernel: [   54.947373]  ? acpi_hw_read_port+0xec/0x100
May  6 16:00:04 feliu-vm kernel: [   54.947382]  ? asm_exc_general_protection+0x2b/0x30
May  6 16:00:04 feliu-vm kernel: [   54.947397]  ? native_write_msr+0x8/0x30
May  6 16:00:04 feliu-vm kernel: [   54.947401]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947405]  ? restore_processor_state+0x2bf/0x350
May  6 16:00:04 feliu-vm kernel: [   54.947411]  x86_acpi_suspend_lowlevel+0xf7/0x160
May  6 16:00:04 feliu-vm kernel: [   54.947416]  acpi_suspend_enter+0x5a/0x1f0
May  6 16:00:04 feliu-vm kernel: [   54.947421]  ? _raw_spin_unlock_irqrestore+0x12/0x40
May  6 16:00:04 feliu-vm kernel: [   54.947425]  suspend_devices_and_enter+0x759/0x8e0
May  6 16:00:04 feliu-vm kernel: [   54.947432]  pm_suspend+0x23e/0x5e0
May  6 16:00:04 feliu-vm kernel: [   54.947435]  state_store+0x85/0xf0
May  6 16:00:04 feliu-vm kernel: [   54.947440]  kobj_attr_store+0x16/0x30
May  6 16:00:04 feliu-vm kernel: [   54.947444]  sysfs_kf_write+0x42/0x50
May  6 16:00:04 feliu-vm kernel: [   54.947448]  kernfs_fop_write_iter+0x143/0x1d0
May  6 16:00:04 feliu-vm kernel: [   54.947453]  vfs_write+0x311/0x430
May  6 16:00:04 feliu-vm kernel: [   54.947461]  ksys_write+0x6b/0xf0
May  6 16:00:04 feliu-vm kernel: [   54.947466]  __x64_sys_write+0x1d/0x30
May  6 16:00:04 feliu-vm kernel: [   54.947469]  x64_sys_call+0x1681/0x20c0
May  6 16:00:04 feliu-vm kernel: [   54.947474]  do_syscall_64+0x71/0x110
May  6 16:00:04 feliu-vm kernel: [   54.947478]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947482]  ? kmem_cache_free+0x3bb/0x3f0
May  6 16:00:04 feliu-vm kernel: [   54.947486]  ? __fput+0x16d/0x2f0
May  6 16:00:04 feliu-vm kernel: [   54.947491]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947494]  ? __fput+0x16d/0x2f0
May  6 16:00:04 feliu-vm kernel: [   54.947498]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947502]  ? __fput_sync+0x1e/0x30
May  6 16:00:04 feliu-vm kernel: [   54.947505]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947508]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947511]  ? syscall_exit_to_user_mode+0x92/0x1c0
May  6 16:00:04 feliu-vm kernel: [   54.947515]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947518]  ? do_syscall_64+0x7d/0x110
May  6 16:00:04 feliu-vm kernel: [   54.947521]  ? set_ptes.constprop.0+0x2b/0x90
May  6 16:00:04 feliu-vm kernel: [   54.947526]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947529]  ? next_uptodate_folio+0x9f/0x310
May  6 16:00:04 feliu-vm kernel: [   54.947534]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947537]  ? _raw_spin_unlock+0x12/0x30
May  6 16:00:04 feliu-vm kernel: [   54.947540]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947543]  ? filemap_map_pages+0x4c3/0x5c0
May  6 16:00:04 feliu-vm kernel: [   54.947547]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947550]  ? __pte_offset_map+0x20/0x190
May  6 16:00:04 feliu-vm kernel: [   54.947557]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947560]  ? do_fault+0x252/0x4a0
May  6 16:00:04 feliu-vm kernel: [   54.947564]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947567]  ? __handle_mm_fault+0x8b0/0xeb0
May  6 16:00:04 feliu-vm kernel: [   54.947570]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947576]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947579]  ? __task_pid_nr_ns+0x70/0xc0
May  6 16:00:04 feliu-vm kernel: [   54.947583]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947586]  ? syscall_exit_to_user_mode+0x92/0x1c0
May  6 16:00:04 feliu-vm kernel: [   54.947590]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947593]  ? do_syscall_64+0x7d/0x110
May  6 16:00:04 feliu-vm kernel: [   54.947596]  ? handle_mm_fault+0x190/0x2f0
May  6 16:00:04 feliu-vm kernel: [   54.947600]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947603]  ? do_user_addr_fault+0x357/0x6b0
May  6 16:00:04 feliu-vm kernel: [   54.947606]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947609]  ? irqentry_exit_to_user_mode+0x63/0x180
May  6 16:00:04 feliu-vm kernel: [   54.947613]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947616]  ? irqentry_exit+0x3f/0x50
May  6 16:00:04 feliu-vm kernel: [   54.947620]  ? srso_return_thunk+0x5/0x5f
May  6 16:00:04 feliu-vm kernel: [   54.947623]  ? exc_page_fault+0x8d/0x190
May  6 16:00:04 feliu-vm kernel: [   54.947627]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
May  6 16:00:04 feliu-vm kernel: [   54.947631] RIP: 0033:0x7f0ebcd14887
May  6 16:00:04 feliu-vm kernel: [   54.947641] Code: 10 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
May  6 16:00:04 feliu-vm kernel: [   54.947644] RSP: 002b:00007ffee36a71d8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
May  6 16:00:04 feliu-vm kernel: [   54.947647] RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 00007f0ebcd14887
May  6 16:00:04 feliu-vm kernel: [   54.947649] RDX: 0000000000000004 RSI: 00007ffee36a7290 RDI: 0000000000000004
May  6 16:00:04 feliu-vm kernel: [   54.947651] RBP: 00007ffee36a7290 R08: 0000000000000004 R09: 000000007fffffff
May  6 16:00:04 feliu-vm kernel: [   54.947653] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000004
May  6 16:00:04 feliu-vm kernel: [   54.947654] R13: 000055f0fdddc2d0 R14: 00007f0ebce16a00 R15: 0000000000000004
May  6 16:00:04 feliu-vm kernel: [   54.947661]  </TASK>
May  6 16:00:04 feliu-vm kernel: [   54.947665] ACPI: PM: Low-level resume complete
May  6 16:00:04 feliu-vm kernel: [   54.947715] ACPI: PM: Restoring platform NVS memory
May  6 16:00:04 feliu-vm kernel: [   54.949557] Enabling non-boot CPUs ...
May  6 16:00:04 feliu-vm kernel: [   54.949649] smpboot: Booting Node 0 Processor 1 APIC 0x1
May  6 16:00:04 feliu-vm kernel: [   54.951003] CPU1 is up
May  6 16:00:04 feliu-vm kernel: [   54.951050] smpboot: Booting Node 0 Processor 2 APIC 0x2
May  6 16:00:04 feliu-vm kernel: [   54.953425] CPU2 is up
May  6 16:00:04 feliu-vm kernel: [   54.953478] smpboot: Booting Node 0 Processor 3 APIC 0x3
May  6 16:00:04 feliu-vm kernel: [   54.954602] CPU3 is up
May  6 16:00:04 feliu-vm kernel: [   54.954654] smpboot: Booting Node 0 Processor 4 APIC 0x4
May  6 16:00:04 feliu-vm kernel: [   54.956254] CPU4 is up
May  6 16:00:04 feliu-vm kernel: [   54.956309] smpboot: Booting Node 0 Processor 5 APIC 0x5
May  6 16:00:04 feliu-vm kernel: [   54.957780] CPU5 is up
May  6 16:00:04 feliu-vm kernel: [   54.957835] smpboot: Booting Node 0 Processor 6 APIC 0x6
May  6 16:00:04 feliu-vm kernel: [   54.959263] CPU6 is up
May  6 16:00:04 feliu-vm kernel: [   54.959312] smpboot: Booting Node 0 Processor 7 APIC 0x7
May  6 16:00:04 feliu-vm kernel: [   54.960734] CPU7 is up
May  6 16:00:04 feliu-vm kernel: [   54.960788] smpboot: Booting Node 0 Processor 8 APIC 0x8
May  6 16:00:04 feliu-vm kernel: [   54.962273] CPU8 is up
May  6 16:00:04 feliu-vm kernel: [   54.962328] smpboot: Booting Node 0 Processor 9 APIC 0x9
May  6 16:00:04 feliu-vm kernel: [   54.963692] CPU9 is up
May  6 16:00:04 feliu-vm kernel: [   54.963739] smpboot: Booting Node 0 Processor 10 APIC 0xa
May  6 16:00:04 feliu-vm kernel: [   54.965708] CPU10 is up
May  6 16:00:04 feliu-vm kernel: [   54.965769] smpboot: Booting Node 0 Processor 11 APIC 0xb
May  6 16:00:04 feliu-vm kernel: [   54.967317] CPU11 is up
May  6 16:00:04 feliu-vm kernel: [   54.967352] smpboot: Booting Node 0 Processor 12 APIC 0xc
May  6 16:00:04 feliu-vm kernel: [   54.968943] CPU12 is up
May  6 16:00:04 feliu-vm kernel: [   54.968991] smpboot: Booting Node 0 Processor 13 APIC 0xd
May  6 16:00:04 feliu-vm kernel: [   54.970551] CPU13 is up
May  6 16:00:04 feliu-vm kernel: [   54.970602] smpboot: Booting Node 0 Processor 14 APIC 0xe
May  6 16:00:04 feliu-vm kernel: [   54.973225] CPU14 is up
May  6 16:00:04 feliu-vm kernel: [   54.973285] smpboot: Booting Node 0 Processor 15 APIC 0xf
May  6 16:00:04 feliu-vm kernel: [   54.975190] CPU15 is up
May  6 16:00:04 feliu-vm kernel: [   54.976011] ACPI: PM: Waking up from system sleep state S3
May  6 16:00:04 feliu-vm kernel: [   54.986071] virtio_net virtio0: godeng virtio device restore
May  6 16:00:04 feliu-vm kernel: [   54.987563] virtio_net virtio0 ens3: godfeng virtnet_restore done
May  6 16:00:04 feliu-vm kernel: [   54.987635] virtio_net virtio0: godfeng: virtio device restore done
May  6 16:00:04 feliu-vm kernel: [   54.988137] usb usb1: root hub lost power or was reset
May  6 16:00:04 feliu-vm kernel: [   54.988640] pci 0000:00:01.0: PIIX3: Enabling Passive Release
May  6 16:00:04 feliu-vm kernel: [   54.989846] usb usb3: root hub lost power or was reset
May  6 16:00:04 feliu-vm kernel: [   54.989908] usb usb4: root hub lost power or was reset
May  6 16:00:04 feliu-vm kernel: [   54.989961] usb usb2: root hub lost power or was reset
May  6 16:00:04 feliu-vm kernel: [   55.272222] usb 2-1: reset high-speed USB device number 2 using ehci-pci
May  6 16:00:04 feliu-vm kernel: [   55.302948] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
May  6 16:00:04 feliu-vm kernel: [   55.303093] ata5: SATA link down (SStatus 0 SControl 300)
May  6 16:00:04 feliu-vm kernel: [   55.303266] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
May  6 16:00:04 feliu-vm kernel: [   55.303367] sd 3:0:0:0: [sdb] Starting disk
May  6 16:00:04 feliu-vm kernel: [   55.303434] sd 2:0:0:0: [sda] Starting disk
May  6 16:00:04 feliu-vm kernel: [   55.303516] ata4.00: configured for UDMA/100
May  6 16:00:04 feliu-vm kernel: [   55.303563] ata3.00: configured for UDMA/100
May  6 16:00:04 feliu-vm kernel: [   55.306912] ata6: SATA link down (SStatus 0 SControl 300)
May  6 16:00:04 feliu-vm kernel: [   55.307071] ata7: SATA link down (SStatus 0 SControl 300)
May  6 16:00:04 feliu-vm kernel: [   55.307221] ata8: SATA link down (SStatus 0 SControl 300)
May  6 16:00:04 feliu-vm kernel: [   55.442048] OOM killer enabled.
May  6 16:00:04 feliu-vm kernel: [   55.442051] Restarting tasks ... done.
May  6 16:00:04 feliu-vm kernel: [   55.443576] random: crng reseeded on system resumption
May  6 16:00:04 feliu-vm kernel: [   55.443582] PM: suspend exit

[Index of Archives]     [KVM Development]     [Libvirt Development]     [Libvirt Users]     [CentOS Virtualization]     [Netdev]     [Ethernet Bridging]     [Linux Wireless]     [Kernel Newbies]     [Security]     [Linux for Hams]     [Netfilter]     [Bugtraq]     [Yosemite Forum]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux Admin]     [Samba]

  Powered by Linux