Re: RBD I/O errors with QEMU [luminous upgrade/osd change]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



The only errors message I see is from dmesg when trying to accessing the
XFS filesystem (see attached image).

Let me know if you need any more logs - luckily I can spin up this VM in
a broken state as often as you want to :-)

Attachment: t.png
Description: PNG image

Jason Dillaman <jdillama@xxxxxxxxxx> writes:

> ... also, do have any logs from the OS associated w/ this log file? I
> am specifically looking for anything to indicate which sector was
> considered corrupt.
>
> On Mon, Sep 11, 2017 at 4:41 PM, Jason Dillaman <jdillama@xxxxxxxxxx> wrote:
>> Thanks -- I'll take a look to see if anything else stands out. That
>> "Exec format error" isn't actually an issue -- but now that I know
>> about it, we can prevent it from happening in the future [1]
>>
>> [1] http://tracker.ceph.com/issues/21360
>>
>> On Mon, Sep 11, 2017 at 4:32 PM, Nico Schottelius
>> <nico.schottelius@xxxxxxxxxxx> wrote:
>>>
>>>
>>> Thanks a lot for the great ceph.conf pointer, Mykola!
>>>
>>> I found something interesting:
>>>
>>> 2017-09-11 22:26:23.418796 7efd7d479700 10 client.1039597.objecter ms_dispatch 0x55b55ab8f950 osd_op_reply(4 rbd_header.df7343d1b58ba [call] v0'0 uv0 ondisk = -8 ((8) Exec format error)) v8
>>> 2017-09-11 22:26:23.439501 7efd7dc7a700 10 client.1039597.objecter
>>> ms_dispatch 0x55b55ab8f950 osd_op_reply(14 rbd_header.2b0c02ae8944a
>>> [call] v0'0 uv0 ondisk = -8 ((8) Exec format error)) v8
>>>
>>> Not sure if those are the ones causing the problem, but at least some
>>> error.
>>>
>>> I have attached the bzip'd log file for reference (1.7MiB, hope it makes
>>> it to the list) and wonder if anyone sees the real reason for the I/O errors?
>>>
>>> Best,
>>>
>>> Nico
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Mykola Golub <mgolub@xxxxxxxxxxxx> writes:
>>>
>>>> On Sun, Sep 10, 2017 at 03:56:21PM +0200, Nico Schottelius wrote:
>>>>>
>>>>> Just tried and there is not much more log in ceph -w (see below) neither
>>>>> from the qemu process.
>>>>>
>>>>> [15:52:43] server4:~$  /usr/bin/qemu-system-x86_64 -name one-17031 -S
>>>>> -machine pc-i440fx-2.1,accel=kvm,usb=off -m 8192 -realtime mlock=off
>>>>> -smp 6,sockets=6,cores=1,threads=1 -uuid
>>>>> 79845fca-9b26-4072-bcb3-7f5206c2a531 -no-user-config -nodefaults
>>>>> -chardev
>>>>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/one-17031.monitor,server,nowait
>>>>> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
>>>>> -no-shutdown -boot strict=on -device
>>>>> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
>>>>> file='rbd:one/one-29-17031-0:id=libvirt:key=DELETEME:auth_supported=cephx\;none:mon_host=server1\:6789\;server3\:6789\;server5\:6789,if=none,id=drive-virtio-disk0,format=raw,cache=none' -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/var/lib/one//datastores/100/17031/disk.1,if=none,id=drive-ide0-0-0,readonly=on,format=raw -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -vnc [::]:21131 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on 2>&1 | tee kvmlogwithdebug
>>>>>
>>>>> -> no output
>>>>
>>>> Try to find where the qemu process writes the ceph log, e.g. with the
>>>> help of lsof utility. Or add something like below
>>>>
>>>>  log file = /tmp/ceph.$name.$pid.log
>>>>
>>>> to ceph.conf before starting qemu and look for /tmp/ceph.*.log
>>>
>>>
>>> --
>>> Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch
>>>
>>
>>
>>
>> --
>> Jason


--
Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux