Re: Ceph on windows (wnbd) rbd.exe keeps crashing

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

Thanks for bringing up this issue. The Pacific release doesn't support ipv6 but fortunately there's a backport that's likely going to merge soon: https://github.com/ceph/ceph/pull/47303.

There are a few other fixes that we'd like to include before releasing new Pacific and Quincy MSIs. I'm seeing increased demand for the ipv6 fix, so we'll try to speed things up.

Regards,

Lucian

On 11.09.2022 09:18, Ilya Dryomov wrote:
On Sun, Sep 11, 2022 at 2:52 AM Angelo Hongens <angelo@xxxxxxxxxx> wrote:
Does that windows driver even support ipv6?
Hi Angelo,

Adding Lucian who would know more, but there is a recent fix for IPv6
on Windows:

https://tracker.ceph.com/issues/53281

Thanks,

                 Ilya

I remember I could not get the driver working as well on my ipv6 setup,
but no logs to help me troubleshoot the issue. I create an issue on
github somewhere, but no response, so I gave up.

Ah, here's my ticket. Might not be related to your issue, but I could
not help my suspicion it might be ipv6 related:
https://github.com/cloudbase/ceph-windows-installer/issues/27



On 09/09/2022 04:22, Stefan Kooman wrote:
Hi,

I try to get wnbd to work on a Windows 2019 virtual machine (Version
1809, OS Build 17763.2183). Unfortunately the process rbd.exe keeps
crashing (according to logs in event viewer).

I have tested with a linux VM in the same network and that just works.

In the ceph.conf I specified the following (besides mon host):

[global]
log to stderr = true
run dir = C:/ProgramData/ceph
crash dir = C:/ProgramData/ceph
debug client = 2

ms bind ipv4 = false
ms bind ipv6 = true

[client]
keyring = C:/ProgramData/ceph/keyring
admin socket = c:/ProgramData/ceph/$name.$pid.asok
debug client = 2

Note: The Ceph network is IPv6 only, and no IPv4 is involved.


I double checked that I can connect with the cluster from the VM.
Eventually I made a tcpdump and from that dump I can conclude that the
client keeps on trying to connect to the cluster (probably because the
rbd.exe process is restarting over and over) but never seems to manage
to actually connect to it. Although debug logging is defined in the
ceph.conf, the client does not write any log output.

Here an example of a crash report:


��Version=1
EventType=APPCRASH
EventTime=133068416700266255
ReportType=2
Consent=1
UploadTime=133068512230149899
ReportStatus=4196
ReportIdentifier=1c521fd8-325f-494e-9b6b-e7a608d9f1b1
IntegratorReportIdentifier=1a2b76ca-248e-4636-9ed6-1cae6c332c0c
Wow64Host=34404
NsAppName=rbd.exe
AppSessionGuid=000003c4-0001-0011-7c4e-cb1b05c1d801
TargetAppId=W:00007c6b388ea9ba05b8df74c0e19907c78c00000904!0000e32ad63d5bac11abc70d42f12a6c189e6b9edfdc!rbd.exe
TargetAppVer=1970//01//01:00:00:00!26b9f4!rbd.exe
BootId=4294967295
TargetAsId=2397
IsFatal=1
EtwNonCollectReason=1
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=rbd.exe
Sig[1].Name=Application Version
Sig[1].Value=0.0.0.0
Sig[2].Name=Application Timestamp
Sig[2].Value=00000000
Sig[3].Name=Fault Module Name
Sig[3].Value=libceph-common.dll
Sig[4].Name=Fault Module Version
Sig[4].Value=0.0.0.0
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=00000000
Sig[6].Name=Exception Code
Sig[6].Value=40000015
Sig[7].Name=Exception Offset
Sig[7].Value=000000000069a3bb
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=10.0.17763.2.0.0.272.7
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=1033
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=74e0
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=74e0e499b0f720be12f39b443eb7059c
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=f912
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=f9121477e3a172a0e72323a2204f3558
UI[2]=C:\Program Files\Ceph\bin\rbd.exe
LoadedModule[0]=C:\Program Files\Ceph\bin\rbd.exe
LoadedModule[1]=C:\Windows\SYSTEM32\ntdll.dll
LoadedModule[2]=C:\Windows\System32\KERNEL32.DLL
LoadedModule[3]=C:\Windows\System32\KERNELBASE.dll
LoadedModule[4]=C:\Windows\System32\msvcrt.dll
LoadedModule[5]=C:\Windows\System32\WS2_32.dll
LoadedModule[6]=C:\Windows\System32\RPCRT4.dll
LoadedModule[7]=C:\Program Files\Ceph\bin\libboost_program_options.dll
LoadedModule[8]=C:\Program Files\Ceph\bin\libwinpthread-1.dll
LoadedModule[9]=C:\Program Files\Ceph\bin\libgcc_s_seh-1.dll
LoadedModule[10]=C:\Program Files\Ceph\bin\libstdc++-6.dll
LoadedModule[11]=C:\Program Files\Ceph\bin\libceph-common.dll
LoadedModule[12]=C:\Windows\System32\ADVAPI32.dll
LoadedModule[13]=C:\Windows\System32\sechost.dll
LoadedModule[14]=C:\Windows\System32\bcrypt.dll
LoadedModule[15]=C:\Program Files\Ceph\bin\librados.dll
LoadedModule[16]=C:\Program Files\Ceph\bin\librbd.dll
LoadedModule[17]=C:\Program Files\Ceph\bin\libboost_random.dll
LoadedModule[18]=C:\Program Files\Ceph\bin\libboost_thread_pthread.dll
LoadedModule[19]=C:\Program Files\Ceph\bin\libcrypto-1_1-x64.dll
LoadedModule[20]=C:\Windows\System32\USER32.dll
LoadedModule[21]=C:\Windows\System32\win32u.dll
LoadedModule[22]=C:\Windows\System32\GDI32.dll
LoadedModule[23]=C:\Program Files\Ceph\bin\libboost_iostreams.dll
LoadedModule[24]=C:\Windows\System32\gdi32full.dll
LoadedModule[25]=C:\Windows\System32\msvcp_win.dll
LoadedModule[26]=C:\Windows\System32\ucrtbase.dll
LoadedModule[27]=C:\Windows\SYSTEM32\IPHLPAPI.DLL
LoadedModule[28]=C:\Program Files\Ceph\bin\libssl-1_1-x64.dll
LoadedModule[29]=C:\Program Files\Ceph\bin\zlib1.dll
LoadedModule[30]=C:\Windows\System32\IMM32.DLL
LoadedModule[31]=C:\Windows\system32\napinsp.dll
LoadedModule[32]=C:\Windows\System32\mswsock.dll
LoadedModule[33]=C:\Windows\SYSTEM32\DNSAPI.dll
LoadedModule[34]=C:\Windows\System32\NSI.dll
LoadedModule[35]=C:\Windows\System32\winrnr.dll
LoadedModule[36]=C:\Windows\system32\NLAapi.dll
LoadedModule[37]=C:\Windows\system32\wshbth.dll
LoadedModule[38]=C:\Windows\System32\CRYPTSP.dll
LoadedModule[39]=C:\Windows\system32\rsaenh.dll
LoadedModule[40]=C:\Windows\SYSTEM32\CRYPTBASE.dll
LoadedModule[41]=C:\Windows\System32\bcryptPrimitives.dll
OsInfo[0].Key=vermaj
OsInfo[0].Value=10
OsInfo[1].Key=vermin
OsInfo[1].Value=0
OsInfo[2].Key=verbld
OsInfo[2].Value=17763
OsInfo[3].Key=ubr
OsInfo[3].Value=2183
OsInfo[4].Key=versp
OsInfo[4].Value=0
OsInfo[5].Key=arch
OsInfo[5].Value=9
OsInfo[6].Key=lcid
OsInfo[6].Value=1033
OsInfo[7].Key=geoid
OsInfo[7].Value=176
OsInfo[8].Key=sku
OsInfo[8].Value=7
OsInfo[9].Key=domain
OsInfo[9].Value=0
OsInfo[10].Key=prodsuite
OsInfo[10].Value=272
OsInfo[11].Key=ntprodtype
OsInfo[11].Value=3
OsInfo[12].Key=platid
OsInfo[12].Value=10
OsInfo[13].Key=sr
OsInfo[13].Value=0
OsInfo[14].Key=tmsi
OsInfo[14].Value=395666
OsInfo[15].Key=osinsty
OsInfo[15].Value=2
OsInfo[16].Key=iever
OsInfo[16].Value=11.1790.17763.0-11.0.1000
OsInfo[17].Key=portos
OsInfo[17].Value=0
OsInfo[18].Key=ram
OsInfo[18].Value=16384
OsInfo[19].Key=svolsz
OsInfo[19].Value=49
OsInfo[20].Key=wimbt
OsInfo[20].Value=0
OsInfo[21].Key=blddt
OsInfo[21].Value=180914
OsInfo[22].Key=bldtm
OsInfo[22].Value=1434
OsInfo[23].Key=bldbrch
OsInfo[23].Value=rs5_release
OsInfo[24].Key=bldchk
OsInfo[24].Value=0
OsInfo[25].Key=wpvermaj
OsInfo[25].Value=0
OsInfo[26].Key=wpvermin
OsInfo[26].Value=0
OsInfo[27].Key=wpbuildmaj
OsInfo[27].Value=0
OsInfo[28].Key=wpbuildmin
OsInfo[28].Value=0
OsInfo[29].Key=osver
OsInfo[29].Value=10.0.17763.2183.amd64fre.rs5_release.180914-1434
OsInfo[30].Key=buildflightid
OsInfo[31].Key=edition
OsInfo[31].Value=ServerStandard
OsInfo[32].Key=ring
OsInfo[32].Value=Retail
OsInfo[33].Key=expid
OsInfo[34].Key=containerid
OsInfo[35].Key=containertype
OsInfo[36].Key=edu
OsInfo[36].Value=0
File[0].CabName=WERInternalMetadata.xml
File[0].Path=WER4224.tmp.WERInternalMetadata.xml
File[0].Flags=327683
File[0].Type=5
File[0].Original.Path=\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER4224.tmp.WERInternalMetadata.xml
FriendlyEventName=Stopped working
ConsentKey=APPCRASH
AppName=rbd.exe
AppPath=C:\Program Files\Ceph\bin\rbd.exe
NsPartner=windows
NsGroup=windows8
ApplicationIdentity=60648466A12126577E345FE09EAA07DC
MetadataHash=1790235354

Any hint on how to proceed with troubleshooting?

Gr. Stefan


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux