Anthony Liguori wrote:
Jim Fehlig wrote:
Daniel Veillard wrote:
On Wed, May 03, 2006 at 09:43:36AM -0600, Jim Fehlig wrote:
Currently I can get info on hvm domains but unable to create them.
The create request is sent to xend but then blocks reading from
xend in xend_req(). No problem creating the domain using xm. I
have looked at the config coming into xend and it is identical
between xm and virsh/libvirt. Need to poke around xend and see why
there is no response to the virsh/libvirt create request for hvm
domains.
Hum, basically to debug this kind of things I spend my time in
/var/log/xend.log especially since it prints what it received at
creation
time, you can at least check the informations coming from libvirt and
from xm look alike.
Unfortunately I won't be able to help you, first because I'm about to
take vacations for 3 weeks :-), second because I don't have one of
those
new Intel CPUs, but I am sure others will give an hand if needed.
Well, I have found the problem but not a fix :-). When posting the
create op to xend, the libvirt code currently waits for a response
from xend (xend_req() is called after posting in xend_post()). For
hvm guests, xend does not respond and libvirt blocks indefinitely on
read. If I skip over the call to xend_req() for the create
operation, wait_for_devices() and subsequently unpause() are called
and the hvm guest is launched.
What do you mean by Xend does not respond?
OK, let me try this again after some experimentation today. First, I'm
using CVS libvirt and it does not use xmlrpc but the "old" S-expression
over http - so I take a different code path to domain_create(). I set a
breakpoint (pdb.set_trace()) in domain_create() and attempted to launch
the hvm guest with xm and virsh/libvirt. The stack trace from xm is
(Pdb) bt
/usr/lib/python2.4/threading.py(442)__bootstrap()
-> self.run()
/usr/lib/python2.4/threading.py(422)run()
-> self.__target(*self.__args, **self.__kwargs)
/usr/lib/python2.4/SocketServer.py(463)process_request_thread()
-> self.finish_request(request, client_address)
/usr/lib/python2.4/SocketServer.py(254)finish_request()
-> self.RequestHandlerClass(request, client_address, self)
/usr/lib/python2.4/SocketServer.py(521)__init__()
-> self.handle()
/usr/lib/python2.4/BaseHTTPServer.py(316)handle()
-> self.handle_one_request()
/usr/lib/python2.4/BaseHTTPServer.py(310)handle_one_request()
-> method()
/usr/lib/python2.4/SimpleXMLRPCServer.py(432)do_POST()
-> response = self.server._marshaled_dispatch(
/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py(103)_marshaled_dispatch()
-> response = self._dispatch(method, params)
/usr/lib/python2.4/SimpleXMLRPCServer.py(406)_dispatch()
-> return func(*params)
/usr/lib/python2.4/site-packages/xen/xend/server/XMLRPCServer.py(65)domain_create()
-> info = XendDomain.instance().domain_create(config)
>
/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py(227)domain_create()
-> self.domains_lock.acquire()
(Pdb)
and the stack trace using virsh/libvirt:
(Pdb) bt
/usr/lib/python2.4/threading.py(442)__bootstrap()
-> self.run()
/usr/lib/python2.4/threading.py(422)run()
-> self.__target(*self.__args, **self.__kwargs)
/usr/lib/python2.4/site-packages/xen/web/httpserver.py(288)run()
-> self.processRequest(sock, addr)
/usr/lib/python2.4/site-packages/xen/web/httpserver.py(316)processRequest()
-> rp.process()
/usr/lib/python2.4/site-packages/xen/web/httpserver.py(110)process()
-> res = req.process()
/usr/lib/python2.4/site-packages/xen/web/httpserver.py(140)process()
-> return self.render(resource)
/usr/lib/python2.4/site-packages/xen/web/httpserver.py(180)render()
-> val = resource.render(self)
/usr/lib/python2.4/site-packages/xen/web/resource.py(59)render()
-> return meth(req)
/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py(135)render_POST()
-> return self.perform(req)
/usr/lib/python2.4/site-packages/xen/web/SrvBase.py(85)perform()
-> return op_method(op, req)
/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py(77)op_create()
-> dominfo = self.xd.domain_create(config)
>
/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py(227)domain_create()
-> self.domains_lock.acquire()
(Pdb)
Continuing in the debugger I find that all routines complete up to
process() in httpserver.py line 110 (note that my line numbers may be
skewed a little due to insertion of debug code). After executing the
statement "res = req.process()", I take the else clause of "if
isinstance(res, ThreadRequest):" - which calls req.finish() which in
turn closes the connection. So no response is sent to libvirt, which is
blocked waiting to read something from xend.
As mentioned before, if I ignore reading a response from xend on the
create op and go directly to wait_for_devices and unpause then the guest
is launched as expected. BTW, I do get responses from xend on
wait_for_devices and unpause ops.
The path is a normal domain_create(). The path within Xend is
server/XMLRPCServer.py but that's a very thin wrapper around a direct
call to XendDomain.py:domain_create(). domain_create() should always
return. If it's not, there's a bug in Xend I think.
Not in the case of libvirt - see above.
Can you give me a very concrete test case for this? The exact
parameters being passed to domain_create() (it should just be an
S-Expression config).
Sent in a previous post.
I will look into this more tomorrow - have other fires that need
attention now.
Thanks for the help :-).
Jim