Automated GUI testing for anaconda with dogtail

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello list,
as proposed in this BZ there is an idea to test anaconda with dogtail:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=172891

For those who have no idea what is this:

http://people.redhat.com/zcerza/dogtail/
"dogtail is a GUI test tool and automation framework written in Python. It uses Accessibility (a11y) technologies to communicate with desktop applications. dogtail scripts are written in Python and executed like any other Python program."

I have been working on another approach to access anaconda from dogtail.
The idea is to run anaconda but display the GUI on another machine.
   e.g. export DISPLAY=remote.host:1.0
The remote host is the host responsible for running the dogtail test.
This is simple, not intrusive to anaconda source code and I would say independent of the installation environment. There is nothing special but just run a X application and show it on remote display as we do frequently.

Before I spam you with details on let me ask a few questions.

Q1) How bad is the need for having automated test environment for Anaconda? For me this is an application with a couple of buttons and I can't really imagine that something can break so badly.

Q2) How does the community / developers feel like this idea? Do you guys like it? Will it benefit you somehow? Is it worth implementing it and setting up a test environment? Isn't it too much overhead just to be able to test one single application?

Q3) How does other vendors using Anaconda feel about that?
I saw on the web that several Linux distros besides Red Hat and Fedora use Anaconda. I will suppose that these vendors have made custom modifications/improvements and will want to test them? Do you really want to? On the other hand how many custom modifications exist?
Do other vendors use the same version that RH/Fedora uses?

Q4) How do the developers really test bug fixes and new features?
I've heard some use xen or another physical machine with VNC installs,
others use a chroot environment and run anaconda from inside.
Is your setup hard to use/maintain? Does a simpler setup benefit you?
Do you think it is going to work for you?

I will like to hear different opinions and ideas.
Any comments and/or suggestions are welcome.
Everybody who is able to test the proposed implementation
on distribution different than RedHat/Fedora please share your experience.

Details are attached.

Greetings,
Alexander.
IDEA

We want to run Anaconda but display the GUI on another machine.
   e.g. export DISPLAY=remote.host:1.0
The remote host is the host responsible for running the Digtail test.

ENVIRONMENT

***
Remote host must have all Dogtail/atspi libs installed and set up.
I think it must have bonobo-activation-server running in order to register AT enabled applications.
For a proof of concept I will use a full blown GNOME desktop while the uses is logged in.

***
The installer image/tree under test requires a small hack.
It needs to know where to forward all X output.
We need: 
/sbin/loader -> <set up X forwarding> -> Anaconda

http://fedoraproject.org/wiki/AnacondaExtendingKickstart
"Unfortunately, /sbin/loader has the path /usr/bin/anaconda hard-wired into it.
 It is possible to change this, of course, but it involves rebuilding Anaconda itself, and there is an easier way."

I created a updates.img:/anaconda which is a bash script. It gets run instead of the real anaconda python script.
We set up X forwarding and then run the installer. It is possible that this file is called more than once.
We are setting up the environment only the first time.

#!/bin/sh

FAKE_LOG='/tmp/ramfs/fake-anaconda.log';
if [ ! -f $FAKE_LOG ]; then
   # I should only be run once - make sure it's obvious I've started to execute!
   touch $FAKE_LOG
   export DISPLAY=$TARGETHOST:$TARGETDISPLAY
   echo "Will run on $DISPLAY" >> $FAKE_LOG
fi

# I wasn't supposed to run more than once - this command must be for Anaconda!
echo "Running Ananconda ..." >> $FAKE_LOG
# This is the real anaconda
exec /usr/bin/anaconda $@ >> $FAKE_LOG 2>&1


The updates.img is an ext2 formated file system and together with the redirection script can be generated by a CGI.
The script can be modified to wget info from a CGI application in a lab environment.
This can be used to automate a lab environment with several X hosts each of them exporting several displays.
We can track hosts/displays in a database and redirect Anaconda to machine that happens to be free.

***
Scheduler
  +--> select a machine to install, stage 1 ---> 
       <redirect X to $HOST> --->
       +      (  wait to set up testing host environment ...............) --> run Anaconda ....................................... [END / CRASH]
       |                                                                  |                    |                                          |
       +----> install the testcase.rpm on $HOST and wait for a connection ^....................^(client attached) -> continue testcase....^ --> report results [END]

BUGS

***
In order this one to work we need some features. The waiting host must be aware that the remote application supports AT.
This is done by the application anouncing(registering) itself into a database called the Registry.
There was a bug that the Registry was user/session dependent but not $DISPLAY dependent.

Relevant links:

https://wiki.ubuntu.com/Accessibility/Specs/SudoAdminAtspi

Bug 163132 â?? at-spi should expose all clients on user's DISPLAY
http://bugzilla.gnome.org/show_bug.cgi?id=163132

This seems to be fixed recently in newer versions of GNOME.
------------------------------------------------------------
Product:  	at-spi     Version: 	1.3.x 	 	 
Component: 	registry   Priority: 	Normal 	
OS: 	        All 	   Severity: 	enhancement 	 
Status: 	RESOLVED   Target Milestone: 	--- 	 
Resolution: 	FIXED      Gnome version: 	2.7/2.8 	 
                           Gnome target: 	Unspecified
------------------------------------------------------------

I tried the gedit-utf8 example included in dogtail package with the following set up:
Ubuntu Feisty, GNOME 2.18.0, apt-get install python-dogtail,
run a second GNOME session on the same machine and:
display_0$ xset +

display_1$ export DISPLAY=:0.0
display_1$ gedit

display_0$ AT-SPI Browser - detects gedit
display_0$ gedit-utf8 example - WORKS

Version of installed packages:
libatk1.0-0  1.18.0-0ubuntu1
at-spi       1.18.0-0ubuntu1
libgail18    1.18.0-0ubuntu1
----------------------------------------------------------


More links and info:
DOES NOT WORK here:  http://bugzilla.gnome.org/show_bug.cgi?id=154487
Opened by Zack Cerza 2004-10-04 18:19 UTC
Versions of stuff (this is an amd64, hence the atk dupe):
$ rpm -q atk at-spi libgail-gnome
 atk-1.8.0-1
 atk-1.8.0-1
 at-spi-1.6.0-3
 libgail-gnome-1.1.0-1

Comment #6 from bill haneman sun com  (at-spi developer) 2006-06-23 16:51 UTC
 
The issue is that the app is connecting to a user/host-based at-spi-registryd instead of using one
associated with the local DISPLAY.

-----------------------------------------------------------

The above Ubuntu example works on latest Fedora (xen guest):
Running an application on the xen guest showing on the second display works.
If you enable a11y you can run the dogtail browser and explore the application
which was started from the other display.

xen$ cat /etc/fedora-release
Fedora release 6.92 (Rawhide)

Version of stuff:
xen$ rpm -q atk at-spi libgail-gnome
 atk-1.18.0-1.fc7
 atk-1.18.0-1.fc7
 at-spi-1.18.0-1.fc7
 at-spi-1.18.0-1.fc7
 libgail-gnome-1.18.0-2.fc7

I can't get X forwarding between hosts working.
I shutdown SELinux and firewall.
Even with xhost + which allows everybody to connect I get

developer_machine$ export DISPLAY=xen_guest:0.0
developer_machine$ gedit 
cannot open display: 
Run 'gedit --help' to see a full list of available command line options.

If I ssh -X between the 2 machines starting X applications works and they show up
as expected (at the machine running ssh client).

[Index of Archives]     [Kickstart]     [Fedora Users]     [Fedora Legacy List]     [Fedora Maintainers]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [Yosemite Photos]     [KDE Users]     [Fedora Tools]
  Powered by Linux