Hi Brad,

I got this mail from Willima Walker, maybe it is late reply. I am in Sun as the QE/RE manager for desktop team. My team has done lots of testing for gnome applications, actually we are finding ways how to do the automation testing.

The principle for automation tools we are using is that it is easy and can be helpful for our project testing quickly. For firefox Tinderbox testing, we are using LDTP for the nightly build check, becasue we can leverage LDTP community work on it. Recently mozilla community is using the Eggplant for their automation testing, we also evaluated and tried it. For the A11Y automation testing, we are using Orca for the automation work. For flash automation testing, we also try the ULLU tool developed by flash team.

For gnome automation testing, we have done one evalutaion comparison about dogtail vs LDTP, please see attachment file.

We don't include the strongwind in here, because we aren't familiar with it. :-) Please let us know your comments.

Because dogtail is integrated into Solaris and is easy for our automation development and maintaining, we preferred to use dogtail for gnome automation now. We have finished some automation works for gnome application sanity testing, it is running for our nightly build testing now. We are tracking the testing, maintaining these automation testing codes, and analyse our result and data.

Actually it is still in the evaluation and trial phase, we are collecting data about ROI for our automation work and decide how to do it. We hope to get more resource to do the automation development work, and especially hope to leverage resource from gnome community. I think this is very good start to discuss about this.

Thanks, Paul. ------------------------------------------------------------------------

Subject: Re: GNOME's testing strategy for GUIs From: Brad Taylor <br...@getcoded.net> Date: Thu, 14 Feb 2008 11:09:15 -0800 To: Willie Walker <Will...@Sun.COM>

To: Willie Walker <Will...@Sun.COM> CC: "desk...@gnome.org" <desk...@gnome.org>

Hi Willie,

First off, in the spirit of full disclosure, I should mention that I work for Medsphere, the company who backs the development of the GPL testing tool, Strongwind[1].

Ken VanDine and I met at SCALE this last week and discussed some plans regarding automated testing in GNOME. We identified a few rough spots:

* Many tools are available for automated testing, and GNOME needs to annoint one with holy gnome pee so that it will become an acceptable dependency for development. Whether this is LDTP, Dogtail, Strongwind, or others, a clear decision needs to be made so that people can go forth with creating tests.

Having a mix of testing tools in use will only make it more difficult for folks to contribute to, and maintain a large body of tests. * Someone needs to step up and write good, thorough documentation for developing tests. To my knowledge, no current testing tool has this.

* A culture of automated testing needs to be developed and fostered inside of GNOME so that maintainers feel real benefit in maintaining a high-quality test library. Maintenance is not a trivial thing.

* Distributions who already have automated tests for GNOME applications need to have a place to contribute and maintain these tests _publicly_, and have them shared by all. This was made clear in discussions with folks at this year's GNOME Summit.

Obviously, there is a lot of work that needs to be done, but these challenges are by no means insurmountable.

The first thing that needs to come into the picture are good docs. I've commited myself to contribute a Gnome Journal article on writing Strongwind tests as a first start into this field.

Cheers,

-Brad