Hey All:

Great discussion so far. I just want to add something about why regression testing is important to me. In the Orca project, we have a regression test suite that we use regularly. It helps give us great sanity checking that our changes and bug fixes don't introduce new problems. It has caught issues introduced by those infamous simple-one-liner-cannot-hurt-a-thing fixes, and it has also been a great tool to help us with performance analysis.

It took a while to ingrain it into the Orca culture, but regression testing is just so useful to us that I cannot imagine NOT doing it. This is not a call to use Orca's stuff, BTW. It's just amplifying the need for and usefulness of regression across all of GNOME.

An interesting scenario might be to hook up with the build brigade folks. We might have a set of systems that just continually build GNOME, but we might dedicate another set of systems to continually build GNOME followed by a complete set of regression tests. At the same time, individual component developers should also be able to set up their own machines to focus testing just on their components. For example, as soon as http://bugzilla.gnome.org/show_bug.cgi?id=508147 is fixed, I hope to work on a cronjob to automatically test Orca against nightly builds of Firefox.

Finally, IMO the focus on the regression tests should be more about coverage of the important stuff and less about the time it takes to run them. For example, even if they took 7 days to run, that would still give us tons better coverage and would detect integration problems far sooner.

Will

Nagappan A wrote: