6 messages in com.canoo.lists.webtestRE: [Webtest] cruisecontrol
FromSent OnAttachments
David Escala24 Jul 2002 04:39 
Dierk Koenig24 Jul 2002 05:06 
EPu...@upstate.com24 Jul 2002 05:33 
Dierk Koenig24 Jul 2002 06:00 
EPu...@upstate.com24 Jul 2002 13:55 
Dierk Koenig25 Jul 2002 03:49 
Subject:RE: [Webtest] cruisecontrol
From:Dierk Koenig (dier@canoo.com)
Date:07/24/2002 05:06:44 AM
List:com.canoo.lists.webtest

Hi David,

i am using webtest within cruisecontrol and I'd like to make cruisecontrol build fail if there is any failing webtest step. my build.xml tasks are like these:

- check repository for changes - if no changes wait and repeat - checkout project - build and unit testing (with phpunit or junit) - deploy on an integration server - functional testing on the integration server (with webtest)

I guess you have some more steps here, that you don't want to get executed, if the webtests fail. (e.g. tagging the current version)

if everything works fine cruisecontrol sends a "Build Successful" email, and if something goes wrong it ought send a "Build Failed" message. an error in junit testing breaks the build and cruisecontrol sends the "Build Failed" message, but, and that's my point, an error in webtest testing does not break the build. why?

By default, WebTests are supposed to work as user acceptance tests rather than technical verifications. Acceptance tests are likely not to run 100% from the time they were delivered. Therefore, WebTest just reports about the progress in terms of running acceptance tests rather than breaking the build. This is in contrast to Unit Tests. These are supposed to run 100% with every build.

You can steer this behaviour of WebTest with the "haltonerror" and "haltonfailure" attributes of the config step. see http://webtest.canoo.com/webtest/manual/syntax.html#2

For Build number 231 you will find the report of the concerned selftest step under http://webtest.canoo.com/selftest/TestResultOverview.html#testspec22

hope that helps and happy testing Mittie