atom feed22 messages in org.oasis-open.lists.tagRe: [tag] Test Assertion Modeling - c...
FromSent OnAttachments
step...@systml.co.ukJul 30, 2007 4:45 pm 
Durand, Jacques R.Aug 13, 2007 1:28 pm 
Dave PawsonAug 14, 2007 12:46 am 
Durand, Jacques R.Aug 14, 2007 11:01 am 
step...@systml.co.ukAug 14, 2007 11:16 am 
Dave PawsonAug 14, 2007 11:25 am 
step...@systml.co.ukAug 14, 2007 1:19 pm 
step...@systml.co.ukAug 14, 2007 1:54 pm 
step...@systml.co.ukAug 14, 2007 2:40 pm 
Durand, Jacques R.Aug 14, 2007 5:42 pm 
Dave PawsonAug 14, 2007 11:34 pm 
step...@systml.co.ukAug 15, 2007 9:43 am 
Serm KulvatunyouAug 16, 2007 8:50 am 
Serm KulvatunyouAug 16, 2007 9:36 am 
step...@systml.co.ukAug 16, 2007 10:54 am 
Dave PawsonAug 16, 2007 11:41 pm 
step...@systml.co.ukAug 17, 2007 11:07 am 
Dave PawsonAug 18, 2007 12:05 am 
step...@systml.co.ukAug 18, 2007 1:41 am 
Dave PawsonAug 18, 2007 2:37 am 
step...@systml.co.ukAug 18, 2007 3:24 am 
step...@systml.co.ukAug 20, 2007 10:38 am 
Subject:Re: [tag] Test Assertion Modeling - comments, etc
From:step...@systml.co.uk (step@systml.co.uk)
Date:Aug 18, 2007 1:41:54 am
List:org.oasis-open.lists.tag

Thanks David

It's probably important to balance these two views (and I could happily accept yours that the markup and schema validity can provide the mainstay of a standard, less so tool validity). OASIS standards traditionally major on markup ("structured information") so it especially applies to OASIS.

Tool creators do need their own validity test but I gather you are proposing that the essense of such tests should be schema validity of XML output where applicable. This concept is kind of growing on me :-)

The concepts of event-behaviour need to be overlaid on this though for standards such as messaging ones where the system behaviour is of the essense. APIs will likely fall into the latter camp too of course.

Maybe we now need some examples to balance the SBS one where there is no requirement or applicability of an emphasis on instance validity.

Best regards

-- Stephen Green

Partner SystML, http://www.systml.co.uk Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice

Quoting Dave Pawson <dave@gmail.com>:

On 17/08/07, step@systml.co.uk <step@systml.co.uk> wrote:

I'll try to defend my stance on validation of both tool and markup.

My opinion is that tool validation is out of scope.

Here's a quote from the html spec for 4.0:

Lots of description of tool use here.

Tool use. Now find an application validator? No. The validation is by schema on the (x)html produced.

The browser presenting the html can surely be validated in how it does so as well as the html itself being validated against the DTD.

Find one. I don't think you will. It's like judging how you won the race. The result is won/2nd etc.

The focus is generally on what is produced.

With UBL there is little point just describing the markup although an attempt has really been made to make conformance a matter of validity of an instance against the schema.

I.e. the conformance is of an instance to the schema.

If my app insists on using an order

number as the invoice number element content I can't claim my app is compliant even though the instances will still validate. So UBL has a focus on saying the definitions of the elements are normative too.

Again Schema based, on what is produced. Data types are included in the schema.

I don't think a test suite can validate the content in those terms though apart from schema validity and maybe some calculation business rule validity too

You just need different tools for business rules. Have a look at Schematron.

This kind of illustrates that not only must the document be valid according to the spec in its structure but maybe with regard to its semantics (where there are normative rules, etc)

I don't know of any semantics check on xml. Thats generally where people come in.

and moreover that an

end user can be using the standard in a way which is valid or not. So the app has to test user input according to test assertions, say, and a test suite might test the app according to the same or other test assertions perhaps and the test suites can be tested using test assertions too.

You're getting too abstract for me here. Any examples? And how far back into meta meta land do we go?

The documents can be tested and its tests tested too.

Usually against a test plan. Answer the question, 'does this test prove compliance to the requirement'.

Lots of tests and test assertions and layers and so on. If the standard is well written it will foster all of this to ensure best implementation and end user experience. A tall order though.

I'll close on this thread now. I think what has been made clear is the need for a clear scope. AFAIK, there isn't one yet.

regards