atom feed22 messages in org.oasis-open.lists.tagRE: [tag] Test Assertion Modeling - c...
FromSent OnAttachments
step...@systml.co.ukJul 30, 2007 4:45 pm 
Durand, Jacques R.Aug 13, 2007 1:28 pm 
Dave PawsonAug 14, 2007 12:46 am 
Durand, Jacques R.Aug 14, 2007 11:01 am 
step...@systml.co.ukAug 14, 2007 11:16 am 
Dave PawsonAug 14, 2007 11:25 am 
step...@systml.co.ukAug 14, 2007 1:19 pm 
step...@systml.co.ukAug 14, 2007 1:54 pm 
step...@systml.co.ukAug 14, 2007 2:40 pm 
Durand, Jacques R.Aug 14, 2007 5:42 pm 
Dave PawsonAug 14, 2007 11:34 pm 
step...@systml.co.ukAug 15, 2007 9:43 am 
Serm KulvatunyouAug 16, 2007 8:50 am 
Serm KulvatunyouAug 16, 2007 9:36 am 
step...@systml.co.ukAug 16, 2007 10:54 am 
Dave PawsonAug 16, 2007 11:41 pm 
step...@systml.co.ukAug 17, 2007 11:07 am 
Dave PawsonAug 18, 2007 12:05 am 
step...@systml.co.ukAug 18, 2007 1:41 am 
Dave PawsonAug 18, 2007 2:37 am 
step...@systml.co.ukAug 18, 2007 3:24 am 
step...@systml.co.ukAug 20, 2007 10:38 am 
Subject:RE: [tag] Test Assertion Modeling - comments, etc
From:Durand, Jacques R. (JDur@us.fujitsu.com)
Date:Aug 14, 2007 5:42:07 pm
List:org.oasis-open.lists.tag

Stephen:

To me that sounds too much like a specification requirement. The fact that you use the expression "An application conforming to ..." does not make it more TA-like. It makes it sound more like a conformance clause, still a different beast.

Now, taking what you wrote as a spec requirement (assuming it is in the original spec :-) you could probably derive a TA for it, like:

Event: Sender implementation sends a UBL doc Condition: (SBS spec is used) and (the doc contains parts external to SBS spec) Behavior: Sender provides some form of warning (...) about the doc.

Jacques

-----Original Message----- From: step@systml.co.uk [mailto:step@systml.co.uk] Sent: Tuesday, August 14, 2007 2:40 PM To: ta@lists.oasis-open.org Subject: Re: [tag] Test Assertion Modeling - comments, etc

How about this as a test assertion for the SBS rule #1

The TA is targeted at / relevant to (maybe it should say so) an application which sends a UBL SBS document

An application conforming to the SBS which sends a UBL document, if it allows the document to include elements not defined in the SBS specification as being required, it SHOULD provide a warning that such an element might not be processed by or visible to the receiver of the document.

-- Stephen Green

Partner SystML, http://www.systml.co.uk Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice

Having to rethink the idea of test assertions for this SBS rule -

The document which conforms to the spec, a UBL SBS document, is a kind

of spec itself - it specifies some business, whether a payment that needs to be made in conformance to an invoice or a supply of goods or services in conformance to an order. The spec is kind of saying with this SBS rule that certain specified parts of that document should be regarded as informative only. Now how

on earth can you test for that? :-)

Maybe you can't in terms of software at all. Maybe it's just a legal thing and not for testing at all. How do you test that something is being treated as normative or informative in a particular implementation? In this case it is a matter of testing that say an order is part informative and part normative and that the informative part, if present at all, does not require normative treatment. That can only be a matter or business and legal vigilance can't it - not a software testing thing at all perhaps. A lesson seems to be here though that not everything need be tested for it to be be worth writing in test assertion formal language - at least not tested with typical software tests. The test assertion technique might be used to facilitate business and legal implementation of a business document and software which produced it in a way that software testing itself cannot do.

-- Stephen Green

Partner SystML, http://www.systml.co.uk Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice

Thanks. Excellent.

Of course, the tests can be limited to a business process of which the document under test is a part. In a way this binds the testing not only to the business document but also to it's business process -

an interesting aspect of the SBS and other business language profiles

perhaps.

So here the TA has to take into account not just the subset but also the business process of which it participates. So the spec can't really include the TAs as such unless it can generalise them for all anticipated business processes or somehow make the assertions independant of the processes.

I did imagine there will be quite a few layers to test and also that these might inter-relate in some way - is that a problem though? inter-relating layers of validation? A test in one layer might depend

on results from another layer. This means there is a sequence in testing with dependencies even crossing between layers of testing. What happens if any tests are mutually dependant or if two layers of tests are mutually dependant? Is it necessary to declare all depend- encies somehow in the TAs? What if dependencies existed between different sets of assertions (if

the assertions were split for different audiences as I think you suggest) but not all test sets were used as supplied. In that case making dependencies explicit would show up the mismatches. But if the

test assertions were put into the same file then ID and IDREF could maybe be used to assure of the integrity of dependencies between id and ref to some extent when validating the TA document. Else XInclude

or the like might be used if dependencies crossed file boundaries.

-- Stephen Green

Partner SystML, http://www.systml.co.uk Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice

Quoting Dave Pawson <dave@gmail.com>:

Wow. Quite a test.... or more correctly a test sequence?

On 14/08/07, step@systml.co.uk <step@systml.co.uk>

wrote:

Hi. Thanks Jacques and David and for comments.

Yes, the testing of this item for the SBS profile for UBL which relates to the Sender (there is another rule which relates to the Receiver) could be like the following:

Essentially it is a business rule, albeit of an unusually technical

nature but there are still tests which can be applied. Some such tests might be best applied by technical auditors though, even by legal experts in some cases.

Not sure how these test requirements would be expressed as test assertions though since maybe the target audience of the test assertion would be a technical auditor or legal expert, even a legal court (if say there was a large sum unpaid because the receiver had ignored some payment terms or tax amounts which were external to the subset) eventually.

Layering needed? A single test (pass or fail). A test group (again pass or fail with test results) If test group passes, output the message in an appropriate format? Appropriate to the audience that is. Groups layered into supergroups as needed, culminating in a complete

application. The end resut is the summation (done automatically, not collated by the application, which cheats) of the test groups. Again either pass or fail.

Nasty: Writing tests with built in debug. I.e. run it with the debug flag set and all the techie garbage flows. Switch it off and the legal eagle sees pass, or fail, test number 1,290.

If you've multiple audiences, the test application layer needs parameters.

regards