|step...@systml.co.uk||Sep 21, 2007 2:19 pm|
|Dave Pawson||Sep 21, 2007 10:10 pm|
|step...@systml.co.uk||Sep 22, 2007 1:47 am|
|Dave Pawson||Sep 22, 2007 2:06 am|
|step...@systml.co.uk||Sep 22, 2007 2:46 am|
|Dave Pawson||Sep 22, 2007 5:25 am|
|step...@systml.co.uk||Sep 22, 2007 1:43 pm|
|Dave Pawson||Sep 22, 2007 10:09 pm|
|step...@systml.co.uk||Sep 23, 2007 5:37 am|
|step...@systml.co.uk||Sep 23, 2007 5:51 am|
|Dave Pawson||Sep 23, 2007 6:11 am|
|step...@systml.co.uk||Sep 23, 2007 8:22 am|
|Dave Pawson||Sep 23, 2007 9:15 am|
|Subject:||Re: [tag] TA Model|
|Date:||Sep 23, 2007 8:22:54 am|
So, Dave, would you not want the TA to put the Schematron validation after the XML Schema validation and to link the two together somehow?
Do you accept that there is no point validating with Schematron until first it is established with XML Schema that the instance has syntax which adheres strictly to the XML Schema (in a typical case, such as for codelists)?
Does it not make sense to cater for a series of tests in a single test assertion when there are clearly cases where this is an essential part of making test assertions for a specification?
Isn't that why we are calling it a test assertion 'flow'?
Isn't it likely that many TAs will have to say that one test must take place after another (especially when it has a pre-condition which has to have previously been verified in a test - a test not specified in this same TA)?
Quoting Dave Pawson <dave...@gmail.com>:
Why? IMHO a test should be independent. Dependencies on other tests goes against this. Perhaps such a test is at too low a level?
An example I'm aware of where a test depends on the pass outcome of a previous test is a Schematron validation test isn't worth performing (and therefore can't result in a pass) unless previously there was a successful pass outcome to a test of the XML syntax such as with XML Schema or RelaxNG.
Test flow issue. The Schematron test shouldn't be run, as determined by the flow control. Not down to the test. The Schematron test should remain independent.
There may, however, be reason to include the XML Schema validation in the same TA as the Schematron validation (feasible but I'll take it as an unproven assumption) but to put the two validation phases into two tests in the flow in such a way that the dependency can be expressed formally.
I'd separate those. I'd expect the latter not to be called. A pre-condition would be belt and braces, requiring the accumulation of test results somewhere... That's getting too close to 'how' for my liking.
Say the second test step of the flow uses an ID from the first step to express the dependency. It could include a reference in the pre-condition of step 2 to the first step's result (which requires a unique ID on that result).
If the two were separate, then the entire TA (rng validation say) would be the unit level, and the item with the ID. I don't think its needed beneath that level, other than internally to the test. Data hiding ideas.
So to summarize so far, things like pre-conditions might need to reference either things like triggers, results/effects (I too prefer 'result' to 'effect'), pre- and post-conditions in tests so each need unique IDs *or* they may need to reference the final outcome of a test flow as a whole so that needs an overall unique ID which could just be the ID of the TA.
I could imagine there might be some situations where the pre-condition might need to reference something other than the entire TA or the result of a particular step/test in a multi-step/test flow.
I'd view that as bad design. Keep each test black box 'ish.
If there is an
aspect of the referenced TA or step which isn't actually formally tested it could be something which is a post-condition. So the post-conditions need unique reference too to facilitate that.
If part of a test doesn't run, then that would need agreement with the design authority, and presumably would allow the test to pass. If that's the case, then the black box model still works.
Either way, I see these as all aspects of a spec (the example is part of the in-progress OASIS Codelist Methodology and plays a part in UBL's spec) and therefore to need TAs and not therefore just for the test designers. They live in the space between spec and tests in such cases. (Two-phase codelist validation is the example I worked from.)
The tests can't be written well without design authority sign off.
-- Dave Pawson XSLT XSL-FO FAQ. http://www.dpawson.co.uk