Title: Welcome

Michael, Matt:

I believe the "Test requirements" document and schema are on good tracks. Let me try to summarize where we are, and raise some remaining issues:

1- The Requirements schema/doc, so far captures Level 1 ("core", or most of it). When the corresponding tests are run (controlled by ANT + JXUnit), one can check that an ebXML message is well-formed and also semantically sound, according to Level 1. That actually captures "wire-conformance" Level 1.

2- We expect to drive the candidate MSH so that it generates a diverse-enough sampling of messages, each of which will undergo the Level 1 "wire-conformance" tests above.

3- Issue: as we will drive the candidate MSH to generate messages to be tested, it is not enough to test each of these through the "Level 1 tests" above. Indeed, we want to make sure in addition that the set of generated messages (test set), uses all features expected in Level 1. Example: if for some reason the candidate MSH fails to generate some optional message elements - e.g. digital signature, etc. - then its message format may still be correct, as tested on the wire, but we cannot claim Level 1 passes, as the test coverage would not have checked all expected message features. Solutions: we need to have a way to "require" and to check, in the generated message test set, that all expected message features have ben produced at least once. That could be done by (a) either keeping a checklist of all message features that are concerned by Level 1, and match it with what has actually been teted, (b) or by making sure that the test associated with each Level 1 Requirement has been used at least once (<condition> part successful?), (c) or by doing a more through comparison between what the test driver has been fed (input test cases) and what is observed on the wire. note that would have some "service-conformance" aspect.

Just following up on Michael schemas (ebXMLCompare.xsd, ebXMLSet.xsd, ebXMLTestSuite.xsd):

4- Michael: could you give a quick overview / use case of how you see instances of these schemas being used? (I assume that instances of ebXMLTestSuite.xsd will be input to a "comparison engine"?)

5- I assume these schemas will then be used to compare actual output / expected output, right? (that could then help on issue #3-c above.)

6- Issue: We'll need to define how to compare schema instances, e.g. in some cases, some distance is allowed with the reference document (e.g. optional element, order, different values...)

7- COnformance reports - should it be instance of an extended Requirements schema with test results?

-Regards,

jacques

-----Original Message----- From: Jacques Durand [mailto:JDur...@fs.fujitsu.com] Sent: Monday, March 18, 2002 3:42 PM To: 'ebxm...@lists.oasis-open.org' Subject: [ebxml-iic-msg] Welcome

Hi:

Welcome to the ebxml-iic-msg sublist.

This is where you will read or post mails about MS conformance testing design, and more generally about the "Conformance Testing Task Force" activities and material.

Regards,