Matt / Michael:

Quite good material here from XMLG - I assume you guys will use this to complete the current (XML) "test reqs" document?

Some comments:

1. About some test items: - the DuplicateElimination tests (section 3.1.7, testing actual duplicates), really belong to "Additional features" (Reliability module), however the "wellformedness" of the header element (if present) can still be considered as core Level (schema compliance)... - a few items in this list are CPA-dependent. We may want to put them aside for now until we introduce CPA-content modeling in our tests(3.1.2, 3.1.5, 4.1.2, 4.2.4.2...).

2. As we have noticed before, many test items are in the form of IF-THEN rules. Either the "IF" is controlled : (a)- by the MSH : (i.e. is an implementation choice, e.g. 2.2.1 &2.2.2 "XML prolog in SOAP") In that case, this test item may never apply, as the MSH will never generate messages that satisfy the "IF". (b)- or the "IF" is controlled by the application (e.g. or 3.1.8 "if the Description elt is present...", 2.1.4 "if there is no payload...") In that case, a well-rounded test suite is expected to produce messages that will satisfy the "IF" of these test items. (c)- or the "IF" is controlled by the CPA (e.g. all test items conditional to the presence of Security elements) In order to test (b), relevant messages need be generated. That means we need also to define a sampling of "test messages" that we want to see sent by the tested MSH. We need then to define a format (XML) for message data to be fed to the MSH. To be more precise, to be fed to the Test Driver that controls the tested MSH. These test messages should be expressed in an "envelope-neutral" XML format (i.e. NOT ebXML!)

3. The XML format for message sampling used by test driver in (2) above, could adopt a JMS style: Part 1 will specify header elements as a list of "property / value" pairs, Part 2 will specify the payload(s). Roughly, that could be of the form:

<messagedata name="testABC"> <header_properties> <fromPartyID>123</fromPartyID> <toPartyID>456</toPartyID> <Service>PurchaseOrdering</Service> <Action>Accept</Action> ... </header_properties> <payloads> <payload> .... an XML payload, or a ref to a binary payload...</payload> </payloads> </messagedata>

The test driver would have to "translate" this in API calls to the MSH (using a small adapter specific to the candidate MSH's API).

Defining such samples of messages will also help down the road to check the "service-conformance", that is, the MSH really sends the message content that  the application (here the test driver) expects it to send out.

4. We may have a similar approach for defining the CPA under which a test is to be performed (i.e. we define an XML format that encodes the subset of CPA needed to "configure" the MSH).

5. As our "test requirements" definition document will also somehow "bind" the test items, with the test procedures that check them, it may be good to distinguish the test procedures these test items require. So far I see the test items you described fall into two test procedures: (P1)- "drive-and-receive" procedures, where the test driver is fed some message data, and controls the candidate MSH so that it generates a messages that we capture on the HTTP destination. Most of the test items you described are tested that way. A Test case definition for such procedures is made of: (1) test item requirement def (and associated procedure), (2) message data to be fed to driver. (P2)- "send-and-receive" procedures, where we send a message to the candidate MSH, and observe its response-message. This usually involves the test driver on candidate MSH side, to provide response data (e.g. 3.1.6.3a), but sometime the response is generated directly by the candidate MSH (in case of errors, e.g. 3.1.6.3bc, 3.1.6.4)  A Test case definition for such procedures is made of: (1) test item requirement def (and associated procedure), (2) message data to be sent to candidate MSH (described using same XML neutral format). (3) message data to be fed to driver for response to test driver of candidate, if applicable.   We probably should start thinking of how to more completely describe such test cases. I can give a first cut at a  "message data" XML format, and give it for review.