|Jacques Durand||Mar 27, 2002 7:33 pm|
|Matthew MacKenzie||Mar 27, 2002 11:14 pm|
|Jacques Durand||Mar 28, 2002 6:55 pm|
|Matthew MacKenzie||Mar 28, 2002 10:30 pm|
|Jacques Durand||Mar 29, 2002 5:09 pm|
|Matthew MacKenzie||Mar 30, 2002 7:40 pm|
|Jacques Durand||Mar 31, 2002 10:20 pm|
|Michael Kass||Apr 3, 2002 7:48 am|
|Subject:||Re: [ebxml-iic-msg] MS Conformance checklist: comments|
|From:||Matthew MacKenzie (ma...@xmlglobal.com)|
|Date:||Mar 27, 2002 11:14:42 pm|
On Wednesday, March 27, 2002, at 07:18 PM, Jacques Durand wrote:
Matt / Michael:
Quite good material here from XMLG - I assume you guys will use this to complete the current (XML) "test reqs" document?
That is the plan. We have one person devoted to doing this stuff, including some test case definition for these conformance issues, and I hope to donate this stuff where possible to kickstart our work here.
1. About some test items: - the DuplicateElimination tests (section 3.1.7, testing actual duplicates), really belong to "Additional features" (Reliability module), however the "wellformedness" of the header element (if present) can still be considered as core Level (schema compliance)... - a few items in this list are CPA-dependent. We may want to put them aside for now until we introduce CPA-content modeling in our tests(3.1.2, 3.1.5, 4.1.2, 184.108.40.206...).
2. As we have noticed before, many test items are in the form of IF-THEN rules. Either the "IF" is controlled : (a)- by the MSH : (i.e. is an implementation choice, e.g. 2.2.1 &2.2.2 "XML prolog in SOAP") In that case, this test item may never apply, as the MSH will never generate messages that satisfy the "IF". (b)- or the "IF" is controlled by the application (e.g. or 3.1.8 "if the Description elt is present...", 2.1.4 "if there is no payload...") In that case, a well-rounded test suite is expected to produce messages that will satisfy the "IF" of these test items. (c)- or the "IF" is controlled by the CPA (e.g. all test items conditional to the presence of Security elements) In order to test (b), relevant messages need be generated. That means we need also to define a sampling of "test messages" that we want to see sent by the tested MSH. We need then to define a format (XML) for message data to be fed to the MSH. To be more precise, to be fed to the Test Driver that controls the tested MSH. These test messages should be expressed in an "envelope-neutral" XML format (i.e. NOT ebXML!)
3. The XML format for message sampling used by test driver in (2) above, could adopt a JMS style: Part 1 will specify header elements as a list of "property / value" pairs, Part 2 will specify the payload(s). Roughly, that could be of the form:
<messagedata name="testABC"> <header_properties> <fromPartyID>123</fromPartyID> <toPartyID>456</toPartyID> <Service>PurchaseOrdering</Service> <Action>Accept</Action> ... </header_properties> <payloads> <payload> .... an XML payload, or a ref to a binary payload...</payload> </payloads> </messagedata>
The test driver would have to "translate" this in API calls to the MSH (using a small adapter specific to the candidate MSH's API).
Why not just have a library of ebXML Message templates that we want to send, and use a simple preprocessor and a tool like netcat to send the messages to the MSH? I don't think we care to test a candidate MSH's client side API, just the behaviour of the MSH itself.
We were thinking of defining a journal or log format that candidate msh's would write during conformance testing. We would define the format of the log, and all of the points where a log must be written. There would also be a Conformance service that the MSH would implement allowing the conformance test suite to "download" the journal after a test run. e.g.
<journal start="2002-8-6T12:43:45"> <entry time="2002-8-6T12:43:47" type="msg_recv"> <![CDATA[ message here ]]> </entry>
<entry time="2002-8-6T12:44:00" type="ack_sent"> <![CDATA[ ack here ]]> </entry> </journal>
Defining such samples of messages will also help down the road to check the "service-conformance", that is, the MSH really sends the message content that the application (here the test driver) expects it to send out.
4. We may have a similar approach for defining the CPA under which a test is to be performed (i.e. we define an XML format that encodes the subset of CPA needed to "configure" the MSH).
A simple property file will likely suffice.
5. As our "test requirements" definition document will also somehow "bind" the test items, with the test procedures that check them, it may be good to distinguish the test procedures these test items require. So far I see the test items you described fall into two test procedures: (P1)- "drive-and-receive" procedures, where the test driver is fed some message data, and controls the candidate MSH so that it generates a messages that we capture on the HTTP destination. Most of the test items you described are tested that way. A Test case definition for such procedures is made of: (1) test item requirement def (and associated procedure), (2) message data to be fed to driver. (P2)- "send-and-receive" procedures, where we send a message to the candidate MSH, and observe its response-message. This usually involves the test driver on candidate MSH side, to provide response data (e.g. 220.127.116.11a), but sometime the response is generated directly by the candidate MSH (in case of errors, e.g. 18.104.22.168bc, 22.214.171.124) A Test case definition for such procedures is made of: (1) test item requirement def (and associated procedure), (2) message data to be sent to candidate MSH (described using same XML neutral format). (3) message data to be fed to driver for response to test driver of candidate, if applicable. We probably should start thinking of how to more completely describe such test cases. I can give a first cut at a "message data" XML format, and give it for review.
The message data format will certainly be useful, and could be integrated with the requirements definition, however, you must remember that the requirements document simply states the requirements, not necessarily give you the details of how to test compliance to the requirements. Nonetheless, being able to pass this kind of data onto the test framework is probably very useful. I'll let Michael weigh in here, as he is definitely the expert on this sort of thing.
-- Matthew MacKenzie XML Global R&D PGP Key available upon request.