Introduction
This page describes a trace driven signaling test tool for sip signaling components. It was driven by the needs of SIpXbridge but can be adapted to regression test other components in the system. The tool that is described here is a developers test tool. The envsioned use is for a developer to run the tool on his component before commit.
Problem Definition
Regression tests for SIP signaling components are difficult to construct. Tools that rely on scripting expect the developer to construct a script to emulate signaling behaviour. Constructing a scenario that exactly emulates a signaling behavior can be difficult and error prone. Any test tool that relies upon individually crafted scripts to run the test falls victim to errors that may exist in those test scripts. Such test scripts may be useful when developing the system but not as a means of regression testing.
Our normal mode of development for the signaling components consists of developer driven testing for various call flows which is done against actual endpoints, gateways etc. This is followed by system testing by the QA team which runs the developed components in a pseudo operational setting by running various call handling scenarios.
For example in SipXbridge testing, we may test call transfer blind to local phone from ITSP, call transfer attended to local phone from ITSP, call pickup for inbound call or outbound call etc. Each of these scenarios generates a call flow.
As the system evolves and goes through interoperabilty testing, we would like to ensure that scenarios that have already worked. One way of doing this is to run tests manually against actual equipment. However it is exceedingly difficult to do this because it requires a lot of configuration. Further, due to the dynamic nature of the protocol, different message interleaving may result in the test case than in the scenario that was debugged and hence the developer is never sure that exactly the same scenario has been re-run.
Principle Of Operation
Signaling services such as sipXbridge are message driven finite state automata. i.e. from an external perspective they are driven by SIP signaling messages that appear at a port and they generate messages as a result of such messages. The important thing with testing such components is to send requests to and expects requests and responses from the SUT in the same order as a working test case does. The regression is a stimulus response tester which tries to maintain the same message ordering as was observed in a trace. The diagram below shows the setup : 
The regression test tool (RT) is meant to test a single component in isolation by emulating actual call flows. Emulation in this case consists of presenting the SUT with a sequence of SIP requests which are suitably re-written. It is not meant to be a system tester i.e. it does not test signaling through multiple hops. RT reads a trace of a successful interoperability test that is presented as a suitably filtered "merged.xml" file (i.e. the output of sipx-trace with appropriate flags to isolate the call Id's of interest. It then consults a second configuration file (monitored-interfaces.xml) that indicates which IP addresses and ports in the captured merged.xml are of interest. A third configuration file (testmaps.xml) specifies an address mapping between a given trace message and an emulated trace message. As each message is presented to the SUT, it is suitably re-written by consulting this configuration file. Finally, the tester address and base port are configured in tester config.xml