Automated interoperability test derivation for communication protocols based on the multiple stimuli principle

Abstract

In this dissertation, we propose automatic test derivation methods for testing interoperability of communication protocols, such as the ATM/B-ISDN signaling protocol and TCP. In the past, researches on testing communication protocols have mainly concentrated on conformance testing to check the conformance of the implementation of a protocol with regard to the protocol specification. Researches on interoperability testing with more than one interoperable entity mainly focused on general concepts and experiences of interoperability testing and systematic generation of interoperability test suites. However, little work has been reported on data part testing in interoperability test and work on test generation dealing with simultaneous inputs. This dissertation addresses two different aspects of automatic derivation of interoperability test suites; for both control and data parts of the protocol, and for complex scenarios caused by simultaneous inputs on several interfaces of the system under test (SUT). Regarding data part testing, our proposed method first derives a $\emph{skeleton}$ test suite for control part through $\emph{reachability analysis}$. Next, it parameterizes each test case. Finally, it completes the test suite by actualizing the test cases by assigning values to their parameters. By constructing test case space based on $\emph{stable states}$ and by basing test case actualization step on $\emph{orthogonal arrays}$ used in experimental design techniques $\emph{state space explosion}$ can be avoided. In order to demonstrate the feasibility and effectiveness, the proposed method is applied to the ATM/B-ISDN signaling protocol and TCP. The application result shows that the method surpasses the conventional methods in both test size and coverage. Conventional methods examine only single input at a time in interoperability test generation, and thus their test cases are unable to test more complex behaviors of the protocol. We extend our test generati...