1 / 23

A Model Driven Tool for Automated System Level Testing of Middleware

A Model Driven Tool for Automated System Level Testing of Middleware. Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute for Software Integrated Systems Vanderbilt University Nashville, TN USA. Outline. Motivation and Challenges Model Driven Approach

season
Télécharger la présentation

A Model Driven Tool for Automated System Level Testing of Middleware

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute for Software Integrated Systems Vanderbilt University Nashville, TN USA

  2. Outline • Motivation and Challenges • Model Driven Approach • Tool Structure • Test Series Definition Modeling Language • Example Test Series Definition • Conclusions and Ongoing Work STV'06 Workshop March 30, 2006

  3. Need for many test configurations Motivation and Challenges • Many middleware framework configurations are generally represented by XML • Configuring functional operation of components • Component deployment in the cluster • Frameworks are complicated and need to be tested • Application topologies • Parameter variations STV'06 Workshop March 30, 2006

  4. Motivation and Challenges • Hand writing test configurations • Cumbersome • Error prone • Inefficient • Big configuration space of control and deployment parameters of applications • Response to changes in hardware resources or other criteria • Difficult to scale up manually written XML configurations • Usual outcome: Under tested software STV'06 Workshop March 30, 2006

  5. Goals • Testing large sets of system configurations • Simple powerful specifications • Tool to take specifications to instances • Configure series of tests from the specifications • Help finding “corner cases” • Potential functional errors • Potential performance problems • Unexpected system interactions STV'06 Workshop March 30, 2006

  6. Model Driven Approach • Domain Specific Modeling • Use domain specific models to encapsulate relevant parts of the system properties • Provide multiple view points • Decouple types, resources, dataflow and deployment aspects • Generative modeling • Vary range of behavioral and structural system parameters • System generation • Automatically generate configurations from higher level models STV'06 Workshop March 30, 2006

  7. Tool Structure Test Series Definition Models STV'06 Workshop March 30, 2006

  8. Test Series Definition • Define tests using a high level modeling language called Test Series Definition Modeling Language (TSDML) • Define application types • Form a template test case from those types • Define dataflow association between applications by connection rules • Define deployment associations for applications • Define the variance on the system size and structure • Define variance on the application parameters • Define available resources • Automatically create series of test configuration models by replicating and adapting the template test case STV'06 Workshop March 30, 2006

  9. Test Configuration Models Tool Structure Test Series Definition Models Generator (1-*) STV'06 Workshop March 30, 2006

  10. Test Configuration Models • Form the configuration layer • Each test configuration model capture: • Specific dataflow scenario • Specific deployment scenario • Configuration Schema • Represents single system representation to be tested • Automatically generate single configuration file per configuration model • Test driver scripts can also be generated STV'06 Workshop March 30, 2006

  11. Test Configuration Models Test Execution & Validation Engine Applications Application Runtime (System Under Test) Tool Structure Test Series Definition Models Generator (1-*) Generator (1) XML Configs Test Scripts STV'06 Workshop March 30, 2006

  12. System Under Test • XDAQ Framework • High performance distributed middleware framework developed at CERN • Will be used for Compact Muon Selenoid (CMS) experiment • Run-time support for clustered applications • Predefined functions to configure and control the applications and their parameters • XML is used for configuration of XDAQ and its applications STV'06 Workshop March 30, 2006

  13. i j r1 Connector r2 App1 App2 Test Series Definition Modeling Language (TSDML) • Parameterized model • Varying system structure (size) • Iterators: e.g. i,j,k = {start, step, stop} • Replicators: e.g. r1=n*i, r2=m*j • Connectors: Boolean function of iterator, and replicator • If src=(i,r1) and dst=(j,r2)  C=f(src,dst)  C=2*src==3*dst STV'06 Workshop March 30, 2006

  14. i j r1 App1 App2 Iterator-Replicator Example Assume i = 1:1:2, j= 1:1:4, r1 = 2, r2 = 4 r2 STV'06 Workshop March 30, 2006

  15. i i r1 r2 App1 App2 Connector Example i = 1:1:2, j= 1:1:4, r1 = 2, r2 = 3 CR = src=floor((dst-1)/(j*r2/i*r1)) + 1 Connector STV'06 Workshop March 30, 2006

  16. App1 App2 App1 App2 App2 App2 Connector Example (ctd…) Rule: src=floor((dst-1)/(j*r2/i*r1)) + 1 (1) 1=floor(2/2) + 1 => FALSE 1=floor(3/2) + 1 => FALSE 1=floor(0/2) + 1 => TRUE 1=floor(1/2) + 1 => TRUE d s t (1) (2) s r c (3) (2) (4) STV'06 Workshop March 30, 2006

  17. Test Series Definition Modeling Language (TSDML) • Varying application parameters with parameter generator • Has a function attribute • Function of iterator • Function attribute may be: • Internal function • e.g. result = src * 256 where src = i • Matlab script • e.g. lut = [10 20 55 88 400 800 1025 4490 7342]; result = lut[src]; • Internal lookup table STV'06 Workshop March 30, 2006

  18. Example TSDML STV'06 Workshop March 30, 2006

  19. 1:1:8 1:1:1 2*i 2*i 16*i 1*i 1:1:1 Example TSDML STV'06 Workshop March 30, 2006

  20. Conclusion • Domain specific modeling for capturing relevant system parts • Vary system structure and application parameters from high level model • Generate series of test configurations from a single high level model • Generate the configuration representation • Mitigate the complexity of creating many system test configurations STV'06 Workshop March 30, 2006

  21. Ongoing Work • Add another layer on top of TSDML • System model • Application behavior • Target framework • Analyze the models • Assess important parts of the system to exercise and test STV'06 Workshop March 30, 2006

  22. Thank you! STV'06 Workshop March 30, 2006

  23. A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute for Software Integrated Systems Vanderbilt University Nashville, TN USA

More Related