100 likes | 103 Vues
IV&V Requirements Tracing using Information Retrieval Techniques. Jane Hayes Alexander Dekhtyar. Department of Computer Science University of Kentucky. The Problem. How can we automate tracing requirments during IV&V?. Relevance to NASA. Alleviate work of NASA IV&V analysts
E N D
IV&V Requirements TracingusingInformation Retrieval Techniques Jane Hayes Alexander Dekhtyar Department of Computer Science University of Kentucky
The Problem How can we automate tracing requirments during IV&V? Relevance to NASA • Alleviate work of NASA IV&V analysts • Improve quality of IV&V for NASA software Importance/Benefits Improve analyst productivity on one of the most time-consuming IV&V tasks
Approach • Use Information Retrieval Techniques for Requirements Tracing • Build RETRO (REquirements TRacing On-target) • Evaluate performance • TF-IDF, Thesaurus, Probabilistic IR, LSI • Analyst Feedback • Metrics • Special-purpose requirments tracing tool • Standalone version • Integrated with SAIC’s SuperTracePlus • MODIS, LOFAR, CM-1 datasets
Analyst Approach: IR for Requirements Tracing Matching algorithm 2 1 3 Design Document representation Feedback Yes Yes No Requirements Document
Approach: Metrics Recall:percentage of correct design specifications retrieved out of all correct design specifications Precision:percentage of correct design specifications amongall design specifications retrieved Time:time it took to produce the result Secondary Metrics(Lag, DiffR):good links rising, bad links sinking
Accomplishments: I RETRO: REquirements TRacing On-target
Accomplishments:RETRO Architecture documents Analyst Feedback processor Filter Build Representation IR toolbox
A SFEP STP Interactive Link Anlysis STP Report Generation Accomplishments: RETRO + SuperTracePlus requirements documents Analyst Review RETRO Build RETRO IR Toolbox RETRO Feedback Representation Traceability Reports STP RETRO
Accomplishments II: Test ResultsMODIS, TF-IDF, Thesaurus Above 70%
Next Steps, Conclusions, Plans, Ideas • IR methods work : need to implement more • Productize RETRO (Check!) • Data Integration with existing tools (Check!) • Other IV&V problems may be alleviated