1 / 108

A Embedded software component quality framework

A Embedded software component quality framework. Fernando Ferreira de Carvalho Advisor: Silvio Romero de Lemos Meira Informatics Center - Federal University of Pernambuco C.E.S.A.R. - Recife Center for Advanced Studies and Systems ffc@cin.ufpe.br 12-Junho-2008.

oma
Télécharger la présentation

A Embedded software component quality framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Embedded software component quality framework Fernando Ferreira de Carvalho Advisor: Silvio Romero de Lemos Meira Informatics Center - Federal University of Pernambuco C.E.S.A.R. - Recife Center for Advanced Studies and Systems ffc@cin.ufpe.br 12-Junho-2008

  2. A Embedded software component quality framework Introduction / Motivation Embedded system is at the heart of many systems So, embedded system industry needs, Low production cost Short time to market High Quality to be more efficient and competitive (Brown, 2000) The CBD with reuse technique had been a nice direction to reach this objectives… But, Component reuse withoutquality assurance give catastrophic results [ariane]. http://www.rise.com.br Fernando Carvalho, PhD Candidate

  3. A Embedded software component quality framework Introduction / Motivation The CBD technologies do not take into account the specifics needs of embedded-systems development: Timing, Memory, power, hardware constraints and others. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  4. A Embedded software component quality framework Introduction / Solution Certification is the future of software components[Wallnau, Heineman, Councill, Shaw] According to Weber et al. (Weber et al., 2002), the need for quality assurance in software development has exponentially increased in the past few years SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  5. A Embedded software component quality framework Introduction / Motivation Benefits More reliability Short time-to-market; More product quality; higher quality levels, reduced maintenance time, investment return, reduced time-to-market, among others Wide used, where certification is mandatory (aircraft) SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  6. A Embedded software component quality framework Rise Framework SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  7. A Embedded software component quality framework Component Certification Process SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  8. A Embedded software component quality framework Component Certification Processfocused in embedded systems An Embedded Computer System: A computer system that is part of a larger system and performs some of the requirements of that system; for example, a computer system used in an aircraft or rapid transit system. (IEEE,1992). SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  9. A Embedded software component quality framework Component Certification Processfocused in embedded systems • Embedded systems : • Used for specifics propose • Used to control systems (ex: mechanical machines) • Ultra small devices with simple specific functionality • Small systems with sophisticated functions • Produced in large scale SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  10. A Embedded software component quality framework Component Certification Processfocused in embedded systems Embedded systems has a specific requirements: - real-time - hi reliability - low power consumption - low data and code memory - low resources - low CPU capabilities - others SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  11. A Embedded software component quality framework Problem Formulation The CBD is an efficient and effective way for design of simple and complex embedded systems. However, quality assurance of components is must be done to take advantage of CBD. The Software industrystill far to reach the maturity level the hardware industry which it have catalogues and datasheets available for its components. For this reason, a well-defined and consistent embedded software component quality assurance is essential for CBD and reuse adoption. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  12. A Embedded software component quality framework Proposed solution An Embedded Software Component Quality Verification Framework It is composed of four inter-relation module:, based on a set of activities, metrics and guidelines. • Embedded software component Quality Model (EQM) • Maturity Level evaluation Techniques • Metrics Approach • Component Certification based on a set of activities, metrics and guidelines. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  13. A Embedded software component quality framework Proposed solution This Framework is based in the standards • ISO/IEC 9126, 2001 - Quality Model for Software Product • ISO/IEC 14598, 1998 - Software Product Evaluation Process This two standards converged to: • ISO/IEC 25010, 2005 - Software product quality - requirements and evaluation • theFramework adapted thequality modelandevaluationto component context and embedded domain. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  14. A Embedded software component quality framework Out of scope This Framework is part of broad context, some aspects were expectedsince initial definition. Nevertheless, other process can be added in the future. • Cost Model • Formal Proof • Prediction of the component assembly SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  15. A Embedded software component quality framework 2 – Embedded System Design Embedded system design comprise: • Ultra-small device x simple functionality • Small system x sophisticated functions • Large systems and distributed systems • Systems produced in large quantities x low production cost • Systems produced in low volume x important features SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  16. A Embedded software component quality framework 2 – Embedded System Design The different requirements of embedded systems have a impact on feasibility, on use of CBD for it. A common characteristic in different area of embedded domain is increasing importance of software [Crnkovic, 2003]. • Example, the software cost in embedded systems: • in industrial robots constitute about 75% of total cots • in car industry it is about 30% • Fifteen year ago: • 25% of total cots in industrial robots • Negligible for cars SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  17. A Embedded software component quality framework 2 – Embedded System Design Properties that involves embedded software component is divided in: • Functional property (component interface) • Non-functionalor Extra-functionalproperty, so called Quality attributes, fox example: • Timing • Performance • Consumption • Resource Behavior, and others. This properties can be classified in run-time and life-cycle. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  18. A Embedded software component quality framework 2 – Specific Requirements for Embedded System In the most of case, embedded system is real-time with limited resource. So, it has specifics characteristics which depends on domain application, but it have strong implication on requirements. TheREQUIREMENTSare relatedExtra-functionalproperty or Quality attributes,and its priority depends on the domain application. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  19. A Embedded software component quality framework 2 – Specific Requirements for Embedded System There has been developed a research in orderto find the most important characteristics in different areas in embedded domain. • Industrial Automation • Automotive • Medical • Electronic consumer • Other domain SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  20. A Embedded software component quality framework 2 – Specific Requirements for Embedded System – Industrial Automation Industrial Automation was classified by research’s Larsson, [Larsson, 2002] The most important characteristics, following the research: • At low level: • Availability • Timeliness • Reliability • At high level: • Performance • Usability • Integrability SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  21. A Embedded software component quality framework 2 – Specific Requirements for Embedded System - Automotive Akerholm [Akerholm, 2005]executed a research in vehicle industry. The resulting list of characteristics is presented below • Safety • Reliability • Predictability • Usability • Extendibility • Maintainability • Efficiency • Testability • Security • Flexibility SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  22. A Embedded software component quality framework 2 – Specific Requirements for Embedded System - Medical Wijnstra [Wijnstra, 2001]describe their experience with characteristics in the development of medical imaging family. The resulting list of characteristics is presented below • Reliability • Safety • Functionality • Portability • Modifiability • Configurability • Extensibility and Evolvability • Security • Serviceability SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  23. A Embedded software component quality framework 2 – Specific Requirements for Embedded System – Others Domain Crnkovic [Crnkovic, 2003]summarized the main characteristics and sub-characteristics in the CBD approach apply to embedded system in his research. The table show the results. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  24. A Embedded software component quality framework 2 – Embedded System Design – Software component quality • So, embedded software component quality verification must be different that general propose component, because the component evaluation is realized focused in specifics requirements • We divided the quality verification in two groups: • General propose software component quality process • desktops, servers, x86 architecture • Specific propose software component quality process • embedded systems SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  25. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey The relevant research explore the theory of component quality and certification in academic scenarios, but not rich in reports in practical experience. The pioneering works focus in mathematical and test model, while recent researchers have focused in techniques and model based on predicting quality requirements. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  26. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey Timeline of research in the embedded software component quality and certification area proposed standard X fail → a work was extended by another SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  27. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey In 1993, Poore [Poore et al., 1993] develop an approach based on three mathematical model (sampling, component and certification models), using test cases to report the failures to achieve a reliability index Poore estimated the reliability of a complete system, and not of individual software units, although, they did consider how each component affected the system’s reliability. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  28. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey Wohlin [Wohlin et al., 1994] presented the first method of component certification using modeling techniques, making it possible not only to certify components but to certify the system. • It is composed of the usage model and the usage profile. • The failure statistics from the usage test form the input of a certification model. • An interesting point of this approach is that the usage and profile models can be reused in subsequent certifications However, even reusing those models, the considerable amount of effort and time that is needed makes the certification process a hard task. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  29. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey In 1994, Merrit (Merrit, 1994) presented an interesting suggestion: the use of components certification levels. These levels depend on the nature, frequency, reuse and importance, as follows: •Level 1: No tests are performed; the degree of completeness is unknown; • Level 2: A source code component must be compiled and metrics are determined; • Level 3: Testing, test data, and test results are added; and • Level 4: A reuse manual is added. These levels represent an initial component maturity model. However, this is just a suggestion of certification levels and no practical work was actually done to evaluate it. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  30. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey In 1996, Rohde (Rohde et al., 1996) provided a reuse and certification of embedded software components at Rome Laboratory of the US Air Force, a Certification Framework (CF), that included: •To define the elements of the reuse context that to certification; •To define the underlying models and methods of certification; and, •To define a decision-support technique to construct a context-sensitive process for selecting and applying the techniques and tools to certify components. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  31. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey •A Cost/Benefit plan that describes a systematic approach of evaluating the costs and benefits. Rohde et al. considered only the test techniques to obtain the defects result in order to certify software components. This is only one of the important techniques that should be applied to component certification. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  32. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey Voas [Voas, 1998] defined a certification methodology using automated technologies, such as black-box testing and fault injection to determine if a component fits into a specific scenario. This methodology uses three quality assessment techniques: (i)Black-box component testingdetermine if the component quality is high enough; (ii)System-level fault injectiondetermine how well a system will tolerate a faulty component; (iii)Operational system testing determine how well the system will tolerate a properly functioning component SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  33. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey According to Voas, this approach is not foolproof and perhaps not well suited to all situations. The methodology does not certify that a component can be used in all systems. This approach certify a component within a specific system and environment. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  34. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey Wohlin and Regnell [Wohlin and Regnell, 1998] extended their previous research (Wohlin et al., 1994), now, focusing on techniques for certifying both components and systems. Thus, the certification process includes : • usage specification (consisting of a usage model and profiles), and • (ii) certification procedure, using a reliability model. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  35. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey The main contribution of that work is the division of components into classes for certification and the identification of three different ways of certifying software systems: • i. Certification process, the functional requirements are validated during usage-based testing; • ii. Reliability certification of component and systems, the component models that were built are revised and integrated to certify the system that they form; and, • iii. Certify or derive system reliability, where the focus is on reusing the models that were built to certify new components or systems. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  36. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey • However, the proposed methods are theoretical without experimental study. According to Wohlin et al., “both experiments in a laboratory environment and industrial case studies are needed to facilitate the understanding of component reliability, its relationship to system reliability and to validate the methods that were used only in laboratory case studies” (pp. 09). • Until now, no progress in those directions was achieved. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  37. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey • In 2000, Jahnke, Niere and Wadsack [Jahnke, Niere and Wadsack, 2000] developed a methodology for semi-automatic analysis of embedded software component quality. • This approach evaluates data memory (RAM) utilization in Java technologyby the component. • The work is restricted because: • - Verifies the component quality from only one point of view, use of data memory in a specific language, • - Java is widely used for the development of desktops systems not useful for embedded development. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  38. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey Stafford (Stafford et al., 2001) developed a model for the component marketplaces that supports prediction of system propertiesprior to component selection. • The model use functional verification and quality-related values associated with a component, called credentials. • This work introduced notable changes in this area. It use a specific notation such as <property,value,credibility>. Through credentials, the developer chooses the bestcomponents to use in the application development based on the “credibility” level. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  39. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey • Stafford introduced the notion of active component dossier, its is an abstract component that defines credentials. • Stafford et al. finalized their work with some open questions, such as: • how to certify measurement techniques? • What level of trust is required under different circumstances? • Are there other mechanisms that might be used to support trust? SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  40. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey In 2002, Comella-Dorda et al. (Comella-Dorda et al., 2002) proposed a COTS software product evaluation process. The process contains four activities, as follows: • Planning the evaluation-> evaluation team, stakeholders, required resources, basic characteristics of the evaluation • Establishing the criteria -> evaluation requirements , evaluation criteria; • Collecting the data -> component data are collected, the evaluations plan is done and the evaluation is executed; and • Analyzing the data -> the results of the evaluation are analyzed and some recommendations are given. The proposed process is an ongoing work and, no real case study was accomplished, becoming unknown the real efficiency. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  41. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey In 2003,Beus-Dukic (Beus-Dukic et al., 2003) proposed a method to measure quality characteristics of COTS components, based on the international standards for software product quality (ISO/IEC 9126, ISO/IEC 12119 and ISO/IEC 14598). The method is composed of four steps: • Establish evaluation requirements, specifying the purpose and scope of the evaluation, specifying evaluation requirements; • Specify the evaluation, selecting the metrics and the evaluation methods; • Design the evaluation, considers the component documentation, development tools, evaluation costs and expertise required in order to make the evaluation plan; • Execute the evaluation, the execution of the evaluation methods and the analysis of the results. However, the method proposed was not evaluated in a real case study SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  42. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey In 2003, Hissam (Hissam et al., 2003) introduced Prediction- Enabled Component Technology (PECT) as a means of packaging predictable assembly. This work, which is an evolution of Stafford et al.’s work (Stafford et al., 2001), attempts to validate the PECT and its components, giving credibility to the model SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  43. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey • During 2003, a CMU/SEI’s report, Wallnau extended Hissam work (Hissam et al., 2003), in order to achieve Predictable Assembly from Certifiable Components (PACC). • This novel model requires a better maturation by the software engineering community in order to achieve trust in it SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  44. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey Magnus Larsson, in 2004 (Larsson, 2004), define A predictability approach of the quality attributes, where one of the main objectives is to enable integration of components as black boxes. • According to composition principles, results types of attributes: • Directly compassable attributes. is a function of only the same attribute. • Architecture-related attributes. is a function of the same attribute and of the software architecture. • Derived attributes. depends on several different attributes • Usage-depended attributes. is determined by its usage profile. This work is very useful, but before the component qualitymust beknown. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  45. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey • Finally, in 2006 Daniel Karlson (Karlson et al., 2006) presented the verification of component-based embedded system designs. These techniques is Formal Methods based modeling approach(Petri net), called PRES+. • Two problems are addressed: • component verification and • Integration verification. • This approach verifies the component from only one perspective: functionality. • Formal verification, it is used only in few cases when it is mandatory, because much time and financial effort are employed. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  46. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey • Failures in Software Component Certification • Two failure cases that can be found in the literature. • First failure occurred in the US government, when trying to establish criteria for certificating components (NIAP). Thus, from 1993 until 1996, NSA and the NIST used the Trusted Computer Security Evaluation Criteria (TCSEC), “Orange Book”. • It had defined no means of features across classes of components, but only for a restricted set of behavioral assemblyproperties (Hissam et al., 2003). • The second failure happened with an IEEE committee, in an attempt to obtain a component certification standard. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  47. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: A Survey • Failures in Software Component Certification • The second failure happened with an IEEE committee, in an attempt to obtain a component certification standard. The initiative was suspended, in this same year. The committee came to a consensus that they were still far from getting to the point where the document would be a strong candidate for a standard. (Goulao et al., 2002a). SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  48. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: Standardization • One of the main objectives of software engineering • Improve the quality of software products, • Establishing methods and technologies to build software products. • The quality area could be basically divided into two main topics (Pressman, 2005): • Software Product Quality: aiming to assure the quality of the generated product; and • Software Processes Quality: looking for the definition, evaluation and improvement of software development processes. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  49. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: Standardization • Software Product Quality: • ISO/IEC 9126 (ISO/IEC 9126, 2001), • ISO/IEC 12119 (ISO/IEC 12119, 1994), • ISO/IEC 14598 (ISO/IEC 14598, 1998), • SQuaRE project (ISO/IEC 25000, 2005) (McCall et al., 1977), (Boehm et al., 1978), among others • Software Processes Quality: • Capability Maturity Model (CMM) (Paulk et al., 1993), • Capability Maturity Model Integrated (CMMI) (CMMI, 2000), • Software Process Improvement and Capability dEtermination (SPICE) (Drouin, 1995), among others SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

  50. A Embedded software component quality framework 3 – Embedded Software Component Quality and Certification: Standardization • Many institutions are creating standards to properly evaluate the quality and development processes of the software product, in different domain. • TheTable shows a set of national and international standards. SAAP 2008.1 http://www.rise.com.br Fernando Carvalho, PhD Candidate

More Related