Software Metrics

Software Metrics Testing MINIX 3

What is Metrics Analysis? Metrics Analysis is a field of Static Analysis. Show us structural attributes of the code: Depth nesting, cyclomatic number, comment/code ratio, comment frequency, number of line of code, etc... 20% of the code will cause 80% of the problems. Metrics Analysis helps to find this high risk, complex areas. All this areas are candidates for reviews and additional dynamic testing in the future. Introduction

The methodology followed by the Metrics Analysis group consists on the next steps: Choose a program to work with for making measurements in the source code. Select the most important and/or interesting files to analyze in relation with their properties, importance for the project or with the interest of the other teams in order to find in them unusual behaviors, excessive complexity, or other relevant properties. Create HTML reports, graphics, etc... about the listed files and the source code of the complete project that represent clearly the information. Methodology

The chosen tool: Understand 2.0 • Supports C/C++/C# and Java languages. • Makes possible to quickly comprehend, measure, maintain and document source code. • Handle big amounts of code. • Those are some of the features of Understand 2.0: - Fast parsing - Batch and command line parsing - Combined Language Analysis - Graphical Views - Semantic change analysis - Advanced metrics - Custom architecture creation - Creation of code analysis snapshots - API support for adding customized code checks and reports - ...

What to test? Some important MINIX3 files: • servers/init/main.c: Father of all MINIX processes. • servers/pm/main.c: Process manager. • servers/rs/main.c: Reincarnation server. • servers/cache.c, servers/cache2.c: Manage the two cache levels. • kernel/main.c: Main program of MINIX and its shutdown code. • kernel/proc.c: Process and message handling. • kernel/system/do_trace.c: Kernel call.

Why to test? Two different approaches to test MINIX 3 files: • choosing them because of their complexity, and so, more possibilities to contain bugs. • do_trace.c (kernel/system) • mii.c (drivers/fxp) • choosing them for their importance for MINIX3 or the case of study of other groups. • reincarnation server files

Analysis of do_trace.c • The kernel call is implemented in this file. • Small file (146 lines). • Most complex file according to average cyclomatic complexity. • Average cyclomatic complexity of 26. According to LaQuSo methodology these are high risk parts of the program because its average cyclomatic complexity is greater than 21.

Analysis of do_trace.c Some possible explanations: • Only one function, function metrics = average metrics. • Switch with several cases, a lot of branches, complex. • Several if statements with a lot of conditions.

Analysis of mii.c Independent (Ethernet) Interface functions: • Second most complex file in MINIX 3. • Very low comment ratio. 198 lines, 16 comment lines = 0.09 comment ratio • Complexity 22 --> Risky according to LaQuSo. • Crowded with nested if statements and in this case one switch statement . • Candidate for bugs because of it's reduced maintainability index.

Solutions These solutions are common to do_trace.c and mii.c: • Split the function or functions in many others to see if the average cyclomatic complexity changes. • Rewrite the code without using switch statement and watch the new behavior. • Try to unite some if expressions to simplify the code.

Analysis of the reincarnation server • Important part of MINIX3 ----> Reliability and robustness • Checks health of drivers and restart them if they are dead. • Composed by 3 three .c files • main.c • manager.c • service.c • We checked the reincarnation server by functions • 14 functions in total

For service.c: print_usage failure parse_arguments main For manager.c do_up do_down do_refresh do_rescue do_shutdown do_exit do_period start_service stop_service do_getsysinfo For main.c main init_server get_work reply Analysis of the reincarnation server Functions to test

We can see that service.c is by far the most complex file and also the average of complexity is the highest of all three. Analysis of the reincarnation server

Analysis of the reincarnation server Most of the code is quite simple, but: • main from main.c complexity = 16. • init_server from main.c complexity = 12. • do_period in manager.c complexity = 11. • do_up from manager.c complexity = 12. • parse_arguments from service.c complexity = 29. Interesting results from the point of view of complexity.

Analysis of the reincarnation server main complexity = 16: • This is the main routine of the reincarnation sever. • The main loop consists of three major activities: getting new work, processing the work, and sending the reply. • The loop never terminates, unless a panic occurs.

init_server = 12: Initialize the reincarnation server. Initialize the system process table, Initialize the table with the *processes in the system image Set alarm to periodically check driver status. Analysis of the reincarnation server

Analysis of the reincarnation server do_period = 11: • Marks the period for checking the state of the services and if they are not responding, or they don’t do it on time, it kills the services and send messages to solve the problem.

Analysis of the reincarnation server do_up = 12: • When a request was made to start a new system service, dismember the request message and gather all information needed to start the service • See if there is a free entry in the table with system processes and try to start the system service.

Analysis of the reincarnation server parse_arguments = 29: • This function is in charge of parse and verify correctness of arguments • Reports problems and exits if an error is found • Stores needed parameters in global variables

Analysis of the reincarnation server The most interesting function to check in future analysis like, reviews, dynamic analysis, etc.. main from main.c. • It’s not the most complex function • It plays an important role in MINIX 3 • 27 lines of comment for 47 lines of code parse_arguments from • Is quite more risky • Less important function • Full of nested if and else statements • Nine lines of comments with 105 of code

Analysis of the reincarnation server Control flow of main.c

Analysis of the reincarnation server Control flow of parse_arguments

Conclusions • Our work in this project has been focused on: • Search in the source code of MINIX3 pieces of source code where other groups are working to get metrics. • Analyze these pieces of code with Understand 2.0, creating tables and graphics related with the collected data showing the information in a simple way to draw conclusions more easily after.

Conclusions • Select real useful information for us. We rely on the importance and functionality of the piece of code we are discussing and also in the metrics obtained. • Come to certain conclusions. In some cases we have proposed any solution if we have seen room for improvements somewhere, some other times we have seen that was not worth changing anything.

Conclusions • We get interesting results (metrics) from parts of code where the other groups were working. • We tried to provide feasible solutions to the parties that we think could be improved. • We leave the parties we did not know or would not change because it was not worth looking a solution.

Conclusions In the particular case of MINIX3: • We have analyzed MINIX 3 files of the reincarnation server and process manager. • We have obtained interesting results especially in the reincarnation server because some unusually complex parts. • We tried to provide solutions to simplify these parts as was our objective.

Conclusions After our analysis we find the MINIX3 source code analyzed generally well commented. About complexity we can also state that the code maintains its average lower than 10, and it makes the code quite reliable and maintainable.

Software Metrics