1 / 27

Gemini: Code Clone Analysis Tool

Gemini: Code Clone Analysis Tool. Yasushi Ueda † , Yoshiki Higo ‡ , Toshihiro Kamiya*, Shinji Kusumoto ‡ , and Katsuro Inoue ‡. †Graduate School of Engineering Science, Osaka Univ., Japan ‡ Graduate School of Information Science and Technology, Osaka Univ., Japan

lenci
Télécharger la présentation

Gemini: Code Clone Analysis Tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Gemini: Code Clone Analysis Tool Yasushi Ueda†, Yoshiki Higo‡, Toshihiro Kamiya*,Shinji Kusumoto‡, and Katsuro Inoue‡ †Graduate School of Engineering Science, Osaka Univ., Japan‡Graduate School of Information Science and Technology, Osaka Univ., Japan *PRESTO, Japan Science and Technology Corp., Japan {y-ueda, y-higo, kamiya, kusumoto, inoue}@ist.osaka-u.ac.jp

  2. Contents • Background • Code Clone Analysis Tool, Gemini • Overview • System structure • Scatter Plot

  3. Background (1/2) • A code clone is a pair/set of code portions in source files that are identical or similar to each other.

  4. We have developed a code clone detection tool, CCFinder [1]. • Token-based clone detector • Its input is a set of source files and output is the locations of clone pairs. [1] T. Kamiya, S. Kusumoto, and K. Inoue, “CCFinder: A multi-linguistic token-based code clone detection system for large scale source code”, IEEE Transactions on Software Engineering, 28(7):654-670, 2002. Background (2/2) • Code clone is one of the factors that make software maintenance more difficult. • If some faults are found in a code portion, it is necessary to correct the faults in its all clone pairs.

  5. Source files Lexical analysis Token sequence Transformation Transformed token sequence Match detection Clones on transformed sequence Formatting 0.1 3,1 9,1 11,1 17,1 Clone pairs 1. static void foo() throws RESyntaxException { 2.String a[] = new String [] { "123,400", "abc", "orange 100" }; 3.org.apache.regexp.RE pat = new org.apache.regexp.RE("[0-9,]+"); 4. int sum = 0; 5. for (int i = 0; i < a.length; ++i) 6. if (pat.match(a[i])) 7. sum += Sample.parseNumber(pat.getParen(0)); 8. System.out.println("sum = " + sum); 9. } 10. static void goo(String [] a) throws RESyntaxException { 11. RE exp = new RE("[0-9,]+"); 12. int sum = 0; 13. for (int i = 0; i < a.length; ++i) 14. if (exp.match(a[i])) 15. sum += parseNumber(exp.getParen(0)); 16. System.out.println("sum = " + sum); 17. } Lexical analysis Lexical analysis Lexical analysis Token sequence Token sequence Token sequence Transformation Transformation Transformation Transformed token sequence Transformed token sequence Transformed token sequence Match detection Match detection Match detection Clones on transformed sequence Clones on transformed sequence Clones on transformed sequence Formatting Formatting Formatting CCFinder • Example of clone detection process 1. static void foo() throws RESyntaxException { 2. String a[] = new String [] { "123,400", "abc", "orange 100" }; 3. org.apache.regexp.RE pat = new org.apache.regexp.RE("[0-9,]+"); 4. int sum = 0; 5. for (int i = 0; i < a.length; ++i) 6. if (pat.match(a[i])) 7. sum += Sample.parseNumber(pat.getParen(0)); 8. System.out.println("sum = " + sum); 9. } 10. static void goo(String [] a) throws RESyntaxException { 11. RE exp = new RE("[0-9,]+"); 12. int sum = 0; 13. for (int i = 0; i < a.length; ++i) 14. if (exp.match(a[i])) 15. sum += parseNumber(exp.getParen(0)); 16. System.out.println("sum = " + sum); 17. }

  6. Gemini overview • A GUI-based code clone analysis tool • Uses CCFinder as a code clone detector. • Has several views to interactive analysis. • Scatter plot view • Select by mouse dragging • Sorting function • Zoom in/out • Metric graph view • Select by metric values • Source code view • Implemented in Java • About 10,000 lines of code

  7. Scatter plot • Both the vertical and horizontal axes represent a token sequence of source code. • A dot means that corresponding two tokens on the two axes are same. • The main diagonal line is always drawn, because each dot on it refers to an identical position of the two axes. • A clone pair is shown as a diagonal line segment. • The distribution is symmetrical with the main diagonal line. a b c a b c a d e c a b c a b c a d e c a, b, c, ... : tokens : matched position

  8. Sorting function • When multiple files are compared in scatter plot, boundaries of their files are shown on the axes. • Depending on the file orders, the distribution of dots is spread widely. • We put similar files as near as possible.

  9. Snapshots of Gemini

  10. Conclusions • We presented a maintenance support environment based on code clone analysis, Gemini. • We are going to evaluate the applicability to large scale softwares in actual maintenance as future research work.

  11. CCFinder: Implementation • CCFinder extracts code clones by direct comparison of source text. • It transforms source text for precise and effective detection of code clones. • Token-based transformation rules to regularize and select code portion, for Java, C++, COBOL, etc. programs • It uses an effective matching algorithm for large source code. • Complexity of algorithm: O(n), where n is a length of source code • Scalability: 108 min. for 7.2 million lines (Pentium III 650 MHz, 640MB memory)

  12. The difference between ‘diff’ and clone detection tools • Diff finds the longest common sub-string. • Given a code portion, diff does not report two or more same code portions (clones). • Clone detection tool finds all the same or similar code portions.

  13. Example of transformation rules in Java • All identifiers defined by user are transformed to same tokens. • Unique identifier is inserted at each end of the top-level definitions and declarations. • Prevents detecting clones that begin at the middle of class definition and end at the middle of another one. • ”java. lang. Math. PI” is transformed to ”Math. PI”. • By using import sentence, a class is referred to with either full package name or a shorter name • ” new int[] {1, 2, 3} ” is transformed to ” new int[] {$} ” • Eliminates table initialization code.

  14. DFL (C ): Estimation of how many tokens would be removed from source files when all code fragments of clone class C are replaced with caller statements of a new identical routine new sub routine caller statements Clone class metrics • LEN (C ): Length of token sequence of each element in clone class C • POP (C ): Number of elements in clone class C • RAD (C ): Distribution in the file system of elements in clone class C

  15. Snapshots of clone class metric graph LEN POP RAD DFL Filtering mode : ON

  16. Aims of clone class metrics • We are interested in • Clone classes whose elements are spread widely. • High value of POP means that there are many similar code fragments. • High value of RAD means that the clones are spread over many subsystems. They are difficult to find all together in maintenance. • Clone classes which are appropriate for refactoring. • High value of DFL (high value POP and high value of LEN) means that the clone class is worth evaluating whether the elements can be merged into one routine.

  17. new sub routine caller statements Definition of DFL and RAD • DFL(C ) • DFL(C) = LEN(C) ×POP(C) - 5×POP(C) + LEN(C) • LEN(C) ×POP(C) : the target code size for restructuring • 5×POP(C) : the code size of new caller statements • LEN(C) : the code size of new identical routine • RAD (C ) • Distribution in the file system of elements in clone class C • RAD(C) = 0 : C is enclosed within a single file. • RAD(C) = 1 : C is enclosed within a single directory. • RAD(C) = n : C is enclosed within a directory tree of n layers.

  18. CCFinder (3/4) • Application of CCFinder • Free software • JDK libraries (Java, 570 KLOC) • Linux, FreeBSD (C, 1.6 + 1.3 MLOC) • FreeBSD, OpenBSD，NetBSD(C) • Qt(C++，240KLOC) • Commercial software • NTT data Corp., Hitachi Ltd., NEC soft Ltd., ASTEC Inc., SRA Inc. • NASDA (Control program for rocket)

  19. Object file ID( file 0 in Group 0 ) Location of a clone pair ( Lines 53 - 63 in file 0.1 and Lines 542 - 553 in file 1.10 are identical or similar to each other) CCFinder (4/4) #version: ccfinder 3.1 #langspec: JAVA #option: -b 30,1 #option: -k + #option: -r abcdfikmnprsv #option: -c wfg #begin{file description} 0.0 52 C:\Gemini.java 0.1 94 C:\GeneralManager.java : : #end{file description} #begin{clone} 0.1 53,9 63,13 1.10 542,9 553,13 35 0.1 53,9 63,13 1.10 624,9 633,13 35 0.2 124,9 152,31 0.2 154,9 216,51 42 : : #end{clone} • Output of CCFinder • It is difficult to analyze source code by only this text-based information of the location of clone pairs.

  20. Gemini CCFinder User Code clone detector Source files System structure of Gemini User Interfaces Clone pair manager Scatter plot view Clone selection information Source code manager Source code view Clone selection information Code clone database Metrics manager Metric graph views

  21. Source files 0.1 3,1, 9,1 11,1 17,1 Lexical analysis Token sequence Transformation Transformed token sequence Lexical analysis Match detection Token sequence Clones on transformed sequence Transformation Formatting Transformed token sequence Clone pairs Match detection Clones on transformed sequence 1. static void foo() throws RESyntaxException { 2.String a[] = new String [] { "123,400", "abc", "orange 100" }; 3.org.apache.regexp.RE pat = new org.apache.regexp.RE("[0-9,]+"); 4. int sum = 0; 5. for (int i = 0; i < a.length; ++i) 6. if (pat.match(a[i])) 7. sum += Sample.parseNumber(pat.getParen(0)); 8. System.out.println("sum = " + sum); 9. } 10. static void goo(String [] a) throws RESyntaxException { 11. RE exp = new RE("[0-9,]+"); 12. int sum = 0; 13. for (int i = 0; i < a.length; ++i) 14. if (exp.match(a[i])) 15. sum += parseNumber(exp.getParen(0)); 16. System.out.println("sum = " + sum); 17. } Formatting Lexical analysis Token sequence Lexical analysis Transformation Token sequence Transformed token sequence Transformation Match detection Transformed token sequence Clones on transformed sequence Match detection Formatting Clones on transformed sequence Formatting CCFinder • Example of clone detection process 1. static void foo() throws RESyntaxException { 2. String a[] = new String [] { "123,400", "abc", "orange 100" }; 3. org.apache.regexp.RE pat = new org.apache.regexp.RE("[0-9,]+"); 4. int sum = 0; 5. for (int i = 0; i < a.length; ++i) 6. if (pat.match(a[i])) 7. sum += Sample.parseNumber(pat.getParen(0)); 8. System.out.println("sum = " + sum); 9. } 10. static void goo(String [] a) throws RESyntaxException { 11. RE exp = new RE("[0-9,]+"); 12. int sum = 0; 13. for (int i = 0; i < a.length; ++i) 14. if (exp.match(a[i])) 15. sum += parseNumber(exp.getParen(0)); 16. System.out.println("sum = " + sum); 17. }

  22. Suffix-tree • Suffix tree is a tree that satisfies the following conditions. • A leaf node represents the starting position of sub-string. • A path from root node to a leaf node represents a sub-string. • First characters of labels of all the edges from one node are different from each other. → A common path means a clone

  23. Case study overview • Application target • Programs developed in a programming exercise of Osaka Univ. • Compiler in C language • Programs of 69 students • Total size is 360,000 lines of code • Issue of Analysis • Similarity among all programs • In the programming exercise, plagiarisms sometimes happen.

  24. Analysis (1/2) • Compiler of 69 students are arranged on the two axes. • The distribution is spread widely. • Rearrangement of scatter plot using sorting function • The grid represents boundary lines between individuals.

  25. B A Analysis (2/2) • The corresponding code • A (2 students) • Similar code fragments were from source code of sample compiler described in textbook. • B (4 students) • Many code fragments were similar even with respect to name of variables or comments.

  26. f1 f2 f3 f4 f1 f2 f3 f4 f5 f6 f5 Step2:From among the remaining files, select the most similar file to F and put it next toF by the value of RST f6 RST(i,j) : Ratio of covered code range in file i by clones between a file i and a file j f1 f5 f4 f4 f2 f1 f1 f1 f1 f6 f6 f6 f6 f3 f3 f3 f1 f1 f1 f1 f1 f6 f6 f6 f6 f3 f3 f3 f4 f4 f2 f5 Sorting function Step1:Select a head file by the value of RSA(Make F the head file) RSA(i) : Ratio of covered code range in file i by clones between one file iof other files Step3:Repeat step2 recursively while any file remains, treating the most similar file in previous step2 as new F

More Related