SYSTEMS AND METHODS FOR FINDING PROJECT-RELATED INFORMATION BY CLUSTERING APPLICATIONS INTO RELATED CONCEPT CATEGORIES

• US 20130086553A1
• Filed: 09/29/2011
• Published: 04/04/2013
• Est. Priority Date: 09/29/2011
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Mark Grechanik
• Inventors: GRECHANIK, Mark
• Granted Patent: US 8,832,655 B2
• Time in Patent Office: Days
• US Class Current: 717/123
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/194 Calculation of difference between files
• CPC Class Codes: G06F 8/70 Software maintenance or management
• CPC Class Codes: G06F 8/71 Version control security arrangements therefor G06F21/57; Configuration management

Abstract

A system, method, and computer-readable medium, is described that finds similarities among programming applications based on semantic anchors found within the source code of such applications. The semantic anchors may be API calls, such as Java'"'"'s package and class calls of the JDK. Latent Semantic Indexing may be used to process the application and semantic anchor data and automatically develop a similarity matrix that contains numbers representing the similarity of one program to another.

Claims

1. A computer-implemented method of determining similar applications comprising:

receiving, by a computer, source code for a plurality of applications;

associating, for each application, semantic anchors found in an application with the application;

comparing, based on the semantic anchors, a first application to a second application; and

assigning, based on the comparison, a number representing the similarity of the first and second applications.

2. The method of claim 1, wherein the semantic anchors comprise Application Programming Interface (API) calls.

3. The method of claim 1, wherein associating semantic anchors comprises:

building at least one term document matrix based on the semantic anchors and source code.

4. The method of claim 3, wherein associating semantic anchors comprises:

using Latent Semantic Indexing (LSI), reducing the dimensionality of the at least one term document matrix to produce at least one matrix of application vectors.

5. The method of claim 4, wherein comparing comprises:

calculating an angle between a first application vector and a second application vector, wherein the first application vector corresponds to the first application, and wherein the second application vector corresponds to the second application; and

wherein assigning comprises;

assigning a number based on the calculated angle to the position in at least one similarity index corresponding to the first and second application.

6. The method of claim 5, wherein building at least one term document matrix comprises:

building two term document matrices, based on two different classifications of semantic anchors; and

wherein using LSI comprises;

using LSI on each term document matrix to produce two matrices of application vectors; and

wherein calculating the angle comprises;

calculating the angle between a first application vector and a second application vector, wherein each of the first and second application vectors is found within one of the two matrices of application vectors; and

calculating the angle between a third application vector and a fourth application vector, wherein each of the third and fourth application vectors is found within the other of the two matrices of application vectors, wherein the third application vector corresponds to the first application, and wherein the fourth application corresponds to the second application; and

combining the two calculated angle values into one new value.

7. The method of claim 5, comprising:

building an additional term document matrix based on words found in each source code and in documentation corresponding to each source code;

using LSI, reducing the dimensionality of the additional term document matrix to produce an additional matrix of application vectors;

calculating an additional angle between an additional first application vector and an additional second application vector, wherein the additional first and second application vectors are found within the additional matrix of application vectors, and wherein the additional first application vector corresponds to the first application, and wherein the additional second application vector corresponds to the second application; and

combining the additional angle with the calculated angle to produce a new similarity value corresponding to the similarity between the first and second applications.

8. The method of claim 6, wherein the semantic anchors comprise Application Programming Interface (API) calls.

9. The method of claim 8, wherein the API calls comprise class and package calls from the Java Development Kit (JDK), wherein class calls comprise a first semantic anchor classification and package calls comprise a second semantic anchor classification.

10. The method of claim 5, wherein the calculating is repeated such that every application source code is compared to every other application source code in the plurality of source code received;

and

wherein the assigning results in a similarity matrix, wherein the similarity matrix is characterized by rows of applications and columns of applications, wherein the value found at the intersection of a row application and a column application represents the similarity index of the row and column applications.

11. The method of claim 1, where in the associating comprises:

building at least one term document matrix based on the semantic anchors and source code, wherein each semantic anchor corresponds to rows of the matrix, and each application corresponds to columns of the matrix; and

calculating a normalized metric of each term in each application'"'"'s source code, by, for each application, dividing a number of times a particular semantic anchor appears in the application by a number of semantic anchors that appear in the application, and multiplying that quotient by a logarithm of a quotient resulting from dividing a total number of applications by a number of applications where the particular semantic anchor appears.

12. A system of calculating similarities between program applications comprising:

a non-transitory memory storing instructions; and

a processor executing the instructions to cause the system to perform a method comprising;

receiving, by a computer, source code for a plurality of applications;

associating, for each application, semantic anchors found in an application with the application;

comparing, based on the semantic anchors, a first application to a second application; and

assigning, based on the comparison, a number representing the similarity of the first and second applications.

13. The system of claim 12, wherein the semantic anchors comprise Application Programming Interface (API) calls.

14. The system of claim 12, wherein associating semantic anchors comprises:

building at least one term document matrix based on the semantic anchors and source code.

15. The system of claim 14, wherein associating semantic anchors comprises:

using Latent Semantic Indexing (LSI), reducing the dimensionality of the at least one term document matrix to produce at least one matrix of application vectors.

16. The system of claim 15, wherein comparing comprises:

calculating the angle between a first application vector and a second application vector, wherein the first application vector corresponds to the first application, and wherein the second application vector corresponds to the second application; and

wherein assigning comprises;

assigning a number based on the calculated angle to the position in at least one similarity index corresponding to the first and second application.

17. The system of claim 16, wherein building at least one term document matrix comprises:

building two term document matrices, based on two different classifications of semantic anchors; and

wherein using LSI comprises;

using LSI on each term document matrix to produce two matrices of application vectors; and

wherein calculating the angle comprises;

calculating the angle between a first application vector and a second application vector, wherein each of the first and second application vectors is found within one of the two matrices of application vectors; and

calculating the angle between a third application vector and a fourth application vector, wherein each of the third and fourth application vectors is found within the other of the two matrices of application vectors, wherein the third application vector corresponds to the first application, and wherein the fourth application corresponds to the second application; and

combining the two calculated angle values into one new value.

18. The system of claim 16, comprising:

building an additional term document matrix based on words found in each source code and in documentation corresponding to each source code;

using LSI, reducing the dimensionality of the additional term document matrix to produce an additional matrix of application vectors;

calculating an additional angle between an additional first application vector and an additional second application vector, wherein the additional first and second application vectors are found within the additional matrix of application vectors, and wherein the additional first application vector corresponds to the first application, and wherein the additional second application vector corresponds to the second application; and

combining the additional angle with the calculated angle to produce a new similarity value corresponding to the similarity between the first and second applications.

19. The system of claim 17, wherein the semantic anchors comprise Application Programming Interface (API) calls.

20. The system of claim 19, wherein the API calls comprise class and package calls from the Java Development Kit (JDK), wherein class calls comprise a first semantic anchor classification and package calls comprise a second semantic anchor classification.

21. The system of claim 16, wherein the calculating is repeated such that every application source code is compared to every other application source code in the plurality of source code received;

and

wherein the assigning results in a similarity matrix, wherein the similarity matrix is characterized by rows of applications and columns of applications, wherein the value found at the intersection of a row application and a column application represents the similarity index of the row and column applications.

22. The system of claim 12, where in the associating comprises:

building at least one term document matrix based on the semantic anchors and source code, wherein each semantic anchor corresponds to rows of the matrix, and each application corresponds to columns of the matrix; and

calculating a normalized metric of each term in each application'"'"'s source code, by, for each application, dividing a number of times a particular semantic anchor appears in the application by a number of semantic anchors that appear in the application, and multiplying that quotient by a logarithm of a quotient resulting from dividing a total number of applications by a number of applications where the particular semantic anchor appears.

23. A non-transitory computer-readable storage medium containing instructions which, when executed on a processor, perform a method comprising:

receiving, by a computer, source code for a plurality of applications;

associating, for each application, semantic anchors found in an application with the application;

comparing, based on the semantic anchors, a first application to a second application; and

assigning, based on the comparison, a number representing the similarity of the first and second applications.

24. The computer-readable storage medium of claim 23, wherein the semantic anchors comprise Application Programming Interface (API) calls.

25. The computer-readable storage medium of claim 23, wherein associating semantic anchors comprises:

building at least one term document matrix based on the semantic anchors and source code.

26. The computer-readable storage medium of claim 25, wherein associating semantic anchors comprises:

using Latent Semantic Indexing (LSI), reducing the dimensionality of the at least one term document matrix to produce at least one matrix of application vectors.

27. The computer-readable storage medium of claim 26, wherein comparing comprises:

calculating the angle between a first application vector and a second application vector, wherein the first application vector corresponds to the first application, and wherein the second application vector corresponds to the second application; and

wherein assigning comprises;

assigning a number based on the calculated angle to the position in at least one similarity index corresponding to the first and second application.

28. The computer-readable storage medium of claim 27, wherein building at least one term document matrix comprises:

building two term document matrices, based on two different classifications of semantic anchors; and

wherein using LSI comprises;

using LSI on each term document matrix to produce two matrices of application vectors; and

wherein calculating the angle comprises;

calculating the angle between a first application vector and a second application vector, wherein each of the first and second application vectors is found within one of the two matrices of application vectors; and

calculating the angle between a third application vector and a fourth application vector, wherein each of the third and fourth application vectors is found within the other of the two matrices of application vectors, wherein the third application vector corresponds to the first application, and wherein the fourth application corresponds to the second application; and

combining the two calculated angle values into one new value.

29. The computer-readable storage medium of claim 27, comprising:

building an additional term document matrix based on words found in each source code and in documentation corresponding to each source code;

using LSI, reducing the dimensionality of the additional term document matrix to produce an additional matrix of application vectors;

calculating an additional angle between an additional first application vector and an additional second application vector, wherein the additional first and second application vectors are found within the additional matrix of application vectors, and wherein the additional first application vector corresponds to the first application, and wherein the additional second application vector corresponds to the second application; and

combining the additional angle with the calculated angle to produce a new similarity value corresponding to the similarity between the first and second applications.

30. The computer-readable storage medium of claim 28, wherein the semantic anchors comprise Application Programming Interface (API) calls.

31. The computer-readable storage medium of claim 30, wherein the API calls comprise class and package calls from the Java Development Kit (JDK), wherein class calls comprise a first semantic anchor classification and package calls comprise a second semantic anchor classification.

32. The computer-readable storage medium of claim 27, wherein the calculating is repeated such that every application source code is compared to every other application source code in the plurality of source code received;

and

wherein the assigning results in a similarity matrix, wherein the similarity matrix is characterized by rows of applications and columns of applications, wherein the value found at the intersection of a row application and a column application represents the similarity index of the row and column applications.

33. The computer-readable storage medium of claim 23, where in the associating comprises:

building at least one term document matrix based on the semantic anchors and source code, wherein each semantic anchor corresponds to rows of the matrix, and each application corresponds to columns of the matrix; and

calculating a normalized metric of each term in each application'"'"'s source code, by, for each application, dividing a number of times a particular semantic anchor appears in the application by a number of semantic anchors that appear in the application, and multiplying that quotient by a logarithm of a quotient resulting from dividing a total number of applications by a number of applications where the particular semantic anchor appears.

34. A method for providing similar applications to a user comprising:

receiving from the user a search request;

sending to the user, a list of applications based on the search request;

receiving from the user a selection of one of the applications on the list;

finding related applications, based on a similarity matrix and the selection; and

sending to the user, a new list of related applications, wherein the similarity matrix is determined by a method comprising;

receiving, by a computer, source code for a plurality of applications;

associating, for each application, semantic anchors found in an application with the application;

comparing, based on the semantic anchors, a first application to a second application; and

assigning, based on the comparison, a number representing the similarity of the first and second applications.

35. The method of claim 34, wherein the semantic anchors comprise Application Programming Interface (API) calls.

36. The method of claim 34, wherein associating semantic anchors comprises:

building at least one term document matrix based on the semantic anchors and source code.

37. The method of claim 36, wherein associating semantic anchors comprises:

using Latent Semantic Indexing (LSI), reducing the dimensionality of the at least one term document matrix to produce at least one matrix of application vectors.

38. The method of claim 37, wherein comparing comprises:

calculating the angle between a first application vector and a second application vector, wherein the first application vector corresponds to the first application, and wherein the second application vector corresponds to the second application; and

wherein assigning comprises;

assigning a number based on the calculated angle to the position in at least one similarity index corresponding to the first and second application.

39. The method of claim 38, wherein building at least one term document matrix comprises:

building two term document matrices, based on two different classifications of semantic anchors; and

wherein using LSI comprises;

using LSI on each term document matrix to produce two matrices of application vectors; and

wherein calculating the angle comprises;

calculating the angle between a first application vector and a second application vector, wherein each of the first and second application vectors is found within one of the two matrices of application vectors; and

calculating the angle between a third application vector and a fourth application vector, wherein each of the third and fourth application vectors is found within the other of the two matrices of application vectors, wherein the third application vector corresponds to the first application, and wherein the fourth application corresponds to the second application; and

combining the two calculated angle values into one new value.

40. The method of claim 38, comprising:

building an additional term document matrix based on words found in each source code and in documentation corresponding to each source code;

using LSI, reducing the dimensionality of the additional term document matrix to produce an additional matrix of application vectors;

calculating an additional angle between an additional first application vector and an additional second application vector, wherein the additional first and second application vectors are found within the additional matrix of application vectors, and wherein the additional first application vector corresponds to the first application, and wherein the additional second application vector corresponds to the second application; and

combining the additional angle with the calculated angle to produce a new similarity value corresponding to the similarity between the first and second applications.

41. The method of claim 39, wherein the semantic anchors comprise Application Programming Interface (API) calls.

42. The method of claim 41, wherein the API calls comprise class and package calls from the Java Development Kit (JDK), wherein class calls comprise a first semantic anchor classification and package calls comprise a second semantic anchor classification.

43. The method of claim 38, wherein the calculating is repeated such that every application source code is compared to every other application source code in the plurality of source code received;

and

wherein the assigning results in a similarity matrix, wherein the similarity matrix is characterized by rows of applications and columns of applications, wherein the value found at the intersection of a row application and a column application represents the similarity index of the row and column applications.

44. The method of claim 34, where in the associating comprises:

building at least one term document matrix based on the semantic anchors and source code, wherein each semantic anchor corresponds to rows of the matrix, and each application corresponds to columns of the matrix; and

calculating a normalized metric of each term in each application'"'"'s source code, by, for each application, dividing a number of times a particular semantic anchor appears in the application by a number of semantic anchors that appear in the application, and multiplying that quotient by a logarithm of a quotient resulting from dividing a total number of applications by a number of applications where the particular semantic anchor appears.