Systems and methods for finding project-related information by clustering applications into related concept categories

• US 8,832,655 B2
• Filed: 09/29/2011
• Issued: 09/09/2014
• Est. Priority Date: 09/29/2011
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Grechanik, Mark
• Primary Examiner(s): Puente, Emerson
• Assistant Examiner(s): Rivera, Anibal
• Application Number:
  US13/248,877
• Publication Number:
  US 20130086553A1
• Time in Patent Office: 1,076 Days
• Field of Search: None
• US Class Current: 717/123
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/194 Calculation of difference between files
• CPC Class Codes: G06F 8/70 Software maintenance or management
• CPC Class Codes: G06F 8/71 Version control security arrangements therefor G06F21/57; Configuration management

Abstract

A system, method, and computer-readable medium, is described that finds similarities among programming applications based on semantic anchors found within the source code of such applications. The semantic anchors may be API calls, such as Java'"'"'s package and class calls of the JDK. Latent Semantic Indexing may be used to process the application and semantic anchor data and automatically develop a similarity matrix that contains numbers representing the similarity of one program to another.

Claims

1. A computer-implemented method of determining whether applications are similar, comprising:

receiving, by a computer, source code for a plurality of applications;

associating, for each application, semantic anchors found in the source code for that application with the application,

wherein associating semantic anchors comprises building at least one weighted term document matrix from the semantic anchors, the at least one weighted term document matrix comprising at least a first term weighted based on at least a number of the plurality of applications in which a first semantic anchor is present in the source code for those applications;

comparing, based on the semantic anchors, a similarity of the first application to a second application; and

assigning, based on the comparison, a number representing the similarity of the first and second applications.

2. The method of claim 1, wherein the semantic anchors comprise Application Programming Interface (API) calls.

3. The method of claim 1, wherein associating semantic anchors comprises:

using Latent Semantic Indexing (LSI), reducing the dimensionality of the at least one weighted term document matrix to produce at least one matrix of application vectors.

4. The method of claim 3,

wherein comparing comprises:

calculating the cosine between a first application vector and a second application vector,

wherein the first application vector corresponds to the first application, and wherein the second application vector corresponds to the second application; and

wherein assigning comprises;

assigning the calculated cosine to the position in at least one similarity index corresponding to the first and second application.

5. The method of claim 4,

wherein building at least one weighted term document matrix comprises:

building two weighted term document matrices, based on two different classifications of semantic anchors; and

wherein using LSI comprises;

using LSI on each weighted term document matrix to produce two matrices of application vectors; and

wherein calculating the cosine comprises;

calculating the cosine between a first application vector and a second application vector,

wherein each of the first and second application vectors is found within one of the two matrices of application vectors; and

calculating the cosine between a third application vector and a fourth application vector,

wherein each of the third and fourth application vectors is found within the other of the two matrices of application vectors, wherein the third application vector corresponds to the first application, and

wherein the fourth application corresponds to the second application; and

combining the two calculated cosine values into one new value.

6. The method of claim 4, comprising:

building an additional term document matrix based on words found in each source code and in documentation corresponding to each source code;

using LSI, reducing the dimensionality of the additional term document matrix to produce an additional matrix of application vectors;

calculating an additional cosine between an additional first application vector and an additional second application vector,

wherein the additional first and second application vectors are found within the additional matrix of application vectors, and

wherein the additional first application vector corresponds to the first application, and

wherein the additional second application vector corresponds to the second application; and

combining the additional cosine with the calculated cosine to produce a new similarity value corresponding to the similarity between the first and second applications.

7. The method of claim 5, wherein the semantic anchors comprise Application Programming Interface (API) calls.

8. The method of claim 7, wherein the API calls comprise class and package calls from the Java Development Kit (JDK),

wherein class calls comprise a first semantic anchor classification and package calls comprise a second semantic anchor classification.

9. The method of claim 4,

wherein the calculating is repeated such that every application source code is compared to every other application source code in the plurality of source code received;

and

wherein the assigning results in a similarity matrix,

wherein the similarity matrix is characterized by rows of applications and columns of applications,

wherein the value found at the intersection of a row application and a column application represents the similarity index of the row and column applications.

10. The method of claim 1, wherein the associating comprises:

associating each semantic anchor with rows of the weighted term document matrix and each application with columns of the weighted term document matrix; and

calculating a normalized metric of each semantic anchor in each application'"'"'s source code, by, for each application, dividing a number of times a particular semantic anchor appears in the application by a number of semantic anchors that appear in the application, and multiplying that quotient by a logarithm of a quotient resulting from dividing a total number of applications by a number of applications where the particular semantic anchor appears.

11. The method of claim 1, wherein the first term is weighted lower when the first semantic anchor is present in at least a majority of the plurality of applications relative to when the first semantic anchor is present in fewer than a majority of the plurality of applications.

12. A system of calculating whether program applications have similarities, comprising:

a non-transitory memory storing instructions; and

a processor executing the instructions to cause the system to perform a method comprising;

receiving, by a computer, source code for a plurality of applications;

associating, for each application, semantic anchors found in the source code for that application with the application,

wherein associating semantic anchors comprises building at least one weighted term document matrix from the semantic anchors and source code, the at least one weighted term document matrix comprising at least a first term weighted based on at least a number of the plurality of applications in which a first semantic anchor is present in the source code for those applications;

comparing, based on the semantic anchors, a similarity of the first application to a second application; and

assigning, based on the comparison, a number representing the similarity of the first and second applications.

13. The system of claim 12, wherein the semantic anchors comprise Application Programming Interface (API) calls.

14. The system of claim 12, wherein associating semantic anchors comprises:

using Latent Semantic Indexing (LSI), reducing the dimensionality of the at least one weighted term document matrix to produce at least one matrix of application vectors.

15. The system of claim 14,

wherein comparing comprises:

calculating the cosine between a first application vector and a second application vector,

wherein the first application vector corresponds to the first application, and

wherein the second application vector corresponds to the second application; and

wherein assigning comprises;

assigning the calculated cosine to the position in at least one similarity index corresponding to the first and second application.

16. The system of claim 15,

wherein building at least one weighted term document matrix comprises:

building two weighted term document matrices, based on two different classifications of semantic anchors; and

wherein using LSI comprises;

using LSI on each weighted term document matrix to produce two matrices of application vectors; and

wherein calculating the cosine comprises;

calculating the cosine between a first application vector and a second application vector,

wherein each of the first and second application vectors is found within one of the two matrices of application vectors; and

calculating the cosine between a third application vector and a fourth application vector,

wherein each of the third and fourth application vectors is found within the other of the two matrices of application vectors,

wherein the third application vector corresponds to the first application, and

wherein the fourth application corresponds to the second application; and

combining the two calculated cosine values into one new value.

17. The system of claim 15, comprising:

building an additional term document matrix based on words found in each source code and in documentation corresponding to each source code;

using LSI, reducing the dimensionality of the additional term document matrix to produce an additional matrix of application vectors;

calculating an additional cosine between an additional first application vector and an additional second application vector,

wherein the additional first and second application vectors are found within the additional matrix of application vectors, and

wherein the additional first application vector corresponds to the first application, and

wherein the additional second application vector corresponds to the second application; and

combining the additional cosine with the calculated cosine to produce a new similarity value corresponding to the similarity between the first and second applications.

18. The system of claim 16, wherein the semantic anchors comprise Application Programming Interface (API) calls.

19. The system of claim 18,

wherein the API calls comprise class and package calls from the Java Development Kit (JDK),

wherein class calls comprise a first semantic anchor classification and package calls comprise a second semantic anchor classification.

20. The system of claim 15,

wherein the calculating is repeated such that every application source code is compared to every other application source code in the plurality of source code received;

and

wherein the assigning results in a similarity matrix,

wherein the similarity matrix is characterized by rows of applications and columns of applications,

wherein the value found at the intersection of a row application and a column application represents the similarity index of the row and column applications.

21. The system of claim 12, wherein the associating comprises:

associating each semantic anchor with rows of the weighted term document matrix and each application with columns of the weighted term document matrix; and

calculating a normalized metric of each semantic anchor in each application'"'"'s source code, by, for each application, dividing a number of times a particular semantic anchor appears in the application by a number of semantic anchors that appear in the application, and multiplying that quotient by a logarithm of a quotient resulting from dividing a total number of applications by a number of applications where the particular semantic anchor appears.

22. The system of claim 12, wherein the first term is weighted lower when the first semantic anchor is present in at least a majority of the plurality of applications relative to when the first semantic anchor is present in fewer than a majority of the plurality of applications.

23. A non-transitory computer-readable storage medium containing instructions which, when executed on a processor, perform a method comprising:

receiving, by a computer, source code for a plurality of applications;

associating, for each application, semantic anchors found in the source code for that application with the application,

wherein associating semantic anchors comprises building at least one weighted term document matrix from the semantic anchors, the at least one weighted term document matrix comprising at least a first term weighted based on at least a number of the plurality of applications in which a first semantic anchor is present in the source code for those applications;

comparing, based on the semantic anchors, a similarity of the first application to a second application; and

assigning, based on the comparison, a number representing the similarity of the first and second applications.

24. The computer-readable storage medium of claim 23, wherein the semantic anchors comprise Application Programming Interface (API) calls.

25. The computer-readable storage medium of claim 23, wherein associating semantic anchors comprises:

using Latent Semantic Indexing (LSI), reducing the dimensionality of the at least one weighted term document matrix to produce at least one matrix of application vectors.

26. The computer-readable storage medium of claim 25,

wherein comparing comprises:

calculating the cosine between a first application vector and a second application vector,

wherein the first application vector corresponds to the first application, and

wherein the second application vector corresponds to the second application; and

wherein assigning comprises;

assigning the calculated cosine to the position in at least one similarity index corresponding to the first and second application.

27. The computer-readable storage medium of claim 26,

wherein building at least one weighted term document matrix comprises:

building two weighted term document matrices, based on two different classifications of semantic anchors; and

wherein using LSI comprises;

using LSI on each weighted term document matrix to produce two matrices of application vectors; and

wherein calculating the cosine comprises;

calculating the cosine between a first application vector and a second application vector,

wherein each of the first and second application vectors is found within one of the two matrices of application vectors; and

calculating the cosine between a third application vector and a fourth application vector,

wherein each of the third and fourth application vectors is found within the other of the two matrices of application vectors,

wherein the third application vector corresponds to the first application, and

wherein the fourth application corresponds to the second application; and

combining the two calculated cosine values into one new value.

28. The computer-readable storage medium of claim 26, comprising:

building an additional term document matrix based on words found in each source code and in documentation corresponding to each source code;

using LSI, reducing the dimensionality of the additional term document matrix to produce an additional matrix of application vectors;

calculating an additional cosine between an additional first application vector and an additional second application vector,

wherein the additional first and second application vectors are found within the additional matrix of application vectors, and

wherein the additional first application vector corresponds to the first application, and

wherein the additional second application vector corresponds to the second application; and

combining the additional cosine with the calculated cosine to produce a new similarity value corresponding to the similarity between the first and second applications.

29. The computer-readable storage medium of claim 27, wherein the semantic anchors comprise Application Programming Interface (API) calls.

30. The computer-readable storage medium of claim 29,

wherein the API calls comprise class and package calls from the Java Development Kit (JDK),

wherein class calls comprise a first semantic anchor classification and package calls comprise a second semantic anchor classification.

31. The computer-readable storage medium of claim 26,

wherein the calculating is repeated such that every application source code is compared to every other application source code in the plurality of source code received;

and

wherein the assigning results in a similarity matrix,

wherein the similarity matrix is characterized by rows of applications and columns of applications,

wherein the value found at the intersection of a row application and a column application represents the similarity index of the row and column applications.

32. The computer-readable storage medium of claim 23, wherein the associating comprises:

associating each semantic anchor with rows of the weighted term document matrix and each application with columns of the weighted term document matrix; and

calculating a normalized metric of each semantic anchor in each application'"'"'s source code, by, for each application, dividing a number of times a particular semantic anchor appears in the application by a number of semantic anchors that appear in the application, and multiplying that quotient by a logarithm of a quotient resulting from dividing a total number of applications by a number of applications where the particular semantic anchor appears.

33. The computer-readable storage medium of claim 23, wherein the first term is weighted lower when the first semantic anchor is present in at least a majority of the plurality of applications relative to when the first semantic anchor is present in fewer than a majority of the plurality of applications.

34. A method for providing similar applications to a user, comprising:

receiving from the user a search request;

sending to the user, a first list of applications based on the search request;

receiving from the user a selection of one of the applications on the first list;

finding related applications, based on a similarity matrix and the selection; and

sending to the user, a second list of related applications, wherein the similarity matrix is determined by a method comprising;

receiving, by a computer, source code for a plurality of applications;

associating, for each application, semantic anchors found in the source code for that application with the application,

wherein associating semantic anchors comprises building at least one weighted term document matrix from the semantic anchors, the at least one weighted term document matrix comprising at least a first term weighted based on at least a number of the plurality of applications in which a first semantic anchor is present in the source code for those applications;

comparing, based on the semantic anchors, a similarity of the first application to a second application; and

assigning, based on the comparison, a number representing the similarity of the first and second applications.

35. The method of claim 34, wherein the semantic anchors comprise Application Programming Interface (API) calls.

36. The method of claim 34, wherein associating semantic anchors comprises:

using Latent Semantic Indexing (LSI), reducing the dimensionality of the at least one weighted term document matrix to produce at least one matrix of application vectors.

37. The method of claim 36,

wherein comparing comprises:

calculating the cosine between a first application vector and a second application vector,

wherein the first application vector corresponds to the first application, and

wherein the second application vector corresponds to the second application; and

wherein assigning comprises;

assigning the calculated cosine to the position in at least one similarity index corresponding to the first and second application.

38. The method of claim 37,

wherein building at least one weighted term document matrix comprises:

building two weighted term document matrices, based on two different classifications of semantic anchors; and

wherein using LSI comprises;

using LSI on each weighted term document matrix to produce two matrices of application vectors; and

wherein calculating the cosine comprises;

calculating the cosine between a first application vector and a second application vector,

wherein each of the first and second application vectors is found within one of the two matrices of application vectors; and

calculating the cosine between a third application vector and a fourth application vector,

wherein each of the third and fourth application vectors is found within the other of the two matrices of application vectors,

wherein the third application vector corresponds to the first application, and

wherein the fourth application corresponds to the second application; and

combining the two calculated cosine values into one new value.

39. The method of claim 37, comprising:

building an additional term document matrix based on words found in each source code and in documentation corresponding to each source code;

using LSI, reducing the dimensionality of the additional term document matrix to produce an additional matrix of application vectors;

calculating an additional cosine between an additional first application vector and an additional second application vector,

wherein the additional first and second application vectors are found within the additional matrix of application vectors, and

wherein the additional first application vector corresponds to the first application, and

wherein the additional second application vector corresponds to the second application; and

combining the additional cosine with the calculated cosine to produce a new similarity value corresponding to the similarity between the first and second applications.

40. The method of claim 38, wherein the semantic anchors comprise Application Programming Interface (API) calls.

41. The method of claim 40,

wherein the API calls comprise class and package calls from the Java Development Kit (JDK),

wherein class calls comprise a first semantic anchor classification and package calls comprise a second semantic anchor classification.

42. The method of claim 37,

wherein the calculating is repeated such that every application source code is compared to every other application source code in the plurality of source code received;

and

wherein the assigning results in a similarity matrix,

wherein the similarity matrix is characterized by rows of applications and columns of applications,

wherein the value found at the intersection of a row application and a column application represents the similarity index of the row and column applications.

43. The method of claim 34, wherein the associating comprises:

associating each semantic anchor with rows of the weighted term document matrix and each application with columns of the weighted term document matrix; and

calculating a normalized metric of each semantic anchor in each application'"'"'s source code, by, for each application, dividing a number of times a particular semantic anchor appears in the application by a number of semantic anchors that appear in the application, and multiplying that quotient by a logarithm of a quotient resulting from dividing a total number of applications by a number of applications where the particular semantic anchor appears.

44. The method of claim 34, wherein the first term is weighted lower when the first semantic anchor is present in at least a majority of the plurality of applications relative to when the first semantic anchor is present in fewer than a majority of the plurality of applications.