Noise reduction for content matching analysis results for protectable content

• US 8,498,982 B1
• Filed: 07/07/2010
• Issued: 07/30/2013
• Est. Priority Date: 07/07/2010
• Status: Active Grant

• Current Assignee: Rogue Wave Software Incorporated (Perforce Software Incorporated)
• Original Assignee: OpenLogic, Inc. (Perforce Software Incorporated)
• Inventors: Cope, Rod
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): VU, THONG H
• Application Number:
  US12/831,775
• Time in Patent Office: 1,119 Days
• Field of Search: 707/3, 707/6, 707/104, 707/769, 707/694, 707/692, 707/952, 600/407, 600/413, 726/32
• US Class Current: 707/722
• CPC Class Codes: G06F 16/2457 with adaptation to user needs
• CPC Class Codes: G06F 16/80 of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML content-based retrieval of web data G06F16/95
• CPC Class Codes: G06F 21/16 Program or content traceability, e.g. by watermarking

Abstract

A system and method (a “utility”) is provided for improving the accuracy of a content matching analysis that identifies a composition of an item of protectable content of a user. The item of protectable content may include a portion of source code or object code, individual or bundled source code or object code files, binary code files, directory structures and/or trees, open source software projects or packages, and/or proprietary software applications or packages. The utility involves storing a number of items of comparison content on a storage structure, receiving an item of user content at a computer-based content exchange, and comparing the item of user content to the items of comparison content to determine, from among the items of comparison content, one or more potential matches that each include a matched portion that is similar to a portion of the item of user content. The utility further includes selecting a noise reduction technique from a number of noise reduction techniques available to the content exchange and applying the noise reduction technique to eliminate noise and/or false positives (e.g., one or more redundant matches and/or erroneously identified matches) from the potential matches.

Claims

1. A method for refining an indication of a composition of protectable content, comprising:

storing, in a memory structure, a number of items of comparison content comprising at least one item of open source software content;

providing, on a computer-based content exchange, instructions regarding a plurality of noise reduction techniques;

receiving, at said computer-based content exchange, an item of user content;

comparing said item of user content to said items of comparison content;

determining, from among said items of comparison content, an identity of one or more potential matches, wherein each said potential match includes a matched portion that is similar to a portion of said item of user content;

selecting, using said computer-based content exchange, at least one of said plurality of noise reduction techniques for application to said potential matches; and

applying, using said computer-based content exchange, said at least one selected noise reduction technique to eliminate noise from said potential matches.

2. A method as set forth in claim 1, wherein each said item of comparison content and said item of user content comprise one or more portions of source code, one or more portions of binary code, one or more source code files or binary code files, or one or more directory structures, software projects, software applications, or software packages.

3. A method as set forth in claim 1, wherein said noise comprises one or more false positives.

4. A method as set forth in claim 1, wherein said noise comprises one or more redundant potential matches.

5. A method as set forth in claim 1, wherein said noise comprises one or more erroneously identified potential matches.

6. A method as set forth in claim 1, further comprising iteratively repeating said selecting and applying steps.

7. A method as set forth in claim 6, wherein said iteratively repeating step comprises iteratively repeating said selecting and applying steps for a defined time-period or for a defined number of iterations.

8. A method as set forth in claim 6, wherein said iteratively repeating step comprises iteratively repeating said selecting and applying steps until said potential matches comprise a set of positively identified confirmed matches, and wherein said instructions regarding said noise reduction techniques are integrated such that each said confirmed match is unique.

9. A method as set forth in claim 8, further comprising providing, via a user interface structure coupled with said computer-based content exchange, an identification of said confirmed matches.

10. A method as set forth in claim 9, further comprising providing, via said user interface structure, information relating to said confirmed matches, wherein said information comprises an identification of one or more of software packages, a number of files contained within each said software package, names of said files, sizes of said files, software creation dates, software revisions dates, software versions, directory structures, license information, use obligations, and conflict information.

11. A method as set forth in claim 10, wherein said license information comprises license types, license restrictions, and license obligations.

12. A method as set forth in claim 1, further comprising:

obtaining providence information relating to said items of comparison content, wherein said providence information comprises information relating to a network of interdependencies and interrelationships between said items of comparison content, and wherein said applying step comprises using said providence information in conjunction with said selected noise reduction technique to eliminate said noise from said potential matches.

13. A method as set forth in claim 12, wherein said providence information identifies dependencies between said items of comparison content, versions of said items of comparison content, plug-ins between said items of comparison content, and parent projects and child projects of said items of comparison content.

14. A method as set forth in claim 12, wherein said applying said selected noise reduction technique comprises:

using said providence information, operating said computer-based content exchange to determine which of said potential matches to which said matched portion is native and which of said potential matches to which said matched portion is redundant; and

eliminating said potential matches to which said matched portion is redundant.

15. A method as set forth in claim 12, wherein said applying said selected noise reduction technique comprises:

using said providence information, operating said computer-based content exchange to determine which of said potential matches is an original source of said matched portion and which of said potential matches are redundant sources of said matched portion; and

eliminating said redundant sources.

16. A method as set forth in claim 12, wherein said applying said selected noise reduction technique comprises:

using said providence information, operating said computer-based content exchange to identify duplicates within said potential matches; and

eliminating said duplicates.

17. A method as set forth in claim 12, wherein said applying said selected noise reduction technique comprises:

analyzing naming patterns associated with said items of comparison content, including said potential matches selected from among said items of comparison content;

using said naming patterns and said providence information, operating said computer-based content exchange to identify a first tier potential match and one or more second tier potential matches, wherein said second tier potential matches are nested within said first tier potential match;

when all of said items of comparison content nested within said first tier potential match are included in said potential matches, eliminating said second tier potential matches; and

when said second tier potential matches form a portion of said items of comparison content nested within said first tier potential match, eliminating said first tier potential match.

18. A method as set forth in claim 12, wherein said applying said selected noise reduction technique comprises:

weighting said providence information according to a level of usage associated with each said item of comparison content;

applying said providence information to said potential matches such that said potential matches are biased toward a center of said network of interrelationships and interdependencies; and

eliminating ones of said potential matches that, after said applying step, remain substantially outside said center of said network of interrelationships and interdependencies.

19. A method as set forth in claim 12, wherein said applying said selected noise reduction technique comprises:

independently determining an identification of one or more license types that apply to said item of user content;

using said providence information, operating said computer-based content exchange to identify associations between ones of said license types and ones of said potential matches; and

eliminating one or more of said potential matches that lack an association with any of said license types.

20. A method as set forth in claim 1, further comprising adaptively adjusting a threshold size of said matched portion, wherein said matched portion meets or exceeds said threshold size.

21. A method as set forth in claim 20, wherein said threshold size is defined by a number of lines of software code.

22. A method as set forth in claim 20, wherein said adaptively adjusting step occurs a defined number of times or over a defined period of time.

23. A method as set forth in claim 20, wherein said adaptively adjusting step is automated.

24. A method as set forth in claim 1, wherein said applying said selected noise reduction technique comprises:

examining a content of said item of user content or of one of said potential matches for a recitation of a name of said one of said potential matches; and

when said name of said one of said potential matches is not recited in either of said content of said item of said user content or said content of said one of said potential matches, eliminating said one of said potential matches.

25. A method as set forth in claim 24, wherein said names comprise one or more of file names, project names, package names, or license names.

26. A method as set forth in claim 1, wherein said applying said selected noise reduction technique comprises:

first applying one of said noise reduction techniques to eliminate one or more of said potential matches; and

second applying one or more conventional content matching techniques to any remaining potential matches, wherein said conventional content matching techniques comprise comparing file names, project names, package names, or directory structures associated with each said remaining potential match to a name of each said remaining potential match to positively identify one or more confirmed matches from among said remaining potential matches.

27. A method as set forth in claim 1, wherein said comparing said file names, project names, package names, or directory structures associated with each said remaining potential match reveals whether each said remaining potential matches is internally consistent or internally inconsistent.

28. A method as set forth in claim 1, wherein said applying said selected noise reduction technique comprises:

directly comparing said matched portion of one or more of said potential matches with said portion of said item of user content; and

using said computer-based content exchange to selectively eliminate ones of said potential matches in response to said directly comparing step.

29. A method as set forth in claim 28, wherein said matched portion of each said potential match and said portion of said item of user content comprise software code windows or local directory structures.

30. The method of claim 28, wherein said applying said noise reduction technique comprises, for at least one of said potential matches, analysis of said at least one potential match independent from others of said potential matches.

31. A method as set forth in claim 1, wherein said steps of comparing, determining, selecting, and applying are controlled using a plurality of adjustable system controls.

32. A method as set forth in claim 31, wherein said adjustable system controls are manually tunable or automatically optimizable.

33. A method as set forth in claim 1, further comprising:

retaining information relating to choices made in connection with at least one of said comparing, determining, selecting, and applying steps; and

using said information relating to said choices, operating said computer-based exchange to inform subsequent repetitions of said receiving, comparing, determining, selecting, and applying steps.

34. A method as set forth in claim 33, wherein said choices are manual or automated.

35. A method as set forth in claim 1, further comprising:

storing, in said memory structure, license information associated with said items of comparison content;

obtaining use information associated with said item of user content and information about a policy of a user regarding use of protectable content; and

using said policy, said license information, and said use information, operating said computer-based content exchange to classify said item of user content in relation to said policy.

36. A method as set forth in claim 35, wherein said license information identifies one or more license provisions regarding one of a license type, license restrictions, and license obligations.

37. A method as set forth in claim 35, further comprising storing package information associated with said items of comparison content, and wherein said operating said computer-based content exchange to classify said item of user content further comprises using said package information.

38. A method as set forth in claim 35, wherein said operating step comprises operating said computer-based content exchange to classify said item of user content into a first category or a second category, and wherein said first category signifies compliance with said policy and said second category signifies noncompliance with said policy.

39. A method as set forth in claim 35, wherein said operating step comprises operating said computer-based content exchange to classify said item of user content into one of three categories in relation to said policy, wherein said three categories include a first category, a second category, and a third category, and wherein said first category signifies compliance with said policy, said second category signifies noncompliance with said policy, and said third category signifies conditional compliance with said policy.

40. A method as set forth in claim 35, wherein said policy defines one of use parameters and user parameters associated with said item of user content.

41. A method as set forth in claim 40, wherein said use parameters define one or more allowable uses for said item of user content.

42. A method as set forth in claim 40, wherein said use parameters define one or more intended uses for said item of user content.

43. A method as set forth in claim 40, wherein said use information identifies one of a use and a user of said item of user content.

44. A method as set forth in claim 43, wherein said user comprises an individual user or a user group.

45. A method as set forth in claim 1, further comprising:

implementing one or more enhancements to the computer-based content exchange; and

in light of said enhancements, repeating said comparing, determining, selecting, and applying steps with respect to said item of user content.

46. A method as set forth in claim 45, wherein said enhancements comprise providing, on said computer-based content exchange, instructions regarding one or more additional noise reduction techniques.

47. A method as set forth in claim 1, wherein said selected noise reduction techniques are at least partially based an adaptive code window, a dependency analysis, match coalescence, name heuristics, directory heuristics, licensing information regarding the matches, or a directory structure comparison.

48. The method of claim 1, wherein said selecting step includes selecting at least two of said plurality of noise reduction techniques and determining an order in which said at least two noise reduction techniques are to be applied to said potential matches.

49. The method of claim 48, wherein said applying step includes applying said at least two noise reduction techniques to said potential matches in said determined order.

50. The method of claim 1, wherein a plurality of said one or more potential matches are provided after said applying step.

51. A method for reducing noise in a set of deep discovery analysis results, comprising:

establishing a computer-based content exchange, wherein said computer-based content exchange receives an item of user content and compares said item of user content to a library of items of comparison content to identify a number of potential matches from among said items of comparison content, wherein each said potential match includes a matched portion that is similar to a portion of said item of user content, and wherein the items of comparison content comprise at least one item of open source software content;

selecting, from among a plurality of noise reduction protocols stored on said computer-based content exchange, one or more of said noise reduction protocols;

applying, using said computer-based content exchange, said one or more noise reduction protocols to said potential matches to eliminate noise from said potential matches; and

depending on an outcome of said selecting and said applying steps, iteratively repeating said selecting and said applying steps until an acceptable amount of said noise is eliminated from said potential matches.

52. A method as set forth in claim 51, wherein said selecting is automatic.

53. A method as set forth in claim 51, wherein said selecting is manual.

54. A method as set forth in claim 51, wherein said noise comprises one or more false positives.

55. A method as set forth in claim 51, wherein said noise comprises one or more redundant potential matches.

56. A method as set forth in claim 51, wherein said noise comprises one or more erroneously identified potential matches.

57. A method as set forth in claim 51, wherein an acceptable amount of said noise is eliminated from said potential matches when said potential matches comprise a set of positively identified confirmed matches.

58. A system for analyzing protectable content, comprising:

one or more memory structures, said memory structures storing a library of items of comparison content including at least one item of open source software content;

a scanner, said scanner receiving an item of user content received from a user; and

a processor communicatively coupled with said memory structures and said scanner, wherein said processor directs a comparison between said item of user content and said items of comparison content to identify a number of potential matches from among said items of comparison content, wherein each said potential match includes a matched portion that is similar to a portion of said item of user content, and wherein said processor iteratively applies a plurality of noise reduction protocols to said potential matches to eliminate individual ones of said potential matches.

59. A system as set forth in claim 58, wherein each said item of comparison content and said item of user content comprise one or more portions of source code, one or more portions of binary code, one or more source code files or binary code files, one or more directory structures, software projects, software applications, or software packages.