DATA PROVENANCE SYSTEM

• US 20180341701A1
• Filed: 05/24/2017
• Published: 11/29/2018
• Est. Priority Date: 05/24/2017
• Status: Abandoned Application

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Inventors: Verma, Vineet, Karukuri, Venkata Swamy
• Application Number:
  US15/604,535
• Publication Number:
  US 20180341701A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 7/026 Magnitude comparison, i.e. determining the relative order of operands based on their numerical value, e.g. window comparator

Abstract

Data is received from a computing system describing particular content of a digital work. The data is processed to identify a particular concept represented in the particular content. A search of a corpus is initiated to identify a set of other digital works in the corpus including content related to the particular concept. Similarity scores are determined representing a degree of similarity between the particular content of the digital work and the respective content of each of the set of digital works related to the particular concept. A data provenance system determines that a particular one of the other digital works is a source of the particular content of the digital work based on the similarity scores. Result data is generated and sent to the computing system to indicate that the particular other digital work is a source of the particular concept.

Claims

1. A method comprising:

receiving data from a computing system describing particular content of a digital work;

processing the data to identify a particular concept represented in the particular content;

initiating a search of a corpus to identify a set of other digital works in the corpus comprising content related to the particular concept;

determining similarity scores representing a degree of similarity between the particular content of the digital work and the respective content of each of the set of digital works related to the particular concept;

determining that a particular one of the other digital works is a source of the particular content of the digital work based on the similarity scores; and

sending result data to the computing system to indicate that the particular other digital work is a source of the particular concept.

2. The method of claim 1, wherein the digital work comprises a first type of media, the set of other digital works comprise one or more types of media different from the first type of media, and the method further comprises translating at least some of the digital works into a common media format, wherein the similarity scores are determined based on comparing the respective digital works in the common media format.

3. The method of claim 2, wherein the types of media comprise two or more of text media, image media, audio media, and video media.

4. The method of claim 1, wherein the corpus comprises a corpus of indexed records corresponding to a plurality of digital works comprising the set of other digital works, and the corpus defines relationships between the plurality of digital works to indicate that content of at least some of the plurality of digital works incorporate content of other digital works in the plurality of digital works.

5. The method of claim 4, wherein the corpus further comprises online resources, and the online resources are to be searched using a web crawler.

6. The method of claim 4, wherein the digital work comprises a first digital work and the method further comprises adding a record to the corpus corresponding to the first digital work to indicate that the particular content of the first digital work is sourced from the particular other digital work.

7. The method of claim 1, wherein the digital work comprises a first digital work and a particular one of the similarity scores determined to represent a degree of similarity between the particular content of the first digital work and content of the particular other digital work indicates a less than perfect match between the particular content and content of the particular other digital work representing the particular concept.

8. The method of claim 7, wherein a second one of the similarity score determined to represent a degree of similarity between the particular content of the first digital work and content of a second one of the other digital works indicates a perfect match between the particular content and content of the second other digital work representing the particular concept, and determining that the particular other digital work is the source of the particular content comprises:

determining that the particular content comprises content copied from the second other digital work;

identifying a data provenance relationship defined between the particular other digital work and the second other digital work; and

determining that the particular other digital work is an original source of content representing the particular concept.

9. The method of claim 1, further comprising:

determining a modification to an original version of the digital work, wherein the modification forms a second version of the digital work; and

generating a modification trail tree data structure for the digital work comprising representations of the original and second versions of the digital work and a relationship definition indicating that the second version is a modification of the original version.

10. The method of claim 9, wherein the modification comprises a first modification and the method further comprises:

determining a second modification to the original version of the digital work to form a third version of the digital work;

determining a modification to the second version of the digital work to form a fourth version of the digital work;

updating the modification trail tree data structure to add a representation of the third version of the digital work with an indication that the third version is a modification of the original version and add a representation of the fourth version of the digital work with an indication that the fourth version is a modification of the second version.

11. The method of claim 9, wherein the corpus comprises a plurality of versions of the particular other digital work and determining that the particular other digital work is a source of the particular content of the digital work is based on a modification trail tree data structure for the particular other digital work.

12. The method of claim 11, wherein the result data indicates a latest one of the plurality of versions of the particular digital work, based on the modification trail tree data structure for the particular other digital work.

13. The method of claim 1, wherein the digital work comprises a first digital work and the method further comprises:

determining that the first digital work is attributable to a first entity; and

determining that the particular digital work is attributable to a different, second entity, wherein the result data indicates an identity of the second entity.

14. The method of claim 13, wherein the result data comprises attribution data to associate with the first digital work to identify that the content of the first digital work representing the particular concept is attributable to the second entity.

15. The method of claim 1, wherein the digital work comprises a first digital work and the method further comprises:

generating a first context image corresponding to the content of the first digital work, wherein the first context image comprises a graph comprising a topic node to identify a topic of the particular concept and attribute nodes to identify respective attributes of the topic of the particular concept, and determining the similarity scores comprises;

identifying context images for each of the set of digital works, and

determining the degrees of similarity based on comparisons of the context images of the set of digital works with the first context image.

16. The method of claim 15, wherein generating the first context image comprises:

converting the particular content of the first digital work to text; and

processing the text using natural language processing to identify a first word in the text corresponding to the topic and a set of second words in the text corresponding to the attributes of the topic, wherein the topic node identifies the first word and the attribute nodes identify the set of second words.

17. A computer program product comprising a computer readable storage medium comprising computer readable program code embodied therewith, the computer readable program code comprising:

computer readable program code configured to generate a first representation of content of a first digital work comprising media of a first type;

computer readable program code configured to determine similarity scores for the first digital work to indicate a degree of similarity between the first digital work and a plurality of other digital works based on comparing the first representation with a plurality of representations of the plurality of other digital works, wherein the plurality of other digital works comprises a second digital work, and the plurality of other digital works comprise media of a plurality of different types;

computer readable program code configured to determine, from the similarity scores, that the first digital work incorporates content originally sourced from the second digital work; and

computer readable program code configured to send result data to a system associated with the first digital work, wherein the result data indicates an attribution to the second digital work to be associated with the first digital work based on determining that the first digital work incorporates content originally sourced from the second digital work.

18. A system comprising:

a processor;

a memory element;

a data provenance service, executable by the processor to;

receive data describing at least a particular portion of a first digital work;

process the data to identify a particular concept represented in the particular content;

identify a set of other digital works in a corpus comprising content related to the particular concept, wherein the first digital work comprises media of a first type, and at least a portion of the digital works in the set of other works comprise media of a different, second type;

determine similarity scores representing a degree of similarity between the particular content of the first digital work and the respective content of each of the set of digital works related to the particular concept;

determine from the similarity scores that a second digital work, in the set of other digital works, is a source of the particular content of the first digital work; and

send result data to a computing system associated with the first digital work to indicate that the second digital work is a source of the particular content.

19. The system of claim 18, further comprising a document generator to:

generate the first digital work, wherein the data is received from the document generator at the data provenance service; and

automatically insert an attribution to the second digital work within the first digital work based on the determination that the second digital work is the source of the particular content.

20. The system of claim 18, further comprising a context image generator to:

convert the content of the first digital work to text; and

processing the text using natural language processing to determine a first word in the text corresponding to a topic of the particular concept and a set of second words in the text corresponding to attributes of the topic; and

generate a context image for the first digital work comprising a graph comprising nodes corresponding to the first word and the set of second words and defining relationships between the nodes to indicate that the set of second words represent attributes of the topic represented by the first word,

wherein identifying the set of other digital works comprises accessing context images of each of the set of other digital works, and determining the similarity scores comprises comparing the context image for the first digital work with the context images for the set of other digital works.