Document annotation sharing

• US 9,530,050 B1
• Filed: 03/31/2008
• Issued: 12/27/2016
• Est. Priority Date: 07/11/2007
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Le, Qifa, Erol, Berna, Hull, Jonathan J., Kishi, Hidenobu, Moraleda, Jorge
• Primary Examiner(s): Rush, Eric
• Application Number:
  US12/060,198
• Time in Patent Office: 3,193 Days
• Field of Search: 382/100, 382/112, 382/229, 382/305, 382/306, 707/608, 707/736, 715/230, 715/233, 715/255, 705/300, 705/301, 705/319
• US Class Current: 1/1
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes

Abstract

A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The present invention provides a system, method, and computer program product for annotating unindexed documents. Captured digital images of unknown paper documents or portions associated with a user are received with annotations. Features are extracted from the captured digital image and are stored, along with the captured digital image and the annotations.

Claims

1. A method of annotating an unindexed document, comprising:

receiving, by a computing device from a portable capture device, a first digital image depicting at least a first portion of the unindexed document, wherein the unindexed document is unknown to an index of stored documents;

receiving, by the computing device, an annotation associated with the unindexed document from a first user of the portable capture device;

extracting, by the computing device, features from the first digital image;

storing, by the computing device, the first digital image, the extracted features, and the annotation as an annotated digital image;

recognizing, by the computing device, a second user from the features of the first digital image, wherein the first digital image depicts at least the first portion of the unindexed document and the unindexed document is associated with the annotation from the first user;

responsive to recognizing the second user from the features of the first digital image depicting at least the first portion of the unindexed document, creating a new tie between the first user and the second user in a network of users who have captured documents with similar features; and

responsive to creating the new tie,

making the annotated digital image available to the second user and sharing an identity of the second user with the first user;

determining a first group of users, including the first user, who have operated on the unindexed document from users who have ties with the second user in the network;

determining a second group of users who have not operated on the unindexed document from the users who have ties with the second user in the network; and

weakening, by the computing device, the ties between the second user and the second group of users of the network based on the first group being larger than the second group.

2. The method of claim 1, wherein the annotation comprises a media clip selected from the group consisting of video, audio, and still images.

3. The method of claim 1, wherein the annotation is text.

4. The method of claim 1, wherein the annotation is an interestingness score.

5. The method of claim 1, wherein the annotation is arbitrary electronic data.

6. The method of claim 1, further comprising transmitting the first digital image, the extracted features, and the annotation prior to storing.

7. The method of claim 1, further comprising:

receiving a second digital image of at least a second portion of the unindexed document from the second user;

extracting features from the second digital image;

determining that the second digital image is similar to the first digital image based on comparing the extracted features from the first digital image to the extracted features from the second digital image; and

strengthening an existing tie between the first user and the second user in the network based at least in part on determining that the second digital image is similar to the first digital image.

8. The method of claim 7, further comprising comparing the extracted features from the first digital image to the extracted features from the second digital image based on a distance score generated from locations of the first user and the second user.

9. The method of claim 1, further comprising:

capturing a time of capture and a location of capture associated with the first user; and

storing the time of capture and the location of capture associated with the first user.

10. The method of claim 1, further comprising, in response to the unindexed document being indexed to form an indexed document, associating the annotated digital image with the indexed document.

11. A computer program product for annotating an unindexed document for shared use, the computer program product comprising:

a non-transitory computer-readable storage medium; and

computer program code, stored on the non-transitory computer-readable storage medium, when executed by one or more processors, for;

receiving, by a computing device from a portable capture device, a first digital image depicting at least a first portion of the unindexed document, wherein the unindexed document is unknown to an index of stored documents;

receiving, by the computing device, an annotation associated with the unindexed document from a first user of the portable capture device;

extracting, by the computing device, features from the first digital image;

storing, by the computing device, the first digital image, the extracted features, and the annotation as an annotated digital image;

recognizing, by the computing device, a second user from the features of the first digital image, wherein the first digital image depicts at least the first portion of the unindexed document and the unindexed document is associated with the annotation from the first user;

responsive to recognizing the second user from the features of the first digital image depicting at least the first portion of the unindexed document, creating a new tie between the first user and the second user in a network of users who have captured documents with similar features; and

responsive to creating the new tie,

making the annotated digital image available to the second user and sharing an identity of the second user with the first user;

determining a first group of users, including the first user, who have operated on the unindexed document from users who have ties with the second user in the network;

determining a second group of users who have not operated on the unindexed document from the users who have ties with the second user in the network; and

weakening, by the computing device, the ties between the second user and the second group of users of the network based on the first group being larger than the second group.

12. The computer program product of claim 11, wherein the annotation comprises a media clip selected from the group consisting of video, audio, and still images.

13. The computer program product of claim 11, wherein the annotation is text.

14. The computer program product of claim 11, wherein the annotation is an interestingness score.

15. The computer program product of claim 11,

further comprising, in response to the unindexed document being indexed to form an indexed document, associating the annotated digital image with the indexed document.

16. A system for annotating an unindexed document for shared use, the system comprising:

one or more processors;

a document matching module stored on a memory and executable by the one or more processors, the document matching module configured to receive a first digital image from a portable capture device, the first digital image depicting at least a first portion of the unindexed document, wherein the unindexed document is unknown to an index of stored documents, receive from a first user of the portable computing device an annotation associated with the unindexed document, extract features from the first digital image, and recognize a second user from the features of the first digital image, wherein the first digital image depicts at least the first portion of the unindexed document and the unindexed document is associated with the annotation from the first user;

a user association component stored on the memory and executable by the one or more processors, the user association component configured to, responsive to recognizing the second user from the features of the first digital image depicting at least the first portion of the unindexed document, create a new tie between the first user and the second user in a network of users who have captured documents with similar features, determine a first group of users, including the first user, who have operated on the unindexed document from users who have ties with the second user in the network, determine a second group of users who have not operated on the unindexed document from the users who have ties with the second user in the network, and weaken the ties between the second user and the second group of users of the network based on the first group being larger than the second group;

a database stored on the memory and accessible by the one or more processors, the database configured to store the first digital image, the extracted features and the annotation as an annotated digital image; and

a document authoring component stored on the memory and executable by the one or more processors, the document authoring component configured to, responsive to creating the new tie, make the annotated digital image available to the second user and share an identity of the second user with the first user.

17. The system of claim 16, further comprising the document matching module configured to receive a second digital image of at least a second portion of the unindexed document from the second user, extract features from the second digital image and determine that the second digital image is similar to the first digital image based on comparing the extracted features from the first digital image to the extracted features from the second digital image, and the user association component configured to strengthen an existing tie between the first user and the second user in the network based on determining that the second digital image is similar to the first digital image.

18. The system of claim 16, wherein the annotation comprises a media clip selected from the group consisting of video, audio, and still images.

19. The system of claim 16, wherein the annotation is text.

20. The system of claim 16, wherein the annotation is an interestingness score.