Obtaining and using data associating annotating activities with portions of recordings

• US 6,665,490 B2
• Filed: 03/25/1999
• Issued: 12/16/2003
• Est. Priority Date: 04/01/1998
• Status: Expired due to Term

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Dymetman, Marc, Copperman, Max
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): CHIEU, PO LIN
• Application Number:
  US09/276,532
• Publication Number:
  US 20030190145A1
• Time in Patent Office: 1,727 Days
• Field of Search: 386/38, 386/45, 386/69-70, 386/83, 386/107, 386/117, 358/906, 358/909.1, 348/231.3, 348/239
• US Class Current: 386/241
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: H04N 1/19594 using a television camera or a still video camera
• CPC Class Codes: H04N 2201/3214 of a date
• CPC Class Codes: H04N 2201/3215 of a time or duration
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data

Abstract

Annotating activities are associated with portions of recordings by activity/recording data such as a table in which annotation identifiers are paired with timestamps. To obtain activity/recording data, an image input device, such as a manually positionable scanner or a fixed position camera, can produce an image signal set with information about a manual annotating activity and a recording/playback device can provide a recording portion signal, such as a timestamp, with information about a portion of a recording. The image signal set and the recording portion signal can then be used to automatically obtain the activity/recording data, associating information about the manual annotating activity with information identifying the portion of the recording. The information about the manual annotating activity can be an annotation identifier that includes information about the position or shape of the annotation. The activity/recording data can also include, for each annotation identifier, an image of the annotation. Activity/recording data can be used to playback a portion of a recording. Another image signal set can be obtained with information about a manual request activity, for example, and if the manual request activity is sufficiently similar to a manual annotating activity, the associated portion can be played back. Or a user can provide a selection signal indicating one of the annotations to obtain playback of the associated portion of the recording. If the annotating activity was performed with a keyboard or other user input device, the annotations can be printed, and the user can use a hand-held scanner to select an annotation to obtaining playback.

Claims

1. A method for obtaining data associating manual annotating activities with portions of recordings, the method comprising:

using an image input device to produce an image signal set, the image signal set defining a set of one or more images that include information about a manual annotating activity that occurs during a period of time within which a portion of a recording is made or played;

using a recording/playback device to obtain a recording portion signal providing information about the portion of the recording; and

using the image signal set and the recording portion signal to automatically obtain activity/recording data that associate information about the manual annotating activity with information identifying the portion of the recording;

in which the manual annotating activity is performed in a region of a marking medium that bears machine-readable markings encoding a position identifier for the region;

the machine-readable markings existing in the region of the marking medium before the manual annotating activity is performed; and

in which the image signal set defines a set of one or more images of the region of the marking medium;

the act of using the image signal set and the recording portion signal to automatically obtain activity/recording data comprising;

using the image signal set to obtain position data indicating the position identifier.

2. The method of claim 1 in which the marking medium has two or more zones, each bearing machine-readable markings that encode a position identifier indicating the position of the zone within the page;

the manual annotating activity being performed in one of the zones;

the image signal set defining an image of the zone that includes the machine-readable markings that encode the zone'"'"'s position identifier.

3. The method of claim 1 in which the machine readable markings encode a position identifier for the position of the machine readable markings and the manual annotating activity is performed next to the machine readable markings;

the image signal set defining an image of the machine readable markings.

4. The method of claim 1 in which the act of using the image signal set and the recording portion signal to automatically obtain activity/recording data comprises:

using the image signal set to obtain an identifier of the manual annotating activity;

the activity/recording data including a pair of items of data, the pair including the identifier of the manual annotating activity and an identifier of the portion of the recording.

5. The method of claim 4 in which the act of receiving a recording portion signal comprises:

receiving, during the period of time within which the manual annotating activity is performed and the portion of the recording is made or played, a time selection signal from a user selecting a time within the period; and

obtaining, in response to the time selection signal, a timestamp indicating the time within the recording that corresponds to the selected time;

the timestamp being the identifier of the portion of the recording.

6. The method of claim 1, further comprising printing the machine-readable markings on the marking medium.

7. The method of claim 6, wherein the machine-readable markings printed on the marking medium are visible or invisible.

8. The method of claim 7, wherein the marking medium is paper.

9. A method for obtaining data associating manual annotating activities with portions of recordings, the method comprising:

using an image input device to produce an image signal set, the image signal set defining a set of one or more images that include information about a manual annotating activity that occurs during a period of time within which a portion of a recording is made or played;

the manual annotating activity being performed on a marking medium;

the image signal set defining a set of one or more images of the marking medium;

using a recording/playback device to obtain a recording portion signal providing information about the portion of the recording; and

using the image signal set and the recording portion signal to automatically obtain activity/recording data that associate information about the manual annotating activity with information identifying the portion of the recording;

in which the manual annotating activity being performed on a marking medium;

the image signal set defining a set of one or more images of the marking medium; and

in which the manual annotating activity is performed at a position of the marking medium and the image signal set defines a before image showing the marking medium before the manual annotating activity and an after image showing the marking medium after the manual annotating activity;

the act of using the image signal set and the recording portion signal to automatically obtain activity/recording data comprising;

comparing the before image and the after image to obtain position data indicating the position at which the manual annotating activity is performed.

10. A system for obtaining and using data associating manual annotating activities with portions of recordings, the system comprising:

recording/playback circuitry for making or playing recordings;

an image input circuitry for providing image signals defining images; and

processing circuitry connected for receiving image signals from the image input circuitry and connected for receiving information about a recording being made or played by the recording/playback circuitry;

the processing circuitry operating to;

receive an image signal set from the image input circuitry, the image signal set defining a set of one or more images that include information about a manual annotating activity that occurs during a period of time within which a portion of a recording is made or played;

receive a recording portion signal from the recording/playback circuitry providing information about the portion of the recording; and

using the image signal set and the recording portion signal to automatically obtain activity/recording data that associate information about the manual annotating activity with information identifying the portion of the recording;

in which the processing circuitry is further connected for causing the recording/playback circuitry to play back portions of the recording;

the processing circuitry further operating to;

receive a second image signal set from the image input circuitry;

the second image signal set providing information about a manual request activity;

use the second image signal set to automatically obtain activity data that include information about the manual request activity;

if the information about the manual request activity and the information about one of the manual annotating activities meets a similarity criterion, use the activity data and the activity/recording data to obtain portion identifying data identifying the portion of the recording associated with the manual annotating activity; and

use the portion identifying data to provide a portion playback signal to the recording/playback circuitry causing playback of the portion of the recording.

11. The system of claim 10 in which the activity/recording data include an annotation identifier and a timestamp, the system further comprising:

a table containing pairs of items of data;

each pair including an annotation identifier and a timestamp;

the processing circuitry further operating to store the activity/recording data in the table.

12. The system of claim 11 in which the manual annotating activity produces an annotation;

the image signal set defining an image of the annotation;

the table further containing, for each pair, annotation image data defining the image of the annotation.

13. The system of claim 10 in which the manual annotating activity produces an annotation;

the system further comprising user selection circuitry for providing selection signals indicating annotations selected by a user;

the processing circuitry being connected for receiving the selection signals and for causing the recording/playback circuitry to play back portions of the recording;

the processing circuitry further operating to;

receive a selection signal indicating one of the annotations from the user selection circuitry;

use the selection signal and the activity/recording data to obtain portion identifying data identifying the portion of the recording associated with the manual annotating activity; and

use the portion identifying data to provide a portion playback signal to the recording/playback circuitry causing playback of the portion of the recording.

14. A method for using activity/recording data associating annotating activities with portions of recordings to play back portions of recordings, the method comprising:

using an image input device to produce an image signal set, the image signal set defining a set of one or more images of a marking medium that include information about a manual activity relating to the marking medium;

using the image signal set to automatically obtain activity data including information about the manual activity;

using the activity data and the activity/recording data to obtain portion identifying data identifying a portion of the recording;

the activity/recording data associating information about an annotating activity with the identified portion of the recording;

the information about the manual activity included in the activity data and the information about the annotating activity meeting a similarity criterion; and

using the portion identifying data to provide a portion playback signal to playback circuitry causing playback of the identified portion of the recording;

in which the manual activity is performed in a region of the marking medium that bears machine-readable markings encoding a position identifier for the region and the image signal set defines a set of one or more images of the region of the marking medium;

the machine-readable markings existing in the region of the marking medium before the manual activity is performed;

the act of using the image signal set to automatically obtain activity data comprising;

using the image signal set to obtain position data indicating the position identifier.

15. The method of claim 14 in which the marking medium has two or more zones, each bearing machine-readable markings that encode a position identifier indicating the position of the zone within the page;

the manual annotating activity being performed in one of the zones;

the image signal set defining an image of the zone that includes the machine-readable markings that encode the zone'"'"'s position identifier.

16. The method of claim 14 in which the machine readable markings encode a position identifier for the position of the machine readable markings and the manual activity is performed next to the machine readable markings;

the image signal set defining an image of the machine readable markings.

17. The method of claim 14 in which the act of using the image signal set to automatically obtain activity data comprises:

using the image signal set to obtain an identifier of the manual activity, the activity data including the identifier;

the activity/recording data including pairs of items of data, each pair associating an identifier of a manual activity and an identifier of a portion of the recording;

the act of using the activity data and the activity/recording data to obtain portion identifying data comprising;

comparing the identifier in the activity data with identifiers in the activity/recording data to find a match; and

upon finding a match, obtaining the associated identifier of a portion of the recording, the portion identifying data including the identifier of the portion.

18. The method of claim 14 in which the annotating activities are input of text by a keyboard;

the marking medium bearing a printed version of the text input by each of the annotating activities;

the image input circuitry comprising a hand-held scanner;

the manual activity including holding the hand-held scanner to scan the printed version of the text input by one of the annotating activities;

the image signal set defining an image of the scanned text input;

the act of using the image signal set to automatically obtain activity data comprising;

performing optical character recognition on the image of the scanned text input to obtain recognized text.

19. The method of claim 14 in which the annotating activities produce annotations;

the marking medium bearing a printed image of the annotations and, with each annotation, machine-readable markings encoding an identifier of the annotation;

the image input circuitry comprising a hand-held scanner;

the manual activity including holding the hand-held scanner to scan the machine-readable markings with one of the annotations;

the image signal set defining an image of the scanned machine-readable markings;

the act of using the image signal set to automatically obtain activity data comprising;

using the image of the scanned machine-readable markings to obtain data indicating the identifier of the annotation.

20. The method of claim 14, further comprising printing the machine-readable markings on the marking medium.

21. The method of claim 20, wherein the machine-readable markings on the marking medium are visible or invisible.

22. The method of claim 21, wherein the marking medium is paper.

23. A method for using activity/recording data associating annotating activities with portions of recordings to play back portions of recordings, the method comprising:

using an image input device to produce an image signal set, the image signal set defining a set of one or more images of a marking medium that include information about a manual activity relating to the marking medium;

using the image signal set to automatically obtain activity data including information about the manual activity;

using the activity data and the activity/recording data to obtain portion identifying data identifying a portion of the recording;

the activity/recording data associating information about an annotating activity with the identified portion of the recording;

the information about the manual activity included in the activity data and the information about the annotating activity meeting a similarity criterion; and

using the portion identifying data to provide a portion playback signal to playback circuitry causing playback of the identified portion of the recording;

in which the manual activity is performed at a position of the marking medium and the image signal set defines a before image showing the marking medium before the manual activity and an after image showing the marking medium after the manual activity;

the act of using the image signal set to automatically obtain activity data comprising;

comparing the before image and the after image to obtain position data indicating the position at which the manual activity is performed.