System and method for thematically analyzing and annotating an audio-visual sequence

• US 20020108112A1
• Filed: 02/01/2002
• Published: 08/08/2002
• Est. Priority Date: 02/02/2001
• Status: Abandoned Application

• Current Assignee: Ensequence Incorporated
• Original Assignee: Ensequence Incorporated
• Inventors: Monday, Stacy Anne, Miller, Eric Brent, Acott, Troy Steven, Wallace, Michael W.
• Application Number:
  US10/061,908
• Publication Number:
  US 20020108112A1
• Time in Patent Office: Days
• US Class Current: 725/40
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

This disclosure describes a method and system for creating an annotated analysis of the thematic content of a film or video work. The annotations may refer to single frames, or to sequences of consecutive frames. The sequences of frames for a given theme may overlap with one or more single frame or sequence of frames from one or more other themes in the work.

Claims

1. A method for generating annotations of viewable segments within a video sequence comprising the steps of:

selecting a start frame from a video sequence;

selecting an end frame from a video sequence to form in conjunction with the selected start frame a designated video segment;

associating an attribute with the designated video segment; and

storing the attribute as metadata within a lookup table for subsequent selection and presentation of the designated video segment to a viewer.

2. The method of claim 1, further including the step of automatically annotating scene division metadata within the lookup table.

3. The method of claim 1, further including the step of annotating a video segment responsive to an automated object recognition sytem.

4. The method of claim 3, wherein the objects automatically recognized by the system include a first-level attribute selected from the group consisting of scene boundaries, the presence of actors, the presence of specific objects, the occurrence of decipherable text in the video images, zoom or pan camera movements, or motion analysis.

5. The method of claim 1, further including the steps of:

selecting a second start frame from a video sequence;

selecting a second end frame from a video sequence to form in conjunction with the selected second start frame a second designated video segment, wherein said second designated video segment at least partially overlaps with said designated video segment;

associating a second attribute with the second designated video segment; and

storing the second attribute as metadata within the lookup table for subsequent selection and presentation of the second designated video segment to a viewer.

6. The method of claim 1 wherein said annotation includes a plurality of elements including a structural element and a thematic element.

7. The method of claim 1, wherein said metadata includes a low-level annotation comprising a type indicator, start time, duration or stop time, and a pointer to a label string.

8. The method of claim 7 wherein the type indicator refers to a one selected from the group consisting at least from a person, event, object, or text.

9. The method of claim 7 wherein the start and stop times are given in absolute terms.

10. The method of claim 7 wherein the start and stop times are given in relative terms to a reference point within the video sequence.

11. The method of claim 7, wherein said metadata includes a second-level annotation comprising a type indicator, a pointer to a label, and a pointer to a first of a linked list of elements.

12. The method of claim 1, further including the steps of:

presenting for visual inspection a list of the attributes contemporaneous with a timeline of the video sequence;

selecting at least one attribute from the list; and

performing the associating step responsive to the step of selecting at least one attribute from the list.

14. The method of claim 13, further including the steps of:

displaying a list of thematic events; and

receiving a selection of one of the thematic events to form a video segment request.

13. A method for retrieving and displaying segments from a video sequence comprising the steps of:

receiving a request for a video segment from a viewer;

retrieving a start frame and an end frame associated with said requested video segment from a memory lookup table;

finding a base frame associated with said start frame according to an offset associated with said start frame;

decoding from said base frame; and

displaying a video segment starting only from said start frame and continuing to said end frame.