Interactive video system with dynamic video object descriptors

• US 5,684,715 A
• Filed: 06/07/1995
• Issued: 11/04/1997
• Est. Priority Date: 06/07/1995
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Information Systems Incorporated (Canon Inc.)
• Inventors: Palmer, Douglas L.
• Primary Examiner(s): Chin, Wellington
• Assistant Examiner(s): Carman, Melissa Kay
• Application Number:
  US08/475,984
• Time in Patent Office: 881 Days
• Field of Search: 395/152, 395/142, 395/143, 395/154, 395/135, 395/161, 395/159, 395/155, 395/156, 395/157, 395/140, 395/358, 395/806, 395/326, 364/514 R, 364/514 A, 364/514 C, 348/396, 348/473, 358/335, 358/311, 358/346, 358/342, 345/145, 345/189, 345/207, 345/167, 345/157, 345/118, 345/115, 345/139, 345/185, 345/186, 345/22, 345/27
• US Class Current: 348/473
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 19/022 Control panels
• CPC Class Codes: G11B 2220/20 Disc-shaped record carriers
• CPC Class Codes: G11B 2220/213 Read-only discs
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 2220/2587 Laser Discs; Optical disc using analog recording
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/107 of operating tapes
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

An interactive video system by which an operator is able to select an object moving in a video sequence and by which the interactive video system is notified which object was selected so as to take appropriate action. Interactive video is achieved through generation and use of video object descriptors which are synchronized to objects in the video sequence. Video object descriptors are generated by a generating tool which decomposes frames of video sequences and tracks movement of objects in those frames so as to generate a frame sequential file of video object descriptors. The file of video object descriptors are then used by an event interpreter which detects a match between the position of a pointing device on a display containing the video sequence and the position of a video object descriptor. When a match is detected, an interactive video operation is performed, such as jumping to a new video sequence, altering flow of the interactive video program or the like.

Claims

1. A generator for generating a frame-sequential file of video object descriptors from successive frames of video information, comprising:

object designation means for accepting an initial designation of video objects in a first video frame;

a video decomposition and object identification unit for decomposing subsequent frames of video information into constituent objects and for identifying objects therein;

an object tracker for tracking sequential positions of designated objects in sequential frames of video information;

an attribute generator for generating attributes corresponding to video objects tracked by said object tracker; and

a frame correlator for correlating frames of video information with objects identified by said video decomposition and identification unit;

whereby a frame-sequential file of video object descriptors is output, each such descriptor including, for each successive frame, a frame identifier correlated by said frame correlator and a video object descriptor for each object tracked by said object tracker.

2. A generator according to claim 1, wherein said video decomposition and object identification unit decomposes each frame of video information based on object specific techniques.

3. A generator according to claim 1, wherein said object tracking unit forms a history of objects so as to trace movement of an object from frame to successive frame.

4. A generator according to claim 1, wherein said attribute generator generates spatial attribute information, temporal attribute information, and action attribute information.

5. A generator according to claim 4, wherein said action attribute information is comprised of default information for action attributes.

6. A generator according to claim 1, wherein said generator is selectively operable in one of a manual mode, an automatic mode, and an assisted mode.

7. A generator according to claim 6, wherein in the manual mode said generator allows for real-time interaction and frame-step interaction or adjustment.

8. A video event interpreter, comprising:

a video source for providing successive frames of video information;

an object descriptor source for providing object descriptors that move in synchronism with movement of objects in the successive frames of video information;

a display for displaying the successive frames of video information and on which a user-manipulable cursor is superimposingly displayed;

an active object list for storing spatial attribute information for valid video objects, said active object list being extracted from said object descriptors;

a cursor position matcher for matching the position of the user-manipulable cursor to spatial attribute information in the active object list;

an action map containing a list of actions initiated in a case where said cursor position matcher matches the position of the cursor to a valid object in a list of active video objects; and

means responsive to user selection of a video object within the video sequence for activating a corresponding action in said action map.

9. A video event interpreter according to claim 8, wherein said action map includes steps to control playout of the successive frames of video information from the video source.

10. A video event interpreter according to claim 8, wherein said action map includes steps for altering flow of an interactive video application program.

11. A video event interpreter according to claim 8, further comprising a synchronizer for synchronizing playout of frames from said video source with access to said object descriptors.