Hypervideo system and method with object tracking in a compressed digital video environment

• US 6,169,573 B1
• Filed: 07/03/1997
• Issued: 01/02/2001
• Est. Priority Date: 07/03/1997
• Status: Expired due to Term

• Current Assignee: Tata America International Corporation (Tata Sons Pvt Ltd.)
• Original Assignee: HOTV Incorporated
• Inventors: Goldman, Kenneth J., Talreja, Kamlesh, Sampath-Kumar, Srihari, Rangan, P. Venkat
• Primary Examiner(s): BRITTON, HOWARD W
• Application Number:
  US08/887,748
• Time in Patent Office: 1,279 Days
• Field of Search: H04/N.718, 348/169, 348/170, 348/171, 348/172
• US Class Current: 348/169
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 21/8583 by creating hot-spots
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

A system and method for hypervideo object tracking in a compressed digital video environment includes a hypervideo authoring system and a hypervideo browser. The hypervideo authoring system permits one or more moving objects in an MPEG compressed video clip to be identified, tracked, refined, edited, and associated with a specified action with a minimum of user intervention. The hypervideo browser allows the MPEG video to be viewed and objects of interest to be selected by a user, thereby hyperlinking to additional information of interest.

Claims

1. A method for object tracking in a digital video environment, comprising the steps of:

identifying an object in a frame of the video; and

tracking an object, utilizing motion compensation information stored in the video, through at least one scene of the video to produce a hot region.

2. The method of claim 1 wherein the video is an MPEG-compressed digital video clip.

3. The method of claim 1, wherein the identifying step comprises the substeps of:

locating a point within the object; and

determining the boundaries of the object.

4. The method of claim 3, wherein the determining step comprises the substeps of:

approximating the contents of the frame; and

attributing all pixels contiguous to the point and within a range of color and brightness values to the object.

5. The method of claim 4, wherein the approximating step utilizes DC coefficients from color and brightness blocks in the frame.

6. The method of claim 1, wherein the tracking step comprises the substeps of:

identifying a current scene in which to track the object;

specifying the hot region in a first frame of the current scene to correspond to the object;

finding the object in each subsequent frame in the current scene.

7. The method of claim 6, wherein the finding step comprises the substeps of:

identifying a forward motion vector for each macroblock in the frame, wherein the forward motion vector corresponds to a forward reference area in a forward reference frame; and

copying any portion of the hot region from the forward reference area to the macroblock.

8. The method of claim 7, wherein the finding step further comprises the substeps of:

identifying a backward motion vector for each macroblock in the frame, wherein the backward motion vector corresponds to a backward reference area in a backward reference frame;

copying any portion of the hot region from the backward reference area to the macroblock; and

copying any portion of the hot region from the macroblock to the backward reference area.

9. The method of claim 6, wherein the tracking step further comprises filling holes in the hot region.

10. The method of claim 6, wherein the tracking step further comprises the substeps of:

locating scene transitions in the video;

determining whether a later scene corresponds to the end of the current scene; and

if so, continuing to track the object in the later scene.

11. The method of claim 10, further comprising the step of refining the hot region.

12. The method of claim 10, wherein the refining step comprises the substeps of:

identifying at least two contiguous hot subregions corresponding to the tracked object;

determining whether each of the hot subregions corresponds to the object;

if not, discarding a hot subregion; and

if so, merging the hot subregions into a merged hot region.

13. The method of claim 1, wherein the hot region is represented as a bitmap.

14. The method of claim 13, further comprising the step of converting the hot region to a polygon.

15. The method of claim 14, further comprising the step of storing the polygon in the video.

16. The method of claim 1, further comprising the step of specifying an action for the object.

17. The method of claim 16, further comprising the steps of:

playing the video;

waiting for a user to select an object; and

upon selection, performing the specified action.

18. The method of claim 16, wherein the video comprises a plurality of scenes and a default path through the scenes.

19. The method of claim 18, further comprising the step of identifying at least one alternate path through the scenes.

20. The method of claim 19, wherein the specified action comprises switching from the default path to the alternate path.

21. The method of claim 17, wherein the specified action comprises linking to additional information to be displayed.

22. The method of claim 1, further comprising the step of converting the hot region to a regular shape.

23. The method of claim 22, wherein the converting step preserves a specified shape.

24. The method of claim 23, wherein the converting step comprises the steps of:

locating a centroid for the hot region;

determining a size of the hot region; and

using the centroid and the size to place the regular shape.

28. The method of claim 1, wherein the tracking step comprises the substeps of:

locating scene transitions in the video;

determining whether a later scene corresponds to the end of the current scene; and

if so, continuing to track the object in the later scene.

25. A method for object tracking and editing in a digital video environment, comprising the steps of:

tracking an object through at least one scene of the video to produce a hot region;

identifying an erasure region of an object in a frame of the video;

editing the object by identifying and tracking the erasure region; and

erasing any portion of the hot region corresponding to the erasure region.

26. The method of claim 25, wherein the video environment is an MPEG-compressed digital video clip.

27. The method of claim 25, wherein the identifying step comprises the substeps of:

locating a point within the object; and

determining the boundaries of the object.