Method and apparatus for authoring and linking video documents

• US 6,462,754 B1
• Filed: 02/22/1999
• Issued: 10/08/2002
• Est. Priority Date: 02/22/1999
• Status: Expired due to Fees

• Current Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Chakraborty, Amit, Liu, Peiya, Hsu, Liang
• Primary Examiner(s): Nguyen, Cao H
• Application Number:
  US09/255,090
• Time in Patent Office: 1,324 Days
• Field of Search: 345/327, 345/328, 345/339, 345/302, 345/357, 345/356, 345/716, 345/723, 345/764, 345/853, 345/854, 308/2, 308/7, 308/8, 308/10, 217/501, 217/502, 217/512, 386/96, 386/52, 386/104, 386/4
• US Class Current: 715/723
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A method for authoring video documents includes the steps of inputting video data to be processed, segmenting the video data into shots by identifying breaks between the shots, subdividing the shots into subshots using motion analysis to provide location information for motions of objects of interest, describing boundaries for the objects of interest in the video data such that the objects of interest are represented by the boundaries in the shots and creating an anchorable information unit file based on the boundaries of the objects of interest such that objects of interest are used to identify portions of the video data. A system is also included.

Claims

1. A method for authoring video documents comprising the steps of:

inputting video data to be processed;

segmenting the video data into shots by identifying breaks between the shots;

subdividing the shots into subshots using motion analysis to provide location information for motions of objects of interest;

describing boundaries for the objects of interest in the video data such that the objects of interest are represented by the boundaries in the shots; and

creating an anchorable information unit file based on the boundaries of the objects of interest such that objects of interest are used to identify portions of the video data.

2. The method as recited in claim 1, wherein the step of segmenting the video data includes the steps of:

defining time segments for the video data

computing metrics as time series data for the time segments;

comparing the video data between the time segments; and

identifying abrupt and gradual changes between the time segments of the video data to define the shots.

3. The method as recited in claim 1, wherein the step of subdividing the shots into subshots using a motion analysis includes the steps of:

estimating motion for objects of interest by computing optical flow;

observing the motion of the objects of interest;

computing an error between the estimated motion and the observed motion; and

if the error is above a threshold value, creating an extra node to further define the motion of the objects of interest.

4. The method as recited in claim 1, wherein the motion analysis includes an affine transform.

5. The method as recited in claim 1, wherein the step of describing boundaries for the objects of interest includes the steps of:

assigning object types to the objects of interest for each shot, the object types including vertices;

interpolating corresponding vertices on object types between frames of shots to define one of a spline and a line such that the spline and the line define the motion of the objects of interest between the frames.

6. The method as recited in claim 1, further comprises the step of linking the objects of interest to other objects to provide an interactive video document.

7. The method as recited in claim 6, wherein the step of linking the objects of interest to other objects to provide an interactive video document includes the step of providing automatic hyperlinking between the video document and the other objects.

8. The method as recited in claim 7, wherein the automatic hyperlinking is provided by a hyperlinker and further comprises the step of providing link specifications processing, pattern matching, and link establishment between sources and destinations.

9. The method as recited in claim 1, further comprises the step of providing keyframes for representing shots and subshots of the video data such that the keyframe representation is used to identify the objects of interest included in the shots and subshots.

10. The method as recited in claim 1, wherein the video is specified in a video AIU specification language which follows Standard Generalized Markup Language (SGML) syntax and further comprising the step of defining syntax for the video specification.

11. A method for authoring video documents comprising the steps of:

providing a video device for playing a video;

associating the video with an anchorable information unit (AIU) file, the AIU file including object types associated with objects of interest within the video, the objects of interest being capable of hyperlinking to objects included in a browser;

interpolating boundaries of the object types between frames to define motions of the objects of interest; and

playing the video and displaying anchorable information units associated with the object types in the video to provide interactive objects of interest for linking the objects of interest with other media upon selection of one the of the objects of interest and objects of the other media.

12. The method as recited in claim 11, wherein the other media includes one of audio, images, hypertext, stored information and video.

13. The method as recited in claim 11, wherein the step of interpolating includes subdividing identified shots of the video into subshots using a motion analysis, the motion analysis including the steps of:

estimating motion for the objects of interest by computing optical flow;

observing the motion of the objects of interest;

computing an error between the estimated motion and the observed motion; and

if the error is above a threshold value, creating an extra node to further define the motion of the objects of interest.

14. The method as recited in claim 11, wherein the motion analysis includes an affine transform.

15. The method as recited in claim 11, wherein the step of interpolating includes the steps of:

assigning object types to the objects of interest, the object types including vertices;

interpolating corresponding vertices on object types between frames of shots of the video to define a spline such that the spline defines the motion of the objects of interest between the frames.

16. The method as recited in claim 11, further comprises the step of linking the objects of interest to other objects to provide an interactive video document which includes the step of providing automatic hyperlinking between the object of interest in the video and information represented in other media.

17. The method as recited in claim 16, wherein the automatic hyperlinking is provided by a hyperlinker and further comprises the step of providing link specifications processing, pattern matching, and link establishment between sources and destinations.

18. The method as recited in claim 11, further comprises the step of providing key frames for representing shots and subshots of the video data such that the key frame representation is used to identify the objects of interest included in the shots and subshots.

19. The method as recited in claim 11, wherein the video is specified in a video AIU specification language which follows Standard Generalized Markup Language (SGML) syntax and further comprising the step of defining syntax for the video specification.

20. A system for authoring and viewing videos comprising:

a video editor for creating an anchorable information unit (AIU) file for objects of interest in a video;

a video device for playing the video, the video having the anchorable information unit file associated therewith, the AIU file including object types associated with objects of interest within the video;

a browser for interacting with the objects of interest to objects wherein playing the video and displaying the AIU file associated with the object types in the video provides interactive objects of interest for linking the objects of interest with other media upon selection of one the of the objects of interest and objects of the other media; and

the video editor including means for interpolating vertices of the object types between frames to define motions of the objects of interest such that motions of the objects of interest are tracked during video play.

21. The system as recited in claim 20, wherein the other media includes one of audio, images, hypertext, stored information and video.

22. The system as recited in claim 20, wherein the means for interpolating further comprises a processor for subdividing identified shots of the video into subshots using a motion analysis, the processor including:

means for estimating motion for the objects of interest by computing optical flow;

means for observing the motion of the objects of interest;

means for computing an error between the estimated motion and the observed motion; and

if the error is above a threshold value, means for creating an extra node to further define the motion of the objects of interest.

23. The system as recited in claim 20, wherein the motion analysis includes an affine transform.

24. The system as recited in claim 20, wherein the means for interpolating further comprises:

means for assigning object types to the objects of interest, the object types including vertices, the vertices between frames of shots of the video defining a spline such that the spline defines the motion of the objects of interest between the frames.

25. The system as recited in claim 20, further comprises an automatic hyperlinker for automatically hyperlinking the objects of interest in the video and related information in other media.

26. The system as recited in claim 25, wherein the automatic hyperlinker provides link specifications processing, pattern matching, and link establishment between sources and destinations.

27. The system as recited in claim 20, wherein the video device includes one of a disk player, a processor and a tape player.

28. The system as recited in claim 20, further includes an input device for selecting the objects of interest in the video.

29. The system as recited in claim 20, wherein the browser includes a processor.

30. The system as recited in claim 20, further comprises a key frame for representing each shot;

and subshots are defined by boundary frames such that the shot and subshots are individually identifiable by the system.

31. The system as recited in claim 30, wherein the keyframes are employed for authoring video documents other than by viewing the entire video.

32. The system as recited in claim 20, wherein the video is specified in a video AIU specification language which follows Standard Generalized Markup Language (SGML) syntax.