System and method for generating a viewable video index for low bandwidth applications

• US 7,447,331 B2
• Filed: 02/24/2004
• Issued: 11/04/2008
• Est. Priority Date: 02/24/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hampapur, Arun, Connell, Jonathan H., Brown, Lisa Marie, Tian, Ying-Li, Cooke, Raymond A., Senior, Andrew William, Pankanti, Sharathchandra UmapathiRao
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US10/785,890
• Publication Number:
  US 20050185823A1
• Time in Patent Office: 1,715 Days
• Field of Search: 382/103, 382/107, 382/154, 382/194, 382/224, 382/243, 382/142, 348/143, 348/169, 348/700, 375/241.1, 375/E7.004, 375/E7.011, 375/E7.012, 375/E7.076, 375/E7.078, 380/212
• US Class Current: 382/103
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 13/19684 Portable terminal, e.g. mobile phone, used for viewing video remotely
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system and method for generating a viewable video index for low bandwidth applications are provided. The exemplary aspects of the present invention solve the problems with the prior art systems by incorporating information for generating a viewable representation of the video data into the index, thus generating a viewable video index. The viewable video index contains information for generating a visual representation of moving objects in the video data, a visual representation of the background of the video capture area, i.e. the scene, a representation of the object trajectory, a representation of the object attributes, and a representation of detected events. The result is that the viewable video index may be transmitted to a low bandwidth application on a client device and may be used along with associated object and background models to generate a representation of the actual video data without requiring that the original video data itself be streamed to the client device.

Claims

1. A method, in a data processing system, for representing an input video data stream on a computing device, comprising:

receiving the input video data stream;

identifying a background image and at least one moving object in the input video data stream;

tracking at least one parameter of the at least one moving object over time in the input video data stream to thereby generate tracking information;

determining user selected parameters for generation of a viewable video index, wherein the user selected parameters include a bandwidth requirement for the computing device;

generating the viewable video index based on the user selected parameter, wherein the viewable video index identifies the tracking information, the at least one moving object and the background image, and wherein generating the viewable video index based on the user selected parameters includes determining at least one of an amount and a type of information that may be included in the viewable video index based on the bandwidth requirement; and

transmitting the viewable video index to the computer device.

2. The method of claim 1, wherein the at least one parameter of the at least one moving object includes a position of the at least one moving object, a size of the at least one moving object, and an orientation of the at least one moving object.

3. The method of claim 1, wherein the viewable video index identifies a model to be used when generating a representation of the at least one moving object.

4. The method of claim 1, further comprising:

receiving the viewable video index at the computing device; and

generating a graphical representation of the input video data stream based on the viewable video index.

5. The method of claim 4, wherein generating a graphical representation of the input video data stream includes superimposing a model of the at least one moving object over the background image and moving the model over the background image in accordance with the tracking information.

6. The method of claim 4, wherein generating a graphical representation of the input video data stream based on the viewable video index includes:

determining if either the tracking information, the background image or a model of the at least one moving object are not present locally on the computing device; and

transmitting a request to a source of the viewable video index for either the tracking information, the background image or the model of the at least one moving object if either the tracking information, the background image or the model is not present locally on the computing device.

7. A method, in a data processing system, for generating a representation of a video data stream, comprising:

receiving a viewable video index corresponding to a portion of the video data stream, wherein the viewable video index correlates tracking information with foreground object information and background object information;

determining if either a foreground object model, a background image, or the tracking information is not present in a local storage device;

requesting the foreground object model, the background image, or the tracking information from a source of the viewable video index if either the foreground object model, the background image or the tracking information is not present in the local storage device.

superimposing the foreground object model over the background image based on the foreground object information and background object information in the viewable video index; and

graphically moving the foreground object model over the background image based on the tracking information.

8. The method of claim 7, further comprising:

retrieving either the foreground object model, the background image, or the tracking information from the local storage device rather than sending a request to the source of the viewable video index, if either the foreground object model, the background image, or the tracking information is stored in the local storage device.

9. The method of claim 7, wherein the video data stream is not received prior to generating the representation of the video data stream.

10. The method of claim 7, further comprising:

parsing the tracking information for event information; and

generating a graphical timeline with event markers based on the tracking information and event information.

11. The method of claim 10, wherein the event markers are selectable to thereby jump the representation of the video data stream to a time point corresponding to a selected event marker.

12. The method of claim 7, wherein the background object information includes information indicating when a background image is to be changed in the representation of the video data stream, and wherein superimposing a foreground object model over a background image includes changing the background image from a previous background image to a current background image based on the information indicating when the background image is to be changed.

13. A computer readable medium encoded with a computer program for representing an input video data stream on a computing device, wherein the computer program product comprises:

first instructions for receiving the input video data stream;

second instructions for identifying a background image and at least one moving object in the input video data stream;

third instructions for tracking at least one parameter of the at least one moving object over time in the input video data stream to thereby generate tracking information;

fourth instructions for determining user selected parameters for generation of the viewable video index, wherein the user selected parameters includes bandwidth requirement for the computing device;

fifth instructions for generating a viewable video index based on the user selected parameters, wherein the viewable video index identifies the tracking information, the at least one moving object and the background image; and

sixth instructions for transmitting the viewable video index to the computing device.

14. The computer program product of claim 13, wherein the at least one parameter of the at least one moving object includes a position of the at least one moving object, a size of the at least one moving object, and an orientation of the at least one moving object.

15. The computer readable medium of claim 13, wherein the viewable video index identifies a model to be used when generating a representation of the at least one moving object.

16. The computer readable medium of claim 13, wherein the computer program further comprises:

seventh instructions for receiving the viewable video index at the computing device; and

eighth instructions for generating a graphical representation of the input video data stream based on the viewable video index.

17. The computer program product of claim 16, wherein the eighth instructions for generating a graphical representation of the input video data stream include instructions for superimposing a model of the at least one moving object over the background image and moving the model over the background image in accordance with the tracking information.

18. The computer program product of claim 16, wherein the eighth instructions for generating a graphical representation of the input video data stream based on the viewable video index include:

instructions for determining if either the tracking information, the background image or a model of the at least one moving object are not present locally on the computing device; and

instructions for transmitting a request to a source of the viewable video index for either the tracking information, the background image or the model of the at least one moving object if either the tracking information, the background image or the model is not present locally on the computing device.

19. A computer readable medium encoded with a computer program for generating a representation of a video data stream, comprising:

first instructions receiving a viewable video index corresponding to a portion of the video data stream, wherein the viewable video index correlates tracking information with foreground object information and background object information;

second instruction for determining if either a foreground object model, a background image, or the tracking information is not present in a local storage device; and

third instructions for requesting either the foreground object model, the background image, or the tracking information from a source of the viewable video index if either the foreground object model, the background image or the tracking information is not present in the local storage device.

fourth instructions superimposing the foreground object model over the background image based on the foreground object information and background object information in the viewable video index; and

fifth instructions for graphically moving the foreground object model over the background image based on the tracking information.

20. The computer program product of claim 19, further comprising:

sixth instructions for retrieving either the foreground object model, the background image, or the tracking information from the local storage device rather than sending a request to the source of the viewable video index, if either the foreground object model, the background image, or the tracking information is stored in the local storage device.

21. The computer program product of claim 19, wherein the video data stream is not received prior to generating the representation of the video data stream.

22. The computer readable medium of claim 19, further comprising:

sixth instructions for parsing the tracking information for event information; and

seventh instructions for generating a graphical timeline with event markers based on the tracking information and event information.

23. The computer program product of claim 22, wherein the event markers are selectable to thereby jump the representation of the video data stream to a time point corresponding to a selected event marker.

24. The computer readable medium of claim 19, wherein the background object information includes information indicating when a background image is to be changed in the representation of the video data stream, and wherein superimposing a foreground object model over a background image includes changing the background image from a previous background image to a current background image based on the information indicating when the background image is to be changed.

25. An apparatus, for representing an input video data stream on a computing device, comprising:

means for receiving the input video data stream;

means for identifying a background image and at least one moving object in the input video data stream;

means for tracking at least one parameter of the at least one moving object over time in the input video data stream to thereby generate tracking information;

means for determining user selected parameters for generation of a viewable video index, wherein the user selected parameters include a bandwidth requirement for the computing device;

means for generating the viewable video index based on the user selected parameter, wherein the viewable video index identifies the tracking information, the at least one moving object and the background image, and wherein generating the viewable video index based on the user selected parameters includes determining at least one of an amount and a type of information that may be included in the viewable video index based on the bandwidth requirement; and

means for transmitting the viewable video index to the computing device.

26. An apparatus for generating a representation of a video data stream, comprising:

means for receiving a viewable video index corresponding to a portion of the video data stream, wherein the viewable video index correlates tracking information with foreground object information and background object information;

means for determining if either a foreground object model, a background image, or the tracking information is not present in a local storage device;

means for requesting either the foreground object model, the background image, or the tracking information from a source of the viewable video index if either the foreground object model, the background image or the tracking information is not present in the local storage device;

means for superimposing the foreground object model over the background image based on the foreground object information and background object information in the viewable video index; and

means for graphically moving the foreground object model over the background image based on the tracking information.