System and method for generating a viewable video index for low bandwidth applications

• US 20050185823A1
• Filed: 02/24/2004
• Published: 08/25/2005
• Est. Priority Date: 02/24/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hampapur, Arun, Connell, Jonathan H., Brown, Lisa Marie, Tian, Ying-Li, Cooke, Raymond A., Senior, Andrew William, Pankanti, Sharathchandra UmapathiRao
• Granted Patent: US 7,447,331 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 13/19684 Portable terminal, e.g. mobile phone, used for viewing video remotely
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system and method for generating a viewable video index for low bandwidth applications are provided. The exemplary aspects of the present invention solve the problems with the prior art systems by incorporating information for generating a viewable representation of the video data into the index, thus generating a viewable video index. The viewable video index contains information for generating a visual representation of moving objects in the video data, a visual representation of the background of the video capture area, i.e. the scene, a representation of the object trajectory, a representation of the object attributes, and a representation of detected events. The result is that the viewable video index may be transmitted to a low bandwidth application on a client device and may be used along with associated object and background models to generate a representation of the actual video data without requiring that the original video data itself be streamed to the client device.

Claims

1. A method, in a data processing system, for representing an input video data stream on a computing device, comprising:

receiving the input video data stream;

identifying a background image and at least one moving object in the input video data stream;

tracking at least one parameter of the at least one moving object over time in the input video data stream to thereby generate tracking information;

generating a viewable video index that identifies the tracking information, the at least one moving object and the background image; and

transmitting the viewable video index to the computing device.

2. The method of claim 1, wherein the at least one parameter of the at least one moving object includes a position of the at least one moving object, a size of the at least one moving object, and an orientation of the at least one moving object.

3. The method of claim 1, wherein the viewable video index identifies a model to be used when generating a representation of the at least one moving object.

4. The method of claim 1, wherein generating a viewable video index includes:

determining user selected parameters for generation of the viewable video index; and

generating the viewable video index based on the user selected parameters.

5. The method of claim 4, wherein the user selected parameters include a bandwidth requirement for the computing device, and wherein generating the viewable video index based on the user selected parameters includes determining at least one of an amount and a type of information that may be included in the viewable video index based on the bandwidth requirement.

6. The method of claim 1, further comprising:

receiving the viewable video index at the computing device; and

generating a graphical representation of the input video data stream based on the viewable video index.

7. The method of claim 6, wherein generating a graphical representation of the input video data stream includes superimposing a model of the at least one moving object over the background image and moving the model over the background image in accordance with the tracking information.

8. The method of claim 6, wherein generating a graphical representation of the input video data stream based on the viewable video index includes:

determining if either the tracking information, the background image or a model of the at least one moving object are not present locally on the computing device; and

transmitting a request to a source of the viewable video index for either the tracking information, the background image or the model of the at least one moving object if either the tracking information, the background image or the model is not present locally on the computing device.

9. A method, in a data processing system, for generating a representation of a video data stream, comprising:

receiving a viewable video index corresponding to a portion of the video data stream, wherein the viewable video index correlates tracking information with foreground object information and background object information;

superimposing a foreground object model over a background image based on the foreground object information and background object information in the viewable video index; and

graphically moving the foreground object model over the background image based on the tracking information.

10. The method of claim 9, further comprising:

determining if either the foreground object model, the background image, or the tracking information is not present in a local storage device; and

requesting either the foreground object model, the background image, or the tracking information from a source of the viewable video index if either the foreground object model, the background image or the tracking information is not present in the local storage device.

11. The method of claim 10, further comprising:

retrieving either the foreground object model, the background image, or the tracking information from the local storage device rather than sending a request to the source of the viewable video index, if either the foreground object model, the background image, or the tracking information is stored in the local storage device.

12. The method of claim 9, wherein the video data stream is not received prior to generating the representation of the video data stream.

13. The method of claim 9, further comprising:

parsing the tracking information for event information; and

generating a graphical timeline with event markers based on the tracking information and event information.

14. The method of claim 13, wherein the event markers are selectable to thereby jump the representation of the video data stream to a time point corresponding to a selected event marker.

15. The method of claim 9, wherein the background object information includes information indicating when a background image is to be changed in the representation of the video data stream, and wherein superimposing a foreground object model over a background image includes changing the background image from a previous background image to a current background image based on the information indicating when the background image is to be changed.

16. A computer program product in a computer readable medium for representing an input video data stream on a computing device, comprising:

first instructions for receiving the input video data stream;

second instructions for identifying a background image and at least one moving object in the input video data stream;

third instructions for tracking at least one parameter of the at least one moving object over time in the input video data stream to thereby generate tracking information;

fourth instructions for generating a viewable video index that identifies the tracking information, the at least one moving object and the background image; and

fifth instructions for transmitting the viewable video index to the computing device.

17. The computer program product of claim 16, wherein the at least one parameter of the at least one moving object includes a position of the at least one moving object, a size of the at least one moving object, and an orientation of the at least one moving object.

18. The computer program product of claim 16, wherein the viewable video index identifies a model to be used when generating a representation of the at least one moving object.

19. The computer program product of claim 16, wherein the fourth instructions for generating a viewable video index include:

instructions for determining user selected parameters for generation of the viewable video index; and

instructions for generating the viewable video index based on the user selected parameters.

20. The computer program product of claim 19, wherein the user selected parameters include a bandwidth requirement for the computing device, and wherein generating the viewable video index based on the user selected parameters includes determining at least one of an amount and a type of information that may be included in the viewable video index based on the bandwidth requirement.

21. The computer program product of claim 16, further comprising:

sixth instructions for receiving the viewable video index at the computing device; and

seventh instructions for generating a graphical representation of the input video data stream based on the viewable video index.

22. The computer program product of claim 21, wherein the fifth instructions for generating a graphical representation of the input video data stream include instructions for superimposing a model of the at least one moving object over the background image and moving the model over the background image in accordance with the tracking information.

23. The computer program product of claim 21, wherein the fifth instructions for generating a graphical representation of the input video data stream based on the viewable video index include:

instructions for determining if either the tracking information, the background image or a model of the at least one moving object are not present locally on the computing device; and

instructions for transmitting a request to a source of the viewable video index for either the tracking information, the background image or the model of the at least one moving object if either the tracking information, the background image or the model is not present locally on the computing device.

24. A computer program product in a computer readable medium for generating a representation of a video data stream, comprising:

first instructions for receiving a viewable video index corresponding to a portion of the video data stream, wherein the viewable video index correlates tracking information with foreground object information and background object information;

second instructions for superimposing a foreground object model over a background image based on the foreground object information and background object information in the viewable video index; and

third instructions for graphically moving the foreground object model over the background image based on the tracking information.

25. The computer program product of claim 24, further comprising:

fourth instructions for determining if either the foreground object model, the background image, or the tracking information is not present in a local storage device; and

fifth instructions for requesting either the foreground object model, the background image, or the tracking information from a source of the viewable video index if either the foreground object model, the background image or the tracking information is not present in the local storage device.

26. The computer program product of claim 25, further comprising:

sixth instructions for retrieving either the foreground object model, the background image, or the tracking information from the local storage device rather than sending a request to the source of the viewable video index, if either the foreground object model, the background image, or the tracking information is stored in the local storage device.

27. The computer program product of claim 24, wherein the video data stream is not received prior to generating the representation of the video data stream.

28. The computer program product of claim 24, further comprising:

fourth instructions for parsing the tracking information for event information; and

fifth instructions for generating a graphical timeline with event markers based on the tracking information and event information.

29. The computer program product of claim 28, wherein the event markers are selectable to thereby jump the representation of the video data stream to a time point corresponding to a selected event marker.

30. The computer program product of claim 24, wherein the background object information includes information indicating when a background image is to be changed in the representation of the video data stream, and wherein superimposing a foreground object model over a background image includes changing the background image from a previous background image to a current background image based on the information indicating when the background image is to be changed.

31. An apparatus for representing an input video data stream on a computing device, comprising:

means for receiving the input video data stream;

means for identifying a background image and at least one moving object in the input video data stream;

means for tracking at least one parameter of the at least one moving object over time in the input video data stream to thereby generate tracking information;

means for generating a viewable video index that identifies the tracking information, the at least one moving object and the background image; and

means for transmitting the viewable video index to the computing device.

32. An apparatus for generating a representation of a video data stream, comprising:

means for receiving a viewable video index corresponding to a portion of the video data stream, wherein the viewable video index correlates tracking information with foreground object information and background object information;

means for superimposing a foreground object model over a background image based on the foreground object information and background object information in the viewable video index; and

means for graphically moving the foreground object model over the background image based on the tracking information.