Method, device and computer program product for demultiplexing of video images

• US 8,233,044 B2
• Filed: 12/17/2007
• Issued: 07/31/2012
• Est. Priority Date: 09/07/2001
• Status: Expired due to Fees

• Current Assignee: Integran Technologies Incorporated (Analog Devices, Inc.)
• Original Assignee: Integran Technologies Incorporated (Analog Devices, Inc.)
• Inventors: Grindstaff, Gene Arthur, Fletcher, Susan Heath Calvin, McKay, Therman Ward III
• Primary Examiner(s): VO, TUNG T
• Application Number:
  US11/958,029
• Publication Number:
  US 20080095437A1
• Time in Patent Office: 1,688 Days
• Field of Search: 348143-148, 348/160, 348/170, 382/233, 382/104, 709/204, 352/129, 725/39, 375/240.01, 704/264
• US Class Current: 348/143
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

A method for demultiplexing time-division multiplexed digital video data which originates from multiple sources. The video data is not indexed nor identified to the sources. A first set of digital video data representative of a first image is identified as representative video data of the first source. A second set of digital video data representative of a current image is then retrieved. A difference ratio is calculated using the representative digital video data and the current set of digital video data. If the difference ratio is above a threshold, a display query is prompted to a user to indicate the source. If the response is indicative with a new source, the current set of digital video data is stored to a second memory location associated with the second source and identified as a representative of the second source.

Claims

1. A method for demultiplexing time-division multiplexed digital video data which originates from multiple sources, the method comprising:

retrieving a first set of digital video data representative of a first image from the time-division multiplexed digital video data;

storing the first set of digital video data to a memory location associated with a first source such that the first set of digital video data is representative digital video data of the first source;

retrieving a current set of digital video data representative of a current image;

calculating a difference ratio using the representative digital video data and the current set of digital video data; and

if the difference ratio is above a threshold, producing a display query requiring response for classifying the current digital video data as belonging to an identified source or originating from a new source, wherein the first source is an identified source.

2. The method according to claim 1 further comprising:

storing the current set of digital video data to a second memory location associated with the second source in accordance with a received response indicating that the current set of digital video data is associated with a new source such that the current set of digital video data is representative digital video data of the second source.

3. The method according to claim 1 further comprising:

if the difference ratio is below the threshold, storing the current digital video data to the memory location associated with the first source such that the current digital video data set is representative digital video data for the first source.

4. The method according to claim 1 further comprising:

normalizing the digital video data to compensate for differences in brightness prior to calculating a difference ratio.

5. The method according to claim 1, wherein the steps of calculating and producing further comprises:

calculating a difference ratio for each representative digital video data set from a source and the current set of digital video data; and

comparing each difference ratio to the threshold.

6. The method according to claim 5 further comprising identifying a source for which the difference ratio is below the threshold;

and

associating the current set of digital video data with the identified source.

7. The method according to claim 1, wherein associating the current set of digital video data with the identified source further comprises:

identifying the current set of digital video data as a representative digital video data set for a source.

8. The method according to claim 1 wherein the first set of digital video data is reduced in data size prior to being stored;

and

wherein a representation of the current set of digital video data is created which has a reduced size and which is used for determining the difference ratio.

9. The method according to claim 1, further comprising:

receiving a signal containing information representative of a section of the image to use for comparison in calculating the difference ratio, wherein the same section of the current frame and the same section of each representative frame are used in calculating the difference ratio.

10. The method according to claim 1, further comprising:

if the difference ratio is below the threshold, identifying the current frame as a representative frame of a source, wherein the set of data for the representative frame is used for comparisons with a set of data for subsequent frames.

11. The method according to claim 1, wherein as each different originating source is identified, the set of data that is representative of the current frame is stored as the representative frame for that source.

12. The method according to claim 1, wherein the sources are stationary surveillance cameras.

13. The method according to claim 1, wherein the sources are rotating surveillance cameras.

26. The method according to claim 1, further comprising:

receiving a signal containing information representative of a threshold.

54. The device according to claim 5, wherein the query module identifies a source for which the difference ratio is below the threshold;

and

wherein the memory module associates the current set of digital video data with the identified source.

14. A method for demultiplexing time-division multiplexed digital video data which originates from multiple sources, the method comprising:

retrieving a first set of digital video data representative of an image;

storing the first set of digital video data to a memory location associated with a first source such that the first set of digital video is representative digital video of the first source;

retrieving a current set of digital video data representative of an image;

selecting a subset of the first set of digital video data;

selecting a subset of the current set of digital video data such that the subset of the first set and the subset of the current set contain identical pixel locations;

calculating a difference measurement between the subset of the first set and the subset of the current set;

calculating a difference ratio based upon the difference measurement; and

if the difference ratio is above a threshold, producing a display query requiring response for classifying the digital video data as belonging to an identified source or originating from a new source.

15. The method according to claim 14 further comprising:

storing the current set of digital video data to a second memory location associated with the second source in accordance with a received response indicating that the current set of digital video data is associated with a new source such that the current set of digital video data is representative digital video data of the new source.

16. The method according to claim 14, further comprising:

if the difference ratio is below the threshold, storing the current digital video data to the memory location associated with the first source such that the current digital video data set is representative digital video data for the first source.

17. The method according to claim 14, further comprising:

normalizing the current digital video data to compensate for differences in brightness prior to calculating a difference ratio.

18. The method according to claim 14, wherein the steps of calculating and producing further comprises:

calculating a difference ratio for each representative digital video data set from a source and the current set of digital video data; and

comparing each difference ratio to the threshold.

19. The method according to claim 18, further comprising:

identifying a source for which the difference ratio is below the threshold; and

associating the current set of digital video data with the identified source.

20. The method according to claim, 14 wherein associating the current set of digital video data with the identified source further comprises:

identifying the current set of digital video data as a representative digital video data set for a source.

21. The method according to claim 14 wherein the first set of digital video data is reduced in data size prior to being stored;

and

wherein a representation of the current set of digital video data is created which has a reduced size and which is used for determining the difference ratio.

22. The method according to claim 14, further comprising:

if the difference ratio is below the threshold, identifying the current frame as a representative frame of a source, wherein the set of data for the representative frame is used for comparisons with a set of data for subsequent frames.

23. The method according to claim 14, wherein as each different originating source is identified, the set of data that is representative of the current frame is stored as the representative frame for that source.

24. The method according to claim 14, wherein the sources are stationary surveillance cameras.

25. The method according to claim 14, wherein the sources are rotating surveillance cameras.

27. The method according to claim 14, further comprising:

receiving a signal containing information representative of a threshold.

28. A computer program product comprising a non-transitory computer readable storage medium storing a computer program having computer readable code executable by a computer for demultiplexing time-division multiplexed digital video data which originates from multiple sources, the computer program comprising:

computer code for retrieving a first set of digital video data representative of a first image;

computer code for storing the first set of digital video data to a memory location associated with a first source;

computer code for retrieving a current set of digital video data representative of a current image;

computer code for calculating a difference ratio using the first set of digital video data and the current set of digital video data; and

computer code for producing a display query requiring response for classifying the digital video data as belonging to an identified source or originating from a new source if the difference ratio is above a threshold, wherein the first source is an identified source.

29. The computer program product according to claim 28 further comprising:

computer code for storing the current set of digital video data to a second memory location associated with the second source in accordance with a received response indicating that the current set of digital video data is associated with a new source.

30. The computer program product according to claim 28, further comprising:

computer code for storing the current digital video data to the memory location associated with the first source if the difference ratio is below the threshold.

31. The computer program product according to claim 28, further comprising:

computer code for normalizing the digital video data to compensate for differences in brightness prior to calculating a difference ratio.

32. The computer program product according to claim 28, wherein the computer code for calculating and producing further comprises:

computer code for calculating a difference ratio for each representative digital video data set from a source and the current set of digital video data; and

computer code for comparing each difference ratio to the threshold.

33. The computer program product according to claim 32, further comprising:

computer code for identifying a source for which the difference ratio is below the threshold; and

computer code for associating the current set of digital video data with the identified source.

34. The computer program product according to claim, 28 wherein the computer code for associating the current set of digital video data with the identified source further comprises computer code for identifying the current set of digital video data as a representative digital video data set for a source.

35. The computer program product according to claim 28 further comprising:

computer code for reducing the first set of digital video data in data size prior to being stored; and

computer code for reducing the current set of digital video data in data size prior to using the current set of digital data for determining the difference ratio.

36. The computer program product according to claim 28, further comprising:

computer code for receiving a signal containing information representative of a section of the image to use for comparison in calculating the difference ratio, wherein the same section of the current frame and the same section of each representative frame are used in calculating the difference ratio.

37. The computer program product according to claim 28, further comprising:

computer code for identifying the current frame as a representative frame of a source if the difference ratio is below the threshold, wherein the set of data for the representative frame is used for comparisons with a set of data for subsequent frames.

38. The computer program product according to claim 28, wherein as each different originating source is identified, the set of data that is representative of the current frame is stored as the representative frame for that source by the computer code for identifying.

39. A computer program product comprising a non-transitory computer readable storage medium having a computer readable code executed by a computer for demultiplexing time-division multiplexed digital video data which originates from multiple sources, the computer program comprising:

computer code for retrieving a first set of digital video data representative of an image;

computer code for storing the first set of digital video data to a memory location associated with a first source such that the first set of digital video is representative digital video of the first source;

computer code for retrieving a current set of digital video data representative of an image;

computer code for selecting a subset of the first set of digital video data;

computer code for selecting a subset of the current set of digital video data such that the subset of the first set and the subset of the current set contain identical pixel locations;

computer code for calculating a difference measurement between the subset of the first set and the subset of the current set;

computer code for calculating a difference ratio based upon the difference measurement; and

computer code for producing a display query requiring response for classifying the digital video data as belonging to an identified source or originating from a new source if the difference ratio is above a threshold.

40. The computer program product according to claim 39 further comprising:

computer code for storing the current set of digital video data to a second memory location associated with the second source in accordance with a received response indicating that the current set of digital video data is associated with a new source such that the current set of digital video data is representative digital video data of the new source.

41. The computer program product according to claim 39, further comprising:

computer code for storing the current digital video data to the memory location associated with the first source such that the current digital video data set is representative digital video data for the first source if the difference ratio is below the threshold.

42. The computer program product according to claim 39, further comprising:

computer code for normalizing the current digital video data to compensate for differences in brightness prior to calculating a difference ratio.

43. The computer program product according to claim 39, wherein the computer code for calculating includes:

computer code for calculating a difference ratio for each representative digital video data set from a source and the current set of digital video data; and

wherein the computer code for producing includes computer code for comparing each difference ratio to the threshold.

44. The computer program product according to claim 43, further comprising:

computer code for identifying a source for which the difference ratio is below the threshold; and

computer code for associating the current set of digital video data with the identified source.

45. The computer program product according to claim, 39 wherein the computer code for associating the current set of digital video data with the identified source further comprises:

computer code for identifying the current set of digital video data as a representative digital video data set for a source.

46. The computer program product according to claim 39 further comprising:

computer code for creating a representation of the first set of digital video data that is reduced in data size; and

computer code for creating a representation of the current set of digital video data which has a reduced size;

wherein the reduced first set of digital video data and the reduced current set of digital video data are used for determining the difference ratio.

47. The computer program product according to claim 39, further comprising:

computer code for identifying the current frame as a representative frame of a source if the difference ratio is below the threshold, wherein the set of data for the representative frame is used for comparisons with a set of data for subsequent frames.

48. The computer program product according to claim 39, further comprising computer code for storing the set of data that is representative of the current frame is stored as the representative frame for that source as each different originating source is identified by the computer code for producing.

49. A device for demultiplexing time-division multiplexed digital video data which originates from multiple sources, the device comprising:

a retrieval module for retrieving a first set of digital video data representative of a first image and retrieving a current set of digital video data representative of a current image;

a storage module for storing the first set of digital video data to a memory location associated with a first source;

a calculation module for calculating a difference ratio using the first set of digital video data and the current set of digital video data; and

a query module for producing a display query requiring response for classifying the digital video data as belonging to an identified source or originating from a new source if the difference ratio is above a threshold, wherein the first source is an identified source.

50. The device according to claim 49 wherein the storage module stores the current set of digital video data to a second memory location associated with the second source in accordance with a received response indicating that the current set of digital video data is associated with a new source.

51. The device according to claim 49, wherein the storage module stores the current digital video data to the memory location associated with the first source if the difference ratio is below the threshold.

52. The device according to claim 49, further comprising:

a normalization module for normalizing the digital video data to compensate for differences in brightness prior to providing the digital video data to the calculation module for calculating a difference ratio.

53. The device according to claim 49, wherein the calculation module calculates a difference ratio for each representative digital video data set representative of a source and the current set of digital video data;

and

wherein the query module compares each difference ratio to the threshold.