Method and device for analyzing video sequences in a communication network

• US 7,466,865 B2
• Filed: 02/13/2004
• Issued: 12/16/2008
• Est. Priority Date: 02/14/2003
• Status: Active Grant

• Current Assignee: Canon Europa N.V. (Canon Inc.)
• Original Assignee: Canon Europa N.V. (Canon Inc.)
• Inventors: Henocq, Xavier, Clare, Maryline
• Primary Examiner(s): Do; Anh Hong
• Application Number:
  US10/779,284
• Publication Number:
  US 20040184529A1
• Time in Patent Office: 1,768 Days
• Field of Search: 382/239, 382/240, 382/238, 375/240, 375/240.21, 600/431, 422/355, 137/1
• US Class Current: 382/239
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19669 Event triggers storage or change of storage policy
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/66 involving data partitioning, i.e. separation of data into packets or partitions according to importance
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

The invention relates to a method of analyzing at least one video sequence present on at least one sending apparatus, in a communication network also comprising at least one receiving apparatus, wherein, the video sequence being compressed with an encoding method which generates a bitstream comprising information representing the temporal activity of the images of the video sequence and information representing the spatial activity of the images of that video sequence, and the temporal and spatial information being distributed between one or more spatio-temporal hierarchical levels, the method comprises the following steps performed at the receiving apparatus:

obtaining information representing the temporal activity of the video sequence present on the sending apparatus,

analyzing the temporal information obtained,

as a function of the result of the analysis, deciding whether the generation of a request destined for the sending apparatus for the obtainment of information representing the spatial activity of the video sequence is necessary.

Claims

1. A method of analyzing a video sequence present on a sending apparatus, in a communication network including a receiving apparatus, the video sequence being compressed and including information representing temporal activity of images of the video sequence and information representing spatial activity of the images of the video sequence, and the temporal and spatial information being distributed between one or more spatio-temporal hierarchical levels, the method comprising:

obtaining information representing the temporal activity of the video sequence present on the sending apparatus,

analyzing the temporal information obtained,

determining whether generation of a request destined for the sending apparatus for obtaining information representing the spatial activity of the video sequence is necessary, based on a result of the analyzing of the temporal information, and

sending to the sending apparatus a request for information representing spatial activity, if it is determined that generation of the request is necessary.

2. The method according to claim 1, wherein the analysis of the temporal information obtained includes:

determining a temporal activity index of the video sequence on the basis of the information obtained, and

comparing the index determined with a predetermined first threshold index.

3. The method according to claim 2, further comprising comparing the index determined with a predetermined second threshold index.

4. The method according to claim 1, further comprising:

obtaining information representing the spatial activity of the video sequence present at the sending apparatus,

analyzing the spatial information obtained, and

determining whether generation of a request destined for the sending apparatus for obtaining supplementary information representing the spatial activity of the video sequence is necessary, based on a result of the analyzing of the spatial information.

5. The method according to claim 4, wherein analyzing the spatial information obtained includes:

determining a spatial activity index of the video sequence on the basis of the spatial information obtained, and

comparing the index determined with a predetermined threshold index.

6. The method according to claim 4, wherein the method further comprises determining whether generation of a request destined for the sending apparatus for obtaining supplementary temporal information is necessary depending on the result of the analysis performed on the temporal and/or spatial information obtained.

7. The method according to claim 4, wherein the determining whether the generation of the request destined for the sending apparatus is necessary includes taking into account the calculation capacity of the receiving apparatus.

8. The method according to claim 4, wherein the determining whether the generation of the request destined for the sending apparatus is necessary includes taking into account the bandwidth available on the network.

9. The method according to claim 6, wherein, in the case that temporal and spatial information has been obtained from a plurality of sending apparatuses and obtaining supplementary temporal and/or spatial information from at least one of them has been deemed necessary, the method comprise classifying each sending apparatus with respect to the other apparatuses, as a function of the temporal information and spatial information obtained by that apparatus.

10. The method according to claim 9, wherein the set of sending apparatuses are represented in two Vitterbi type trellises, one dedicated to the temporal activity and the other to the spatial activity, and in each of which each apparatus is represented along the x axis and, according to the trellis concerned, each spatial or temporal hierarchical level obtained and/or to be obtained is represented along the y axis.

11. The method according to claim 10, comprising, for each trellis, determining an optimum path between the apparatuses by only taking into account, for each of them, the highest spatial or temporal hierarchical level.

12. The method according to claim 11, comprising, for each trellis, determining one or more alternative paths between the apparatuses by performing, for a given trellis, the following operations which are reiterated in order to determine a plurality of alternative paths:

going through the trellis considered,

deleting, according to the trellis considered, the highest spatial or temporal hierarchical level obtained and/or to be obtained for the sending apparatus having, as may be the case, the least spatial activity or temporal activity and having at least two hierarchical levels of information, so leading to the determination of an alternative path to which the aforementioned apparatus contributes for a reduced spatial and/or temporal hierarchical level.

13. The method according to claim 12, comprising associating a decoding cost with each alternative path determined, the decoding cost associated with a path corresponding to the estimation of the calculations necessary to perform the decoding of encoded spatial and/or temporal information coming from the different sending apparatuses placed on the path.

14. The method according to claim 13, comprising selecting the optimum path by determining among all the paths the one for which the decoding cost is the closest to the calculation capacity of the receiving apparatus, while remaining less than that capacity.

15. The method according to claim 13, comprising selecting the optimum path by determining among all the paths the one for which the rate induced is less than the bandwidth available on the network.

16. A device for analyzing a video sequence present on a sending apparatus, in a communication network including a receiving apparatus, wherein, the video sequence is compressed and includes information representing temporal activity of images of the video sequence and information representing spatial activity of the images of the video sequence, the temporal and spatial information being distributed between one or more spatio-temporal hierarchical levels, the device comprising:

a processor; and

a memory for storing a plurality of code portions, including;

a code portion for causing the processor to obtain information representing the temporal activity of the video sequence present at the sending apparatus,

a code portion for causing the processor to analyze the temporal information obtained, and

a code portion for causing the processor to determine whether generation of a request destined for the sending apparatus for obtaining information representing the spatial activity of the video sequence is necessary, depending on a result obtained in the code portion for causing the processor to analyze the temporal information.

17. The device according to claim 16, wherein the code portion for causing the processor to analyze the temporal information obtained includes:

a code portion to determine a temporal activity index of the video sequence on the basis of the information obtained, and

a code portion to compare the index determined with a predetermined first threshold index.

18. The device according to claim 17, comprising a code portion to compare the index determined with a predetermined second threshold index.

19. The device according to claim 16, wherein the video sequence includes image data blocks and video data in a plurality of spatio-temporal hierarchical levels.

20. The device according to claim 19, wherein the plurality of spatio-temporal levels includes a first hierarchical level including header data and movement vectors of the blocks of the images of the video sequence.

21. The device according to claim 20, wherein the information representing the temporal activity is the movement vectors.

22. The device according to claim 16, further comprising:

a code portion for causing the processor to obtain information representing the spatial activity of the video sequence present at the sending apparatus,

a code portion for causing the processor to analyze the spatial information obtained, and

a code portion for causing the processor to determine whether the generation of a request destined for the sending apparatus for obtaining the obtainment of supplementary information representing the spatial activity of the video sequence is necessary, as a function of the result of the analysis.

23. The device according to claim 22, wherein the code portion for causing the processor to analyze the spatial information obtained includes:

a code portion to determine a spatial activity index of the video sequence on the basis of the spatial information obtained,

a code portion to compare the index determined with a predetermined threshold index.

24. The device according to claim 22, wherein the device further comprises a code portion for causing the processor to determine whether the generation of a request destined for the sending apparatus for obtaining supplementary temporal information is necessary as a function of the result of the analysis performed on the temporal and/or spatial information obtained.

25. The device according to claim 22, wherein the code portion for causing the processor to determine whether to generate a request for obtaining supplementary information representing spatial activity takes into account the calculation capacity of the receiving apparatus.

26. The device according to claim 22, wherein the code portion for causing the processor to determine whether to generate a request for obtaining supplementary information representing spatial activity takes into account the bandwidth available on the network.

27. The device according to claim 24 further comprising a code portion for causing the processor to classify each sending apparatus with respect to the other sending apparatuses, as a function of the temporal information and spatial information obtained by that apparatus, as a function of the temporal information and spatial information obtained by that apparatus, in the case that temporal and spatial information has been obtained from a plurality of sending apparatuses and obtaining supplementary temporal and/or spatial information from at least one of them has been deemed necessary.

28. The device according to claim 16, wherein the set of sending apparatuses are represented in two Vitterbi type trellises, one dedicated to the temporal activity and the other to the spatial activity, and in each of which each apparatus is represented along the x-axis and, according to the trellis concerned, each spatial or temporal hierarchical level obtained and/or to be obtained is represented along the y-axis.

29. The device according to claim 28, comprising, for each trellis, a code portion for causing the processor to determine an optimum path between the apparatuses by only taking into account, for each of them, the highest spatial or temporal hierarchical level.

30. The device according to claim 29, comprising, for each trellis, a code portion for causing the processor to determine one or more alternative paths between the sending apparatuses and to make a provision for performing, for a given trellis, the following operations which are reiterated in order to determine a plurality of alternative paths:

going through the trellis considered, and

deleting, according to the trellis considered, the highest spatial or temporal hierarchical level obtained and/or to be obtained for the sending apparatus having, as may be the case, the least spatial activity or temporal activity and having at least two hierarchical levels of information, so leading to the determination of an alternative path to which the aforementioned apparatus contributes for a reduced spatial and/or temporal hierarchical level.

31. The device according to claim 30, comprising a code portion for causing the processor to associate a decoding cost with each alternative path determined, the decoding cost associated with a path corresponding to the estimation of the calculations necessary to perform the decoding of encoded spatial and/or temporal information coming from the different sending apparatuses placed on the path.

32. The device according to claim 31, comprising a code portion for causing the processor to select the optimum path by determining among all the paths the one for which the decoding cost is the closest to the calculation capacity of the receiving apparatus, while remaining less than that capacity.

33. The device according to claim 31, comprising a code portion for causing the processor to select the optimum path by determining among all the paths the one for which the rate induced is less than the bandwidth available on the network.

34. A communication receiving apparatus including an analyzing device according to claim 16.

35. A computer-readable medium storing executable instructions for causing a computer to perform a method of analyzing a video sequence present on a sending apparatus, in a communication network also including a receiving apparatus, the video sequence being compressed and including information representing the temporal activity of the images of the video sequence and information representing the spatial activity of the images of that video sequence, and the temporal and spatial information being distributed between one or more spatio-temporal hierarchical levels, the method comprising:

obtaining information representing the temporal activity of the video sequence present on the sending apparatus,

analyzing the temporal information obtained,

determining whether the generation of a request destined for the sending apparatus for obtaining information representing the spatial activity of the video sequence is necessary based on the result of the analysis, and

sending to the sending apparatus a request for information representing spatial activity, if it is determined that generation of the request is necessary.

36. A device for analyzing a video sequence present on a sending apparatus, in a communication network including a receiving apparatus, wherein the video sequence is compressed and includes information representing the temporal activity of the images of the video sequence and information representing the spatial activity of the images of that video sequence, and the temporal and spatial information is distributed between one or more spatio-temporal hierarchical levels, the device comprising at the receiving apparatus:

means for obtaining information representing the temporal activity of the video sequence present at the sending apparatus;

means for analyzing the temporal information obtained; and

means for determining whether generation of a request destined for the sending apparatus for obtaining information representing the spatial activity of the video sequence is necessary, depending on the result of the analyzing of the temporal information.