Method and device for analyzing video sequences in a communication network

• US 20040184529A1
• Filed: 02/13/2004
• Published: 09/23/2004
• Est. Priority Date: 02/14/2003
• Status: Active Grant

• Current Assignee: Canon Europa N.V. (Canon Inc.)
• Original Assignee: Canon Europa N.V. (Canon Inc.)
• Inventors: Henocq, Xavier, Clare, Maryline
• Granted Patent: US 7,466,865 B2
• Time in Patent Office: Days
• US Class Current: 375/240.01
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19669 Event triggers storage or change of storage policy
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/66 involving data partitioning, i.e. separation of data into packets or partitions according to importance
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

The invention relates to a method of analyzing at least one video sequence present on at least one sending apparatus, in a communication network also comprising at least one receiving apparatus, wherein, the video sequence being compressed with an encoding method which generates a bitstream comprising information representing the temporal activity of the images of the video sequence and information representing the spatial activity of the images of that video sequence, and the temporal and spatial information being distributed between one or more spatio-temporal hierarchical levels, the method comprises the following steps performed at the receiving apparatus:

obtaining information representing the temporal activity of the video sequence present on the sending apparatus,

analyzing the temporal information obtained,

as a function of the result of the analysis, deciding whether the generation of a request destined for the sending apparatus for the obtainment of information representing the spatial activity of the video sequence is necessary.

Claims

1. A method of analyzing at least one video sequence present on at least one sending apparatus, in a communication network also comprising at least one receiving apparatus, wherein, the video sequence being compressed with an encoding method which generates a bitstream comprising information representing the temporal activity of the images of the video sequence and information representing the spatial activity of the images of that video sequence, and the temporal and spatial information being distributed between one or more spatio-temporal hierarchical levels, the method comprises the following steps performed at the receiving apparatus:

obtaining information representing the temporal activity of the video sequence present on the sending apparatus,

analyzing the temporal information obtained,

depending on the result of the analysis, deciding whether the generation of a request destined for the sending apparatus for the obtainment of information representing the spatial activity of the video sequence is necessary.

2. The method according to claim 1, wherein the step of analyzing the temporal information obtained more particularly comprises the steps of:

determining a temporal activity index of the video sequence on the basis of the information obtained,

comparing the index determined with a predetermined first threshold index.

3. The method according to claim 2, wherein it comprises a step of comparing the index determined with a predetermined second threshold index.

4. The method according to claim 1, wherein the method of encoding is a hybrid method of encoding based on the encoding of data blocks of an image and which generates a bitstream of video data in a plurality of spatio-temporal hierarchical levels.

5. The method according to claim 4, wherein the first hierarchical level comprises header data and movement vectors of the blocks of the images of the video sequence.

6. The method according to claim 5, wherein the information representing the temporal activity is the movement vectors.

7. The method according to claim 1, wherein it comprises the following steps:

obtaining information representing the spatial activity of the video sequence present at the sending apparatus,

analyzing the spatial information obtained,

depending on the result of the analysis, deciding whether the generation of a request destined for the sending apparatus for the obtainment of supplementary information representing the spatial activity of the video sequence is necessary.

8. The method according to claim 7, wherein the step of analyzing of the spatial information obtained more particularly comprises the steps of:

determining a spatial activity index of the video sequence on the basis of the spatial information obtained,

comparing the index determined with a predetermined threshold index.

9. The method according to claim 1, wherein the method further comprises a step of deciding whether the generation of a request destined for the sending apparatus for obtaining supplementary temporal information is necessary depending on the result of the analysis performed on the temporal and/or spatial information obtained.

10. The method according to claim 7, wherein the decision step takes into account the calculation capacity of the receiving apparatus.

11. The method according to claim 7, wherein the decision step takes into account the bandwidth available on the network.

12. The method according to claim 1, wherein, when temporal and spatial information has been obtained from a plurality of sending apparatuses and a decision for obtaining supplementary temporal and/or spatial information from at least one of them has been deemed necessary, the method comprises a step of classifying each sending apparatus with respect to the other apparatuses, as a function of the temporal information and spatial information obtained by that apparatus.

13. The method according to claim 12, wherein the set of sending apparatuses are represented in two Vitterbi type trellises, one dedicated to the temporal activity and the other to the spatial activity, and in each of which each apparatus is represented along the x-axis and, according to the trellis concerned, each spatial or temporal hierarchical level obtained and/or to be obtained is represented along the y-axis.

14. The method according to claim 13, wherein it comprises, for each trellis, a step of determining an optimum path between the apparatuses by only taking into account, for each of them, the highest spatial or temporal hierarchical level.

15. The method according to claim 14, wherein it comprises, for each trellis, a step of determining one or more alternative paths between the apparatuses by performing, for a given trellis, the following operations which are reiterated in order to determine a plurality of alternative paths:

going through the trellis considered,

deleting, according to the trellis considered, the highest spatial or temporal hierarchical level obtained and/or to be obtained for the sending apparatus having, as may be the case, the least spatial activity or temporal activity and having at least two hierarchical levels of information, so leading to the determination of an alternative path to which the aforementioned apparatus contributes for a reduced spatial and/or temporal hierarchical level.

16. The method according to claim 15, wherein it comprises a step of associating a decoding cost with each alternative path determined, the decoding cost associated with a path corresponding to the estimation of the calculations necessary to perform the decoding of encoded spatial and/or temporal information coming from the different sending apparatuses placed on the path.

17. The method according to claim 16, wherein it comprises a step of selecting the optimum path by determining among all the paths the one for which the decoding cost is the closest to the calculation capacity of the receiving apparatus, while remaining less than that capacity.

18. The method according to claim 16, wherein it comprises a step of selecting the optimum path by determining among all the paths the one for which the rate induced is less than the bandwidth available on the network.

38. An information storage means which can be read by a computer or a microprocessor containing code instructions of a computer program for executing the steps of the analyzing method according to claim 1.

39. A partially or totally removable information storage means which can be read by a computer or a microprocessor containing code instructions of a computer program for executing the steps of the analyzing method according to claim 1.

40. A computer program which can be loaded into a programmable apparatus, wherein it contains sequences of instructions or portions of software code for implementing the steps of the analyzing method according to claim 1, when that computer program is loaded and executed by the programmable apparatus.

19. A device for analyzing at least one video sequence present on at least one sending apparatus, in a communication network also comprising at least one receiving apparatus, wherein, the video sequence being compressed with an encoding method which generates a bitstream comprising information representing the temporal activity of the images of the video sequence and information representing the spatial activity of the images of that video sequence, and the temporal and spatial information being distributed between one or more spatio-temporal hierarchical levels, the device comprises at the receiving apparatus:

means for obtaining information representing the temporal activity of the video sequence present at the sending apparatus,

means for analyzing the temporal information obtained,

means for deciding whether the generation of a request destined for the sending apparatus for the obtainment of information representing the spatial activity of the video sequence is necessary, depending on the result of the analysis.

20. The device according to claim 19, wherein the means for analyzing the temporal information obtained more particularly comprises:

means for determining a temporal activity index of the video sequence on the basis of the information obtained,

means for comparing the index determined with a predetermined first threshold index.

21. The device according to claim 20, wherein it comprises means for comparing the index determined with a predetermined second threshold index.

22. The device according to claim 19, wherein the method of encoding is a hybrid method of encoding based on the encoding of data blocks of an image and which generates a bitstream of video data in a plurality of spatio-temporal hierarchical levels.

23. The device according to claim 22, wherein the first hierarchical level comprises header data and movement vectors of the blocks of the images of the video sequence.

24. The device according to claim 23, wherein the information representing the temporal activity is the movement vectors.

25. The device according to claim 19, wherein it comprises:

means for obtaining information representing the spatial activity of the video sequence present at the sending apparatus,

means for analyzing the spatial information obtained,

means for deciding whether the generation of a request destined for the sending apparatus for the obtainment of supplementary information representing the spatial activity of the video sequence is necessary, as a function of the result of the analysis.

26. The device according to claim 25, wherein the means for analyzing the spatial information obtained more particularly comprises:

means for determining a spatial activity index of the video sequence on the basis of the spatial information obtained,

means for comparing the index determined with a predetermined threshold index.

27. The device according to claim 19, wherein the device further comprises means for deciding whether the generation of a request destined for the sending apparatus for obtaining supplementary temporal information is necessary as a function of the result of the analysis performed on the temporal and/or spatial information obtained.

28. The device according to claim 25, wherein the means for deciding take into account the calculation capacity of the receiving apparatus.

29. The device according to claim 25, wherein the means for deciding take into account the bandwidth available on the network.

30. The device according to claim 19, wherein, when temporal and spatial information has been obtained from a plurality of sending apparatuses and a decision for obtaining supplementary temporal and/or spatial information from at least one of them has been deemed necessary, the device comprises means for 30 classifying each sending apparatus with respect to the other sending apparatuses, as a function of the temporal information and spatial information obtained by that apparatus.

31. The device according to claim 19, wherein the set of sending apparatuses are represented in two Vitterbi type trellises, one dedicated to the temporal activity and the other to the spatial activity, and in each of which each apparatus is represented along the x-axis and, according to the trellis concerned, each spatial or temporal hierarchical level obtained and/or to be obtained is represented along the y-axis.

32. The device according to claim 31, wherein it comprises, for each trellis, means for determining an optimum path between the apparatuses by only taking into account, for each of them, the highest spatial or temporal hierarchical level.

33. The device according to claim 32, wherein it comprises, for each trellis, means for determining one or more alternative paths between the sending apparatuses and which make provision for performing, for a given trellis, the following operations which are reiterated in order to determine a plurality of alternative paths:

going through the trellis considered,

deleting, according to the trellis considered, the highest spatial or temporal hierarchical level obtained and/or to be obtained for the sending apparatus having, as may be the case, the least spatial activity or temporal activity and having at least two hierarchical levels of information, so leading to the determination of an alternative path to which the aforementioned apparatus contributes for a reduced spatial and/or temporal hierarchical level.

34. The device according to claim 33, wherein it comprises means for associating a decoding cost with each alternative path determined, the decoding cost associated with a path corresponding to the estimation of the calculations necessary to perform the decoding of encoded spatial and/or temporal information coming from the different sending apparatuses placed on the path.

35. The device according to claim 34, wherein it comprises means for selecting the optimum path by determining among all the paths the one for which 30 the decoding cost is the closest to the calculation capacity of the receiving apparatus, while remaining less than that capacity.

36. The device according to claim 34, wherein it comprises means for selecting the optimum path by determining among all the paths the one for which the rate induced is less than the bandwidth available on the network.

37. A communication receiving apparatus, wherein it comprises an analyzing device according to claim 19.