Region-sensitive compression of digital video

• US 20060062478A1
• Filed: 08/15/2005
• Published: 03/23/2006
• Est. Priority Date: 08/16/2004
• Status: Abandoned Application

• Current Assignee: Grandeye Limited (Oncam Global Group AG)
• Original Assignee: Grandeye Limited (Oncam Global Group AG)
• Inventors: Cuce, Halil I., Castellari, Andrea Elvis, Mulayim, Adem, Davey, Mark Kenneth, Cetin, Ahmet Enis
• Application Number:
  US11/203,807
• Publication Number:
  US 20060062478A1
• Time in Patent Office: Days
• US Class Current: 382/232
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/643 Communication protocols network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP], H04L65/65
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A video coding method for surveillance videos allowing some regions of the scene to be encoded in an almost lossless manner. Such Regions of Interest (RoI) can be determined a priori or they can be automatically determined in real-time by an intelligent system. The user can set high priority in such regions a priori or the intelligent video analysis algorithm can automatically assign some windows a higher priority compared to the rest of the video. In a preferred embodiment, this can be achieved by canceling the motion estimation and compensation operations, and then decreasing the size of the quantization levels during the encoding process in the RoI. The present inventions can produce MPEG compatible bit-streams without sending any side information specifying the RoI.

Claims

1. A method of intelligently compressing video by applying varying compression methodology to selected regions of interest, comprising the steps of

a. identifying one or more regions of interest in a frame of a video sequence,

b. compressing said one or more regions of interest of said frame using only intra-frame information,

c. compressing exterior of said one or more regions of interest using a plurality of frames of said video sequence, and

d. repeating steps a) to c) and combining compressed data from said exterior of regions of interest with said regions of interest to create a bit-stream representing the plurality of frames of said video sequence.

2. The method of claim 1 in which the said detection algorithm are human posture detection algorithm.

3. The method of claim 1 in which the said detection algorithm are human face detection algorithm.

4. The method of claim 1 in which the linear transformation technique is the Discrete Cosine Transform.

5. The method of claim 1 in which the linear transformation technique is a wavelet transform.

6. The method of claim 1 in which the transform operation is carried out over a plurality of image pixel blocks whose union covers the entire image frame.

7. The method of claim 1 in which the quantized transform domain coefficients are encoded in binary form using Huffman coding.

8. The method of claim 1 in which the quantized transform domain coefficients are encoded in binary form using arithmetic coding.

9. The method of claim 1 wherein said regions of interest are compressed using intra-frame information by canceling the motion estimation and the camera motion compensation process in differentially compressed video data.

10. The method of claim 1 in which differential video data compression methods include MPEG-1 video compression standard.

11. The method of claim 9 in which differential video data compression methods include MPEG-1 video compression standard.

12. The method of claim 1 in which differential video data compression methods include MPEG-2 video compression standard.

13. The method of claim 9 in which differential video data compression methods include MPEG-2 video compression standard.

14. The method of claim 1 in which differential video data compression methods include MPEG-4 video compression standard.

15. The method of claim 9 in which differential video data compression methods include MPEG-4 video compression standard.

16. The method of claim 1 in which motion estimation and camera motion compensation are effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

17. The method of claim 9 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

18. The method of claim 10 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

19. The method of claim 11 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

20. The method of claim 12 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

21. The method of claim 13 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

22. The method of claim 1 in which said regions of interests in the video can be also determined manually by a user.

23. The method of claim 1 in which exterior of RoIs can be compressed using any image and video coding method.

24. The method of claim 1 in which no side information describing said Regions of Interest is transmitted to the encoder.

59. The method of claim 1 wherein said identification of regions of interest involves automatic motion analysis, which includes motion detection, and/or moving region tracking and/or object tracking.

60. The method of claim 1 wherein said identification of regions of interest is sensitive to the position of objects with well defined features such as those of humans and vehicles in said frame.

61. The method of claim 1 wherein said compression operations are based on a MPEG video compression standard, such as MPEG-1, MPEG-2 or MPEG-4.

62. The method of claim 1 wherein said compression of exterior of regions of interest is based on key intra-frames in close sequential (temporal) proximity to the current frame.

63. The method of claim 61 wherein said regions of interest are compressed using intra-frame information by canceling the motion estimation and the camera motion compensation process in differentially compressed video data.

25. A computer readable medium containing programming instructions for intelligently encoding a video sequence by applying varying compression methodology to selected regions of interest comprising

a detection algorithm for automatically identifying one or more regions of interest within a video sequence,

encoding said video sequence, while automatically using different encoding parameters for said regions of interest

26. The computer-readable medium of claim 25 in which the detection algorithm can be human posture detection algorithm.

27. The computer-readable medium of claim 25 in which the detection algorithm can be human face detection algorithm.

28. The computer-readable medium of claim 25, wherein the linear transform can be the Discrete Cosine Transform.

29. The computer-readable medium of claim 25, wherein the linear transform can be a wavelet transform.

30. The computer-readable medium of claim 25, wherein the transform operation can be carried out over plurality of image pixel blocks whose union covers the entire image frame.

31. The computer-readable medium of claim 25, wherein the quantized transform domain coefficients can be encoded in binary form using Huffman coding.

32. The computer-readable medium of claim 25, wherein the quantized transform domain coefficients can be encoded in binary form using arithmetic coding.

33. The computer-readable medium of claim 25, in which said regions of interest are compressed by using intra-frame information by canceling the motion estimation and the camera motion compensation process in differentially compressed video data.

34. The computer-readable medium of claim 25 in which differential video data compression methods include MPEG-1 video compression standard.

35. The computer-readable medium of claim 33 in which differential video data compression methods include MPEG-1 video compression standard.

36. The computer-readable medium of claim 25 in which differential video data compression methods include MPEG-2 video compression standard.

37. The computer-readable medium of claim 33 in which differential video data compression methods include MPEG-2 video compression standard.

38. The computer-readable medium of claim 25 in which differential video data compression methods include MPEG-4 video compression standard.

39. The computer-readable medium of claim 33 in which differential video data compression methods include MPEG-4 video compression standard.

40. The computer-readable medium of claim 25 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

41. The computer-readable medium of claim 34 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

42. The computer-readable medium of claim 35 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

43. The computer-readable medium of claim 36 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

44. The computer-readable medium of claim 37 in which motion estimation and camera motion compensation can be effectively cancelled by artificially defining all of the motion vectors within the said regions of interest with respect to a DC only encoded block outside the regions of interest.

45. The computer-readable medium of claim 25 in which said regions of interests in the video can be also determined manually by a user.

46. The computer-readable medium of claim 25 in which exterior of RoIs can be compressed using any image and video coding method.

47. The computer-readable medium of claim 25 in which no side information describing said Regions of Interest is transmitted to the encoder.

48. A system for processing video streams comprising:

an encoder which automatically uses a first level of resolution coding on identified regions of interest, and a lower level of resolution on other regions within the scene;

an operation which transmits an output stream from said encoder, representing both said regions of interest and said other regions, said output stream being compatible with one or more standard types of decoder; and

a data channel connected to receive said output stream.

49. A system for processing video streams comprising:

an operation which identifies regions of interest within a scene;

an encoder which automatically uses a first level of resolution coding on said regions of interest, and a lower level of resolution on other regions within the scene;

an operation which transmits the combined output stream from said encoder, representing both said regions of interest and said other regions, said output stream being compatible with one or more standard types of decoder; and

a data channel connected to receive said output stream.

50. The system of claim 49 in which no additional information identifying said regions of interest is transmitted with said output stream.

51. A method of intelligently encoding video comprising the actions of:

encoding said video sequence, while automatically using different encoding parameters for said regions of interest.

52. The method of claim 51, in which at least one of said regions of interest is a predefined location.

53. The method of claim 51, in which at least one of said regions of interest is defined by an algorithm detecting changes in the scene.

64. The method of claim 51 wherein said automatic identification of regions of interest involves automatic motion analysis, which includes motion detection, and/or moving region tracking and/or object tracking.

65. The method of claim 51 wherein said identification of regions of interest is sensitive to the position of objects with well defined features such as those of humans and vehicles in said frame.

66. The method of claim 51 wherein said compression is based on a linear transformation technique including the Discrete Cosine Transform or Wavelet Transform wherein the computed transform domain coefficients are quantised and encoded in binary form using a known coding scheme, including Huffman or arithmetic coding.

67. The method of claim 51 wherein said encoding operation is carried out over a plurality of image pixel blocks whose union covers the entire image frame.

68. The method of claim 51 wherein said encoding operation is based on a MPEG video compression standard, such as MPEG-1, MPEG-2 or MPEG-4.

54. A system for communicating video streams comprising:

an operation which identifies regions of interest in a scene;

an encoder which encodes said regions of interest and other regions in the scene using different encoding parameters,

which automatically encodes said regions of interest using compression technology based on intra-frame data,

which encodes said other regions using compression technology based at least in part on inter-frame data,

and which produces an output stream which is compatible with one or more standard types of decoders; and

a data channel connected to receive said output stream.

55. A method comprising the actions of:

identifying regions of interest in a scene;

encoding a video stream;

wherein said encoding automatically compresses said regions of interest less than other regions in said scene during the encoding process, and

wherein said encoding of said other regions uses an inter-frame comparison process.

56. The method of claim 55, in which the encoding of said other regions is more lossy than the encoding of said regions of interest.

57. The method of claim 55, in which the encoding of said regions of interest involves automatically decreasing the size of the quantization levels during the encoding process.

58. A method comprising the actions of:

identifying regions of interest within a scene, wherein said regions of interest are specified as those regions in which a human face is most likely to reside;

encoding both said regions of interest and other regions within the scene, while automatically using different encoding parameters for said regions of interest, to produce an encoded representation of said regions of interest and said other regions; and

transmitting said encoded representation, to thereby represent said regions of interest and said other regions together.

69. A method of intelligently processing video comprising the steps of:

a) identifying one or more regions of interest in a frame of a video sequence;

b) compressing interior of said one or more regions of interest of said frame at a first compression ratio.

c) compressing exterior of said one or more regions of interest of said frame at a second higher compression ratio; and

d) repeating steps a) to c) and combining compressed data from said interior and exterior of regions of interest to create a bit-stream representing the plurality of frames of said video sequence.

70. The method of claim 69 wherein said identification of regions of interest involves automatic motion analysis, which includes motion detection, and/or moving region tracking and/or object tracking.

71. The method of claim 69 wherein said identification of regions of interest is sensitive to the position of objects with well defined features such as those of humans and vehicles in said frame.

72. The method of claim 69 wherein said compression is based on a linear transformation technique including the Discrete Cosine Transform or Wavelet Transform wherein the computed transform domain coefficients are quantised and encoded in binary form using a known coding scheme, including Huffman or arithmetic coding.

73. The method of claim 69 wherein said compression operations are carried out over a plurality of image pixel blocks whose union covers the entire image frame.

74. The method of claim 69 wherein said compression operations are based on a MPEG video compression standard, such as MPEG-1, MPEG-2 or MPEG-4.

75. The method of claim 69 wherein said one or more regions of interest in said video frames are static and can be determined manually by an operator.

76. The method of claim 69 wherein the steps of identification of regions of interest and encoding are performed in separate computer systems.

77. The method of claim 69 wherein side information describing said regions of interest is included with the bit-stream.