Intelligent video signal encoding utilizing regions of interest information

• US 8,351,513 B2
• Filed: 12/19/2007
• Issued: 01/08/2013
• Est. Priority Date: 12/19/2006
• Status: Active Grant

• Current Assignee: Allot Communications Limited
• Original Assignee: Allot Communications Limited
• Inventors: Wen, Jiangtao Gene, Dey, Sujit, Arole, Parag, Zan, Jinwen, Bhat, Supriya, Illic, Andjela
• Primary Examiner(s): Wu, Jianye
• Assistant Examiner(s): Russell, Wanda Z
• Application Number:
  US11/960,385
• Publication Number:
  US 20080165861A1
• Time in Patent Office: 1,847 Days
• Field of Search: 375/240.16, 37524023-24026, 382/107, 382/236, 348/700
• US Class Current: 375/240.24
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/11 among a plurality of spatial predictive coding modes
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/194 involving only two passes
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

Digital video content is processed for delivery over a communications channel by segmenting the digital video content into one or more regions of interest (ROI) in accordance with content signature of the video content and encoding the digital video content in accordance with the ROI segmentation and the communications channel.

Claims

1. A method of encoding digital video content for delivery over a communications channel, the method comprising:

receiving output of a trial analysis encoding of the digital video content at an intelligent encoder from a content analysis block, the output of the trial analysis encoding based on the content such that the trial analysis encoding comprises a segmented digital video content that identifies one or more regions of interest (ROI) for each frame in the segmented digital video content in accordance with content signature of each corresponding frame in the digital video content;

determining encoding options for the segmented digital video content of the received trial analysis encoding at the intelligent encoder in accordance with identified ROI of frames in the segmented digital video content, including motion level information, and in accordance with available processing resources;

performing an encoding of the segmented digital video content at the intelligent encoder in accordance with the determined encoding options and the communications channel;

wherein the encoded digital video content is viewable on a display device after decoding by a decoder.

2. A method as defined in claim 1, wherein the content signature indicates the ROI comprises a spatial ROI that defines a region over space in a frame of the digital video content.

3. A method as defined in claim 2, wherein the spatial region is defined according to a macroblock division of the frame.

4. A method as defined in claim 2, wherein performing the trial analysis comprises determining the content signature such that a region that has increased complexity is identified as comprising a spatial ROI.

5. A method as defined in claim 1, wherein the content signature indicates the ROI comprises a temporal ROI that identifies one or more frames of the digital video content as a ROI.

6. A method as defined in claim 5, wherein performing the trial analysis comprises determining the content signature such that motion level information of the video content is determined.

7. A method as defined in claim 6, wherein the motion level information comprises motion vectors that indicates motion for a pixel block of a frame with respect to a reference frame of the digital video content such that a motion vector of greater than a threshold magnitude is assessed to be a high motion frame comprising a temporal ROI.

8. A method as defined in claim 7, wherein encoding is performed such that a frame assessed to be a temporal ROI is encoded at a higher bitrate than a frame that is not assessed to be a temporal ROI.

9. A method as defined in claim 7, wherein the motion level information further includes motion vector difference information that is used to assess whether a frame is a high motion frame.

10. A method as defined in claim 7, wherein a motion level indicator for a frame is computed according to bits expended coding video information for motion information as compared with bits expended coding video information of a different type, such that the ratio determines the value of the motion level indicator.

11. A method as defined in claim 1, wherein performing the trial analysis comprises:

determining the content signature of the digital video content in accordance with an analysis trial;

designating one or more portions of the digital video content as an ROI in accordance with the determined content signature.

12. A method as defined in claim 11, wherein the analysis trial comprises an encoding in which frames of the video content are processed such that a frame region that has increased complexity is identified as comprising a spatial ROI.

13. A method as defined in claim 11, wherein determining the content signature includes performing motion estimation processing for frames of the digital video content such that frames of the digital video content with motion vectors greater than a threshold value are identified as comprising a temporal ROI.

14. A method as defined in claim 11, wherein determining the content signature comprises performing an analysis trial that includes:

a trial analysis encoding comprising motion estimation processing such that frames of the digital video content with motion vectors greater than a threshold value are identified as comprising a temporal ROI;

adjusting frames of the video content in accordance with the temporal ROI trial analysis encoding to eliminate unnecessary frames and produce a temporal ROI adjusted video content;

a trial analysis encoding in which frames of the temporal ROI adjusted video content are processed such that a frame region that has increased complexity is identified as comprising a spatial ROI.

15. A method as defined in claim 1, wherein performing the encoding includes:

determining bandwidth restrictions of the communications channel in real time; and

output encoding in accordance with the ROI and determined bandwidth restrictions.

16. A method as defined in claim 15, wherein output encoding includes selecting quantization parameters of the encoding.

17. A method as defined in claim 15, wherein output encoding includes selecting motion search parameters of the encoding.

18. A method as defined in claim 15, wherein output encoding includes selecting bitrate allocation of the encoding.

19. A method as defined in claim 1, wherein performing the trial analysis encoding is performed with an encoder different from performing the encoding operation.

20. A video coding system for encoding digital video content for delivery over a communications channel, the system comprising:

a content analysis block that performs a trial analysis encoding of the digital video content based on the content, such that the trial analysis encoding produces a segmented digital video content that identifies one or more regions of interest (ROI) for each frame in the segmented digital video content in accordance with content signature of each corresponding frame in the digital video content;

an intelligent encoder that determines encoding options for the segmented digital video content of the trial analysis encoding in accordance with identified ROI of frames in the segmented digital video content, including motion level information, and in accordance with available processing resources, and that encodes the segmented digital video content in accordance with the determined encoding options and the communications channel;

wherein the encoded digital video content is viewable on a display device after decoding by a decoder.

21. A system as defined in claim 20, wherein the content signature indicates the ROI comprises a spatial ROI that defines a region over space in a frame of the digital video content.

22. A system as defined in claim 21, wherein the spatial region is defined according to a macroblock division of the frame.

23. A system as defined in claim 21, wherein the content analysis block determines the content signature such that a region that has increased complexity is identified as comprising a spatial ROI.

24. A system as defined in claim 20, wherein the content signature indicates the ROI comprises a temporal ROI that identifies one or more frames of the digital video content as a ROI.

25. A system as defined in claim 24, wherein the content analysis block segments by determining the content signature such that motion level information of the video content is determined.

26. A system as defined in claim 25, wherein the motion level information comprises motion vectors that indicates motion for a pixel block of a frame with respect to a reference frame of the digital video content such that a motion vector of greater than a threshold magnitude is assessed to be a high motion frame comprising a temporal ROI.

27. A system as defined in claim 26, wherein a frame that is assessed to be a temporal ROI is encoded at a higher bitrate than a frame that is not assessed to be a temporal ROI.

28. A system as defined in claim 26, wherein the motion level information further includes motion vector difference information that is used to assess whether a frame is a high motion frame.

29. A system as defined in claim 26, wherein a motion level indicator for a frame is computed according to bits expended coding video information for motion information as compared with bits expended coding video information of a different type, such that the ratio determines the value of the motion level indicator.

30. A system as defined in claim 20, wherein the content analysis block performs segmenting by operations comprising:

determining the content signature of the digital video content in accordance with an analysis trial;

designating one or more portions of the digital video content as an ROI in accordance with the determined content signature.

31. A system as defined in claim 30, wherein the analysis trial comprises an encoding in which frames of the video content are processed such that a frame region that has increased complexity is identified as comprising a spatial ROI.

32. A system as defined in claim 30, wherein the content analysis block determines the content signature by performing motion estimation processing for frames of the digital video content such that frames of the digital video content with motion vectors greater than a threshold value are identified as comprising a temporal ROI.

33. A system as defined in claim 30, wherein the content analysis block determines the content signature by performing an analysis trial that includes:

a trial analysis encoding comprising motion estimation processing such that frames of the digital video content with motion vectors greater than a threshold value are identified as comprising a temporal ROI;

adjusting frames of the video content in accordance with the temporal ROI trial analysis encoding to eliminate unnecessary frames and produce a temporal ROI adjusted video content;

a trial analysis encoding in which frames of the temporal ROI adjusted video content are processed such that a frame region that has increased complexity is identified as comprising a spatial ROI.

34. A system as defined in claim 20, wherein the encoding performed by the intelligent encoder includes operations comprising:

determining bandwidth restrictions of the communications channel in real time; and

output encoding in accordance with the ROI and determined bandwidth restrictions.

35. A system as defined in claim 34, wherein output encoding includes selecting quantization parameters of the encoding.

36. A system as defined in claim 34, wherein output encoding includes selecting motion search parameters of the encoding.

37. A system as defined in claim 34, wherein output encoding includes selecting bitrate allocation of the encoding.

38. A system as defined in claim 20, wherein the content analysis block performs segmenting with the intelligent encoder.

39. A system as defined in claim 20, wherein the content analysis block performs its operations with an encoder that operates independently of the intelligent encoder.

40. A system as defined in claim 20, wherein the intelligent encoder includes:

an option generator that generates encoding options for the digital video content in accordance with the identified spatial ROIs and temporal ROIs; and

an encoding rule generator that generates encoding rules such that the encoded digital video content is produced in accordance with encoding parameters indicated by the rules.

41. A method of encoding digital video content for delivery over a communications channel, the method comprising:

performing an analysis trial that determines content signature of the digital video content such that the analysis trial comprises an encoding in which frames of the video content are processed such that a frame region that has increased complexity is identified as comprising a spatial ROI and motion estimation processing for frames of the digital video content such that frames of the digital video content with motion vectors greater than a threshold value are identified as comprising a temporal ROI;

segmenting the digital video content into one or more regions of interest (ROI) in accordance with the content signature of each corresponding frame in the digital video content;

determining encoding options for the segmented digital video content of the analysis trial encoding in accordance with identified ROI of frames in the segmented digital video content, including motion level information, and in accordance with available processing resources;

performing an encoding of the segmented digital video content in accordance with the determined encoding options and the communications channel;

wherein the encoded digital video content is viewable on a display device after decoding by a decoder.

42. A method as defined in claim 41, wherein performing the analysis trial comprises:

motion estimation processing wherein frames of the digital video content with motion vectors greater than a threshold value are identified as comprising a temporal ROI;

adjusting frames of the video content in accordance with the temporal ROI trial analysis encoding to eliminate unnecessary frames and produce a temporal ROI adjusted video content;

spatial analysis processing wherein frames of the temporal ROI adjusted video content are processed such that a frame region that has increased complexity is identified as comprising a spatial ROI.

43. A method as defined in claim 41, wherein encoding comprises:

generating encoding options for the digital video content in accordance with the identified spatial ROIs and temporal ROIs; and

generating encoding rules such that the encoded digital video content is produced in accordance with encoding parameters indicated by the rules.

44. A program product for use in a computer system that executes program instructions recorded in a non-transitory computer-readable media to perform a method for encoding digital video content for delivery over a communications channel, the program product comprising:

a non-transitory recordable media;

a program of computer-readable instructions recorded on the non-transitory recordable media, wherein the instructions are executable by the computer system to perform operations comprising;

performing an analysis trial that determines content signature of the digital video content such that the analysis trial comprises an encoding in which frames of the video content are processed such that a frame region that has increased complexity is identified as comprising a spatial ROI and motion estimation processing for frames of the digital video content such that frames of the digital video content with motion vectors greater than a threshold value are identified as comprising a temporal ROI;

segmenting the digital video content into one or more regions of interest (ROI) in accordance with content signature of each corresponding frame in the digital video content;

determining encoding options for the segmented digital video content of the analysis trial encoding in accordance with identified ROI of frames in the segmented digital video content, including motion level information, and in accordance with available processing resources;

performing an encoding of the segmented digital video content in accordance with the determined encoding options and the communications channel;

wherein the encoded digital video content is viewable on a display device after decoding by a decoder.

45. A program product as defined in claim 44, wherein the operation of segmenting comprises:

performing an analysis trial that determines content signature of the digital video content such that the analysis trial comprises an encoding in which frames of the video content are processed such that a frame region that has increased complexity is identified as comprising a spatial ROI and motion estimation processing for frames of the digital video content such that frames of the digital video content with motion vectors greater than a threshold value are identified as comprising a temporal ROI;

segmenting the digital video content into one or more regions of interest (ROI) in accordance with the content signature of the digital video content.