CONTROL OF VIDEO ENCODING BASED ON IMAGE CAPTURE PARAMETERS

• US 20110292997A1
• Filed: 05/24/2011
• Published: 12/01/2011
• Est. Priority Date: 11/06/2009
• Status: Abandoned Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: AN, CHEOLHONG, Liang, Liang, Hung, Szepo Robert
• Application Number:
  US13/114,844
• Publication Number:
  US 20110292997A1
• Time in Patent Office: Days
• US Class Current: 375/240.03
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/521 for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

This disclosure describes techniques for improving functionalities of a back-end device, e.g., a video encoder, using parameters detected and estimated by a front-end device, e.g., a video camera. The techniques may involve estimating a blurriness level associated with frames captured during a refocusing process. Based on the estimated blurriness level, the quantization parameter (QP) used to encode blurry frames is adjusted either in the video camera or in the video encoder. The video encoder uses the adjusted QP to encode the blurry frames. The video encoder also uses the blurriness level estimate to adjust encoding algorithms by simplifying motion estimation and compensation in the blurry frames.

Claims

1. A method comprising:

estimating a blurriness level of a frame of video data based on a type of motion detected in the frame; and

encoding, in a video encoder, the frame based at least in part on the estimated blurriness level of the frame.

2. The method of claim 1, wherein encoding comprises selecting a level of quantization to be used for encoding the frame based on the estimated blurriness level.

3. The method of claim 1, further comprising determining whether to estimate the blurriness level of the frame based on the type of motion detected.

4. The method of claim 1, wherein the frame of video data is captured by a video capture module.

5. The method of claim 1, wherein detecting the motion comprises determining a global motion vector associated with the frame of video data.

6. The method of claim 5, further comprising:

comparing the global motion vector to a global motion vector threshold;

estimating the blurriness level when the global motion vector exceeds the global motion vector threshold; and

encoding the frame without estimating the blurriness level when the global motion vector is equal to or less than the global motion vector threshold.

7. The method of claim 6, further comprising:

determining a local motion vector associated with the frame;

comparing the local motion vector to a local motion vector threshold;

estimating the blurriness level when the global motion vector exceeds the global motion vector threshold and the local motion vector exceeds the local motion vector threshold; and

encoding the frame without estimating the blurriness level when the global motion vector is equal to or less than the global motion vector threshold or the local motion vector is equal to or less than the local motion vector threshold.

8. The method of claim 5, wherein estimating the blurriness level comprises estimating the blurriness level based on the global motion vector and one or more parameters associated with the video capture module.

9. The method of claim 8, wherein the parameters associated with the video capture device comprise time exposure and frame rate.

10. The method of claim 1, further comprising:

detecting the motion by detecting change in optical zooming by a zoom factor associated with the frame; and

estimating the blurriness level based on the zoom factor.

11. The method of claim 1, further comprising:

detecting the motion by detecting panning motion associated with the video capture module; and

estimating the blurriness level based on a focus value associated with the frame, when the frame is captured after the panning motion.

12. The method of claim 1, further comprising:

detecting the motion by detecting a face in the frame; and

estimating the blurriness level based on a size of the detected face in the frame.

13. An apparatus comprising:

a blurriness unit configured to estimate a blurriness level of a frame of video data based on a type of motion detected in the frame; and

a video encoder configured to encode the frame based at least in part on the estimated blurriness level of the frame.

14. The apparatus of claim 13, wherein to encode the frame, the video encoder selects a level of quantization to be used for encoding the frame based on the estimated blurriness level.

15. The apparatus of claim 13, wherein the blurriness unit is further configured to determine whether to estimate the blurriness level of the frame based on the type of motion detected.

16. The apparatus of claim 13, further comprising a video capture module configured to capture the frame of video data.

17. The apparatus of claim 13, wherein to detect the motion, the video capture device is further configured to detect a global motion vector associated with the frame of video data.

18. The apparatus of claim 17, wherein the blurriness unit is further configured to:

compare the global motion vector to a global motion vector threshold; and

estimate the blurriness level when the global motion vector exceeds the global motion vector threshold,

wherein the video encoder is further configured to encode the frame without estimating the blurriness level when the global motion vector is equal to or less than the global motion vector threshold.

19. The apparatus of claim 18, wherein the video encoder is further configured to determine a local motion vector associated with the frame, the blurriness unit is further configured to compare the local motion vector to a local motion vector threshold and estimate the blurriness level when the global motion vector exceeds the global motion vector threshold and the local motion vector exceeds the local motion vector threshold, and the video encoder is further configured to encode the frame without estimating the blurriness level when the global motion vector is equal to or less than the global motion vector threshold or the local motion vector is equal to or less than the local motion vector threshold.

20. The apparatus of claim 17, wherein the blurriness unit is configured to estimate the blurriness level based on the global motion vector and one or more parameters associated with the video capture device.

21. The apparatus of claim 20, wherein the parameters associated with the video capture device comprise time exposure and frame rate.

22. The apparatus of claim 13, further comprising:

a video capture module configured to detect the motion by detecting change in optical zooming by a zoom factor associated with the frame; and

a blurriness unit configured to estimate the blurriness level based on the zoom factor.

23. The apparatus of claim 13, further comprising:

a video capture module configured to detect the motion by detecting panning motion associated with the video capture module; and

a blurriness unit configured to estimate the blurriness level based on a focus value associated with the frame, when the frame is captured after the panning motion.

24. The apparatus of claim 13, further comprising:

a video capture module configured to detect the motion by detecting a face in the frame; and

a blurriness unit configured to estimate the blurriness level based on a size of the detected face in the frame.

25. A computer-readable medium comprising instructions for causing a programmable processor to:

estimate a blurriness level of a frame of video data based on a type of motion detected in the frame; and

encode, in a video encoder, the frame based at least in part on the estimated blurriness level of the frame.

26. The computer-readable medium of claim 25, wherein the instructions to encode comprise instruction that cause the processor to select a level of quantization to be used for encoding the frame based on the estimated blurriness level.

27. The computer-readable medium of claim 25, further comprising instructions that cause the processor to determine whether to estimate the blurriness level of the frame based on the type of motion detected.

28. The computer-readable medium of claim 25, wherein the instructions to detect the motion comprise instructions that cause the processor to detect a global motion vector associated with the frame of video data.

29. The computer-readable medium of claim 28, further comprising instructions that causes the processor to:

compare the global motion vector to a global motion vector threshold;

estimate the blurriness level when the global motion vector exceeds the global motion vector threshold; and

encode the frame without estimating the blurriness level when the global motion vector is equal to or less than the global motion vector threshold.

30. The computer-readable medium of claim 29, further comprising instructions that cause the process to:

determine a local motion vector associated with the frame;

compare the local motion vector to a local motion vector threshold;

estimate the blurriness level the global motion vector exceeds the global motion vector threshold and the local motion vector exceeds the local motion vector threshold; and

encode the frame without estimating the blurriness level when the global motion vector is equal to or less than the global motion vector threshold or the local motion vector is equal to or less than the local motion vector threshold.

31. The computer-readable medium of claim 28, wherein the instructions to estimate the blurriness level comprise instructions that cause the processor to estimate the blurriness level based on the global motion vector and one or more parameters associated with the video capture device.

32. The computer-readable medium of claim 31, wherein the parameters associated with the video capture device comprise time exposure and frame rate.

33. The computer-readable medium of claim 25, further comprising instructions that cause the processor to:

detect the motion by detecting change in optical zooming by a zoom factor associated with the frame; and

estimate the blurriness level based on the zoom factor.

34. The computer-readable medium of claim 25, further comprising instructions that cause the processor to:

detect the motion by detecting panning motion associated with the video capture module; and

estimate the blurriness level based on a focus value associated with the frame, when the frame is captured after the panning motion.

35. The computer-readable medium of claim 25, further comprising instructions that cause the processor to:

detect the motion by detecting a face in the frame; and

estimate the blurriness level based on a size of the detected face in the frame.

36. A system comprising:

means for estimating a blurriness level of a frame of video data based on a type of motion detected in the frame; and

means for encoding, the frame based at least in part on the determination whether to estimate the blurriness level of the frame.

37. The system of claim 36, wherein the means for encoding comprise means for selecting a level of quantization to be used for encoding the frame based on the estimated blurriness level.

38. The system of claim 36, further comprising means for determining whether to estimate the blurriness level of the frame based on the type of motion detected.

39. The system of claim 36, wherein the frame of video data is captured by a video capture module.

40. The system of claim 36, wherein the means for detecting the motion comprises means for detecting a global motion vector associated with the frame of video data.

41. The system of claim 40, further comprising:

means for comparing the global motion vector to a global motion vector threshold;

means for estimating the blurriness level when the global motion vector exceeds the global motion vector threshold; and

means for encoding the frame without estimating the blurriness level when the global motion vector is equal to or less than the global motion vector threshold.

42. The system of claim 41, further comprising:

means for determining a local motion vector associated with the frame;

means for comparing the local motion vector to a local motion vector threshold;

means for estimating the blurriness level the global motion vector exceeds the global motion vector threshold and the local motion vector exceeds the local motion vector threshold; and

means for encoding the frame without estimating the blurriness level when the global motion vector is equal to or less than the global motion vector threshold or the local motion vector is equal to or less than the local motion vector threshold.

43. The system of claim 40, wherein the means for estimating the blurriness level comprises means for estimating the blurriness level based on the global motion vector and one or more parameters associated with the video capture device.

44. The system of claim 43, wherein the parameters associated with the video capture device comprise time exposure and frame rate.

45. The system of claim 36, further comprising:

means for detecting the motion by detecting change in optical zooming by a zoom factor associated with the frame; and

means for estimating the blurriness level based on the zoom factor.

46. The system of claim 36, further comprising:

means for detecting the motion by detecting panning motion associated with the video capture module; and

means for estimating the blurriness level based on a focus value associated with the frame, when the frame is captured after the panning motion.

47. The system of claim 36, further comprising:

means for detecting the motion by detecting a face in the frame; and

means for estimating the blurriness level based on a size of the detected face in the frame.