Camera parameter-assisted video encoding

• US 8,837,576 B2
• Filed: 05/05/2010
• Issued: 09/16/2014
• Est. Priority Date: 11/06/2009
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Liang, Liang, Hung, Szepo R., Velarde, Ruben M., Noyes, Ying X., Wang, Haiyin
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Werner, David N
• Application Number:
  US12/774,467
• Publication Number:
  US 20110109758A1
• Time in Patent Office: 1,595 Days
• Field of Search: 348/222.1, 348/223.1, 348/345, 348/349, 348/353, 348/E5.067, 375/240
• US Class Current: 375/240
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence

Abstract

This disclosure describes techniques for improving functionalities of a back-end device, e.g., a video encoder, using parameters detected by a front-end device, e.g., a video camera. The techniques may involve detecting a scene change in a captured frame, based on one or more parameters of auto exposure (AE), auto white balance (AWB), and auto focus (AF) functions. If a scene change is detected in a captured frame, a video processing device, which may be a stand-alone device, or may be integrated into one of the front-end or back-end devices, provides an indication of the scene change. The video encoder interprets the signal as a trigger to encode the frame indicated as the frame where a scene change occurred as a reference I frame.

Claims

1. A method comprising:

obtaining one or more video capture parameters associated with captured video frames, wherein the one or more video capture parameters comprise an automatic exposure parameter, an automatic focus parameter, and an automatic white-balance parameter;

calculating a first scene change detection result based on the automatic exposure parameter;

calculating a second scene change detection result based on the automatic focus parameter;

calculating a third scene change detection result based on the automatic white-balance parameter;

calculating an overall scene change decision value based on a weighted sum of the first scene change detection result, the second scene change detection result, and the third scene change detection result;

detecting a scene change in the captured video frames if the overall scene change decision value exceeds a threshold value; and

encoding the captured video frames based on the detected scene change.

2. The method of claim 1, wherein obtaining the one or more video capture parameters comprises obtaining the one or more video capture parameters in a video capture device that captured the video frames.

3. The method of claim 2, wherein detecting the scene change comprises detecting the scene change in the video capture device.

4. The method of claim 1, wherein encoding the captured video frames comprises encoding the captured video frames in a video encoding device, and detecting the scene change comprises detecting the scene change in the video encoding device based on the one or more video capture parameters.

5. The method of claim 1, wherein the one or more video capture parameters comprises an automatic exposure parameter, wherein the automatic exposure parameter indicates a change in brightness between at least two of the captured video frames, and wherein detecting the scene change comprises detecting the scene change if the change in brightness of the at least two captured video frames exceeds a threshold value.

6. The method of claim 5, wherein the automatic exposure parameter indicates at least one of a change in average brightness of the at least two of the captured video frames and a change in block-level brightness of the at least two of the captured video frames.

7. The method of claim 1, wherein the one or more video capture parameters comprises an automatic focus parameter, wherein the automatic focus parameter indicates a position of a lens associated with one of the captured video frames, and wherein detecting the scene change comprises:

determining an amount of change in the lens position of at least two of the captured video frames;

comparing the amount of change with a range value associated with the lens; and

detecting the scene change if the amount of change exceeds the range value.

8. The method of claim 1, wherein the one or more video capture parameters comprises an automatic white balance parameter, wherein the automatic white balance parameter indicates a white balance category of one of the captured video frames, wherein detecting the scene change comprises:

comparing the white balance categories of at least two of the captured video frames; and

detecting the scene change if the white balance categories are different.

9. The method of claim 1, wherein encoding the captured video frames comprises encoding a reference I frame when the scene change is detected.

10. A device comprising:

a video capture device that obtains one or more video capture parameters associated with captured video frames, wherein the one or more video capture parameters comprise an automatic exposure parameter, an automatic focus parameter, and an automatic white-balance parameter;

a processor configured to

calculate a first scene change detection result based on the automatic exposure parameter,

calculate a second scene change detection result based on the automatic focus parameter,

calculate a third scene change detection result based on the automatic white-balance parameter,

calculate an overall scene change decision value based on a weighted sum of the first scene change detection result, the second scene change detection result, and the third scene change detection result, and

detect a scene change in the captured video frames if the overall scene change decision value exceeds a threshold value; and

a video encoder device that encodes the captured video frames based on the detected scene change.

11. The device of claim 10, wherein the processor resides in the video capture device.

12. The device of claim 10, wherein the processor resides in the video encoder device.

13. The device of claim 10, wherein the one or more video capture parameters comprises an automatic exposure parameter, wherein the automatic exposure parameter indicates a change in brightness between at least two of the captured video frames, and wherein detecting the scene change comprises detecting the scene change if the change in brightness of the at least two captured video frames exceeds a threshold value.

14. The device of claim 13, wherein the automatic exposure parameter indicates at least one of a change in average brightness of the at least two of the captured video frames and a change in block-level brightness of the at least two of the captured video frames.

15. The device of claim 10, wherein the one or more video capture parameters comprises an automatic focus parameter, wherein the automatic focus parameter indicates a position of a lens associated with one of the captured video frames, and wherein detecting the scene change comprises:

determining an amount of change in the lens position of at least two of the captured video frames;

comparing the amount of change with a range value associated with the lens; and

detecting the scene change if the amount of change exceeds the range value.

16. The device of claim 10, wherein the one or more video capture parameters comprises an automatic white balance parameter, wherein the automatic white balance parameter indicates a white balance category of one of the captured video frames, wherein detecting the scene change comprises:

comparing the white balance categories of at least two of the captured video frames; and

detecting the scene change if the white balance categories are different.

17. The device of claim 10, wherein encoding the captured video frames comprises encoding a reference I frame when the scene change is detected.

18. A computer-readable medium storing instructions for causing a programmable processor to:

obtain one or more video capture parameters associated with captured video frames, wherein the one or more video capture parameters comprise an automatic exposure parameter, an automatic focus parameter, and an automatic white-balance parameter;

calculate a first scene change detection result based on the automatic exposure parameter;

calculate a second scene change detection result based on the automatic focus parameter;

calculate a third scene change detection result based on the automatic white-balance parameter;

calculate an overall scene change decision value based on a weighted sum of the first scene change detection result, the second scene change detection result, and the third scene change detection result;

detect a scene change in the captured video frames if the overall scene change decision value exceeds a threshold value; and

encode the captured video frames based on the detected scene change.

19. The computer-readable medium of claim 18, wherein the instructions to obtain the one or more video capture parameters comprise instructions to obtain the one or more video capture parameters in a video capture device that captured the video frames.

20. The computer-readable medium of claim 19, wherein the instructions to detect the scene change comprise instructions to detect the scene change in the video capture device.

21. The computer-readable medium of claim 18, wherein the instructions to encode the captured video frames comprise instructions to encode the captured video frames in a video encoding device, and the instructions to detect the scene change comprise instructions to detect the scene change in the video encoding device based on the one or more video capture parameters.

22. The computer-readable medium of claim 18, wherein the one or more video capture parameters comprises an automatic exposure parameter, wherein the automatic exposure parameter indicates a change in brightness between at least two of the captured video frames, and wherein the instructions to detect the scene change comprise instructions to detect the scene change if the change in brightness of the at least two captured video frames exceeds a threshold value.

23. The computer-readable medium of claim 22, wherein the automatic exposure parameter indicates at least one of a change in average brightness of the at least two of the captured video frames and a change in block-level brightness of the at least two of the captured video frames.

24. The computer-readable medium of claim 18, wherein the one or more video capture parameters comprises an automatic focus parameter, wherein the automatic focus parameter indicates a position of a lens associated with one of the captured video frames, and wherein the instructions to detect the scene change comprise instructions to:

determine an amount of change in the lens position of at least two of the captured video frames;

compare the amount of change with a range value associated with the lens; and

detect the scene change if the amount of change exceeds the range value.

25. The computer-readable medium of claim 18, wherein the one or more video capture parameters comprises an automatic white balance parameter, wherein the automatic white balance parameter indicates a white balance category of one of the captured video frames, wherein the instructions to detect the scene change comprise instructions to:

compare the white balance categories of at least two of the captured video frames; and

detect the scene change if the white balance categories are different.

26. The computer-readable medium of claim 18, wherein instructions to encode the captured video frames comprise instructions to encode a reference I frame when the scene change is detected.

27. A device comprising:

means for obtaining one or more video capture parameters associated with captured video frames, wherein the one or more video capture parameters comprise an automatic exposure parameter, an automatic focus parameter, and an automatic white-balance parameter;

means for calculating a first scene change detection result based on the automatic exposure parameter;

means for calculating a second scene change detection result based on the automatic focus parameter;

means for calculating a third scene change detection result based on the automatic white-balance parameter;

means for calculating an overall scene change decision value based on a weighted sum of the first scene change detection result, the second scene change detection result, and the third scene change detection result;

means for detecting a scene change in the captured video frames if the overall scene change decision value exceeds a threshold value; and

means for encoding the captured video frames based on the detected scene change.

28. The device of claim 27, wherein the means for obtaining the one or more video capture parameters comprises means for obtaining the one or more video capture parameters in a video capture device that captured the video frames.

29. The device of claim 28, wherein the means for detecting the scene change comprise means for detecting the scene change in the video capture device.

30. The device of claim 27, wherein the means for encoding the captured video frames comprises means for encoding the captured video frames in a video encoding device, and the means for detecting the scene change comprises means for detecting the scene change in the video encoding device based on the one or more video capture parameters.

31. The device of claim 27, wherein the one or more video capture parameters comprises an automatic exposure parameter, wherein the automatic exposure parameter indicates a change in brightness between at least two of the captured video frames, and wherein the means for detecting the scene change comprises means for detecting the scene change if the change in brightness of the at least two captured video frames exceeds a threshold value.

32. The device of claim 31, wherein the automatic exposure parameter indicates at least one of a change in average brightness of the at least two of the captured video frames and a change in block-level brightness of the at least two of the captured video frames.

33. The device of claim 27, wherein the one or more video capture parameters comprises an automatic focus parameter, wherein the automatic focus parameter indicates a position of a lens associated with one of the captured video frames, and wherein the means for detecting the scene change comprises:

means for determining an amount of change in the lens position of at least two of the captured video frames;

means for comparing the amount of change with a range value associated with the lens; and

means for detecting the scene change if the amount of change exceeds the range value.

34. The device of claim 27, wherein the one or more video capture parameters comprises an automatic white balance parameter, wherein the automatic white balance parameter indicates a white balance category of one of the captured video frames, wherein the means for detecting the scene change comprises:

means for comparing the white balance categories of at least two of the captured video frames; and

means for detecting the scene change if the white balance categories are different.

35. The device of claim 27, wherein the means for encoding the captured video frames comprise means for encoding a reference I frame when the scene change is detected.