Different rate controller configurations for different cameras of a mobile device

• US 8,917,632 B2
• Filed: 06/06/2010
• Issued: 12/23/2014
• Est. Priority Date: 04/07/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Zhou, Xiaosong, Wu, Hsi-Jung
• Primary Examiner(s): Ngo, Ricky
• Assistant Examiner(s): KAO, WEI PO ERIC
• Application Number:
  US12/794,773
• Publication Number:
  US 20110249076A1
• Time in Patent Office: 1,661 Days
• Field of Search: 370/235, 370/259, 370/260, 370310-350, 375242-254
• US Class Current: 370/260
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04M 1/72469 for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/617 Upgrading or updating of programs or applications for camera control
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/141 between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 : Conference systems

Abstract

Some embodiments provide a method for encoding video pictures captured by a mobile device having at least first and second cameras. The method identifies a first rate controller parameter that is based on at least a first value that quantifies one characteristic of the first camera. The method uses the first rate controller parameter to configure a set of rate controllers to adjust a bit rate for encoding video pictures captured by the first camera. The method identifies a second rate controller parameter that is based on at least a second value that quantifies the same characteristic of the second camera. The first and second values are different, and the difference causes the first and second rate controller parameters to be different. The method uses the second rate controller parameter to configure the set of rate controllers to adjust a bit rate for encoding video pictures captured by the second camera.

Claims

1. A non-transitory computer readable medium of a mobile device having at least first and second cameras, the non-transitory computer readable medium storing a computer program for encoding video pictures captured by said cameras, the computer program comprising sets of instructions for:

identifying a first rate controller parameter that is based, at least in part, on a first value that quantifies a first image capture characteristic of the first camera;

using the first rate controller parameter to configure one or more of a set of rate controllers to adjust a bit rate for encoding video pictures captured by the first camera;

storing a set of state information while encoding video pictures from the first camera, wherein the set of state information comprises at least one of;

buffer fullness, maximum buffer fullness, and data regarding sizes of recently coded video pictures;

identifying a second rate controller parameter that is based, at least in part, on a second value that quantifies the first image capture characteristic of the second camera, wherein the first and second values are different and said difference causes, at least in part, the first and second rate controller parameters to be different;

using the second rate controller parameter to configure one or more of the set of rate controllers to adjust a bit rate for encoding video pictures captured by the second camera;

switching from the first camera to the second camera while the mobile device is being used; and

after switching from the first camera to the second camera, using the set of state information to configure the set of rate controllers to adjust the bit rate for encoding the video pictures captured by the second camera.

2. The computer readable medium of claim 1, wherein the set of rate controllers comprises one rate controller.

3. The computer readable medium of claim 1, wherein using the first rate controller parameter to configure one or more of the set of rate controllers comprises configuring a first rate controller, and wherein using the second rate controller parameter to configure one or more of the set of rate controllers comprises configuring a second rate controller.

4. The computer readable medium of claim 1, wherein the set of instructions for identifying the first rate controller parameter comprises a set of instructions for computing the first rate controller parameter.

5. The computer readable medium of claim 1, wherein the first image capture characteristic of the first and second camera is camera noise that expresses an amount of picture artifacts produced by the camera.

6. The computer readable medium of claim 1, wherein the first image capture characteristic of the first and second camera is the camera resolution.

7. The computer readable medium of claim 1,

wherein the first and second values are visual masking strengths that quantify visibility of encoding artifacts in encoding video pictures captured by the cameras;

and

wherein the first and second rate controller parameters are first and second quantization parameters.

8. The computer readable medium of claim 1,

wherein the first and second cameras are for capturing videos while the mobile device is being used for a video conference.

9. The computer readable medium of claim 1, wherein the set of state information comprises a target bit rate for encoding captured video pictures.

10. A mobile device comprising:

first and second cameras for capturing video pictures;

a module for allowing a switching from the first camera to the second camera during use;

a storage for storing a set of state information while encoding video pictures from either the first camera or the second camera, wherein the set of state information comprises at least one of;

buffer fullness, maximum buffer fullness, and data regarding sizes of recently coded video pictures; and

a set of rate controllers having;

(i) a first configuration that is based on a first rate controller parameter identified based, at least in part, on a first value that quantifies a first image capture characteristic of the first camera; and

(ii) a second configuration that is based on a second rate controller parameter identified based, at least in part, on a second value that quantifies the first image capture characteristic for the second camera,

wherein the first and second values are different and wherein the difference between the first and second values causes, at least in part, the first and second rate controller parameters to differ, and

wherein the set of rate controllers produces a quantization parameter based, at least in part, on the rate controller parameters; and

a video encoder for encoding video pictures using the quantization parameter received from the set of rate controllers,

wherein, after a switch is made from the first camera to the second camera, the set of state information is used to configure the set of rate controllers to adjust the bit rate for encoding the video pictures captured by the second camera.

11. The mobile device of claim 10, wherein the set of rate controllers comprises one rate controller, and wherein the first and second configurations are different configurations of the rate controller.

12. The mobile device of claim 10, wherein the set of rate controllers comprises a first rate controller and a second rate controller, wherein (i) the first configuration is a configuration of the first rate controller, and (ii) the second configuration is a configuration of the second rate controller.

13. The mobile device of claim 10,

wherein the first and second values are visual masking strengths that quantify visibility of encoding artifacts in encoding video pictures captured by the cameras;

and

wherein the first and second rate controller parameters are first and second quantization parameters.

14. The mobile device of claim 10, wherein the video encoder comprises a quantizer that quantizes an encoding of the video pictures based on the quantization parameter.

15. The mobile device of claim 10, wherein the first image capture characteristic of the first and second camera is camera noise that expresses an amount of picture artifacts produced by the camera.

16. The mobile device of claim 10, wherein the first image capture characteristic of the first and second camera is the camera resolution.

17. A method of encoding videos captured by first and second cameras of a mobile device, the method comprising:

generating a first rate controller parameter based on at least a first value that quantifies one image capture characteristic of the first camera;

generating a second rate controller parameter based on at least a second value that quantifies the same image capture characteristic of the second camera, wherein the first and second values differ;

storing a set of state information while encoding video pictures from the first camera, wherein the set of state information comprises at least one of;

buffer fullness, maximum buffer fullness, and data regarding sizes of recently coded video pictures;

defining a set of rate controllers for adjusting a bit rate for encoding video pictures captured by either the first camera or the second camera based on the generated first or second rate controller parameter;

switching from the first camera to the second camera while the mobile device is being used; and

after switching from the first camera to the second camera, using the set of state information to configure the set of rate controllers to adjust the bit rate for encoding the video pictures captured by the second camera.

18. The method of claim 17, wherein the set of rate controllers comprises:

one rate controller; and

first and second configurations, wherein the first and second configurations are different configurations of the rate controller.

19. The method of claim 17, wherein the set of rate controllers comprises:

a first rate controller;

a second rate controller; and

first and second configurations, wherein the first configuration is a configuration of the first rate controller, and wherein the second configuration is a configuration of the second rate controller.