Exposure control methods and apparatus

• US 9,544,503 B2
• Filed: 02/20/2015
• Issued: 01/10/2017
• Est. Priority Date: 12/30/2014
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Shroff, Sapna A.
• Primary Examiner(s): PRABHAKHER, PRITHAM DAVID
• Application Number:
  US14/627,983
• Publication Number:
  US 20160191776A1
• Time in Patent Office: 690 Days
• Field of Search: 348/362
• US Class Current: 1/1
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

Methods and apparatus for controlling exposure in a camera device are described. A depth map is used in combination with user selection of a portion of scene as part of an exposure control operation. Exposure control is based on portions of the scene, e.g., in a local window surrounding the user selected point, at the same depth as the user selected portion of the scene with other portions of the scene being excluded from consideration when controlling exposure or being given less weight than the portion or portions at the same depth as the user selected scene portion. Color maybe and in some embodiments is used in combination with depth information to identify an object of interest identified by the user selection. The identified object is then used in some embodiments in making exposure control determinations with portions of a scene outside the object being ignored or given less weight in determining an exposure to be used than the portions corresponding to the identified object.

Claims

1. A method of controlling a camera, the method comprising:

receiving user input identifying a portion of a scene to be used in controlling image capture;

determining a depth to which the user identified portion of the scene corresponds; and

performing an exposure control operation based on the user selected portion of the scene and a depth map, said step of performing an exposure control operation including;

determining an exposure time to be used in capturing a new image based on pixel values of a first image of said scene corresponding to said depth to which the user identified portion of the scene corresponds while excluding from consideration pixel values in said first image which do not correspond to said depth or giving pixel values in said first image which do not correspond to said depth less weight when determining the exposure time than pixel values in said first image corresponding to said depth.

2. The method of claim 1, further comprising:

prior to performing the exposure control operation, generating the depth map, said depth map corresponding to the scene.

3. The method of claim 2, wherein generating the depth map includes generating the depth map from a plurality of images captured by different camera modules of said camera.

4. The method of claim 2, wherein generating the depth map includes using one of:

i) a time of flight array of sensors, ii) active illumination with depth information being extracted based on the illumination, or iii) depth determination based on focusing information.

5. The method of claim 1, wherein performing an exposure control operation based on the user selected portion of the scene and said depth map includes:

identifying portions of said first image of said scene corresponding to said depth.

6. The method of claim 5, wherein identifying portions of said first image of said scene corresponding to said depth includes identifying an object corresponding to the user identified portion of the scene.

7. The method of claim 6, wherein identifying the object is based on color of the user identified portion of said scene as well as said depth.

8. The method of claim 1, wherein said camera includes a plurality of optical chains.

9. The method of claim 1, further comprising:

prior to excluding from consideration pixel values in said first image which do not correspond to said depth or giving such pixel values in said first image which do not correspond to said depth less weight when determining the exposure time than pixel values in said first image corresponding to said depth, identifying pixel values in said first image corresponding to said depth, said identifying pixel values in said first image corresponding to said depth including;

identifying as pixel values in said first image corresponding to said depth pixel values in said first image within a predetermined range of said determined depth.

10. The method of claim 9, wherein determining an exposure time includes:

selecting for use in determining said exposure time pixel values in said first image within a predetermined area surrounding the user identified portion of the scene.

11. The method of claim 10, wherein said predetermined area surrounding the user identified portion of the scene is a local window of interest.

12. The method of claim 11, wherein determining an exposure time includes:

determining the dynamic range of elements in the local window of interest.

13. The method of claim 12, further comprising:

performing a high dynamic ranging control operation based on a determined dynamic range of elements in the local window of interest.

14. The method of claim 12, wherein determining the dynamic range of elements in the local window of interest includes estimating the dynamic range from elements in said local window of interest at said depth while excluding from use in said step of determining the dynamic range elements at a different depth.

15. The method of claim 13, wherein performing a high dynamic ranging control operation includes determining a number of images of said scene to be taken and different exposures to be used when taking different ones of said number of images.

16. The method of claim 12, wherein determining the dynamic range of elements in the local window of interest includes giving elements at the different depth less weight in said dynamic range determination than elements at said depth.

17. A camera device, comprising:

a touch sensitive screen for receiving user input identifying a portion of a scene to be used in controlling image capture; and

an exposure control module for performing an exposure control operation based on the user selected portion of the scene and a depth map, wherein performing an exposure control operation includes;

determining a depth to which the user identified portion of the scene corresponds; and

determining an exposure time to be used in capturing a new image based on pixel values of a first image of said scene corresponding to said depth to which the user identified portion of the scene corresponds while excluding from consideration pixel values in said first image which do not correspond to said depth or giving pixel values in said first image which do not correspond to said depth less weight when determining the exposure time than pixel values in said first image corresponding to said depth.

18. A camera device, comprising:

a user input for receiving user input identifying a portion of a scene to be used in controlling image capture; and

a processor configured to;

determine a depth to which the user identified portion of the scene corresponds; and

perform an exposure control operation based on the user selected portion of the scene and a depth map, performing an exposure control operation including determining an exposure time to be used in capturing a new image based on pixel values of a first image of said scene corresponding to said depth to which the user identified portion of the scene corresponds while excluding from consideration pixel values in said first image which do not correspond to said depth or giving pixel values in said first image which do not correspond to said depth less weight when determining the exposure time than pixel values in said first image corresponding to said depth.

19. The camera device of claim 18, wherein said processor is further configured to generate a depth map corresponding to a scene.

20. The camera device of claim 19, wherein said processor is further configured to:

identify portions of said first image of said scene corresponding to said depth.