Systems and methods for adjusting focus based on focus target information

• US 10,270,958 B2
• Filed: 09/01/2016
• Issued: 04/23/2019
• Est. Priority Date: 09/01/2016
• Status: Active Grant

• Current Assignee: Duelight LLC
• Original Assignee: Duelight LLC
• Inventors: Rivard, William Guie, Kindle, Brian J., Feder, Adam Barry
• Primary Examiner(s): Tissire, Abdelaaziz
• Application Number:
  US15/254,964
• Publication Number:
  US 20180063409A1
• Time in Patent Office: 964 Days
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A system, method, and computer program product are provided for adjusting focus based on focus target information. In use, an image is sampled as image data, using a camera module. Next, the image data is transmitted for processing, wherein the processing includes identifying one or more focus regions. Additionally, focus target information is determined corresponding to one or more focus regions. Further, a focus is adjusted, based on the focus target information, and the image is captured. Additional systems, methods, and computer program products are also presented.

Claims

1. An apparatus, comprising:

a camera module, comprising;

an image sensor, configured to convert a first optical scene information to a first electrical representation, wherein the first electrical representation includes first focus pixel information;

a lens assembly, configured to adjust focus of the first optical scene information based on a focus control signal;

a controller, coupled to the image sensor and the lens assembly and configured to;

receive focus target information, wherein the focus target information includes at least one of;

two dimensional (2D) coordinates, pixel image plane coordinates, integer pixel coordinates, normalized coordinates, or XY coordinates;

receive the first electrical representation from the image sensor;

continuously generate, by a focus control system within the controller, the focus control signal based on the focus target information and the first focus pixel information;

transmit the focus control signal to the lens assembly;

generate, by an array of analog-to-digital converter circuits, a digital representation of the first electrical representation; and

a camera interconnect, coupled to the controller, and configured to transmit the focus target information to the controller, and the digital representation from the controller,

wherein the lens assembly, the image sensor and the controller are packaged together in a rigid housing; and

an application processor, coupled to the camera interconnect, and configured to;

receive the digital representation;

identify one or more focus regions based on the digital representation;

continuously generate the focus target information based on the one or more focus regions; and

cause transmission of the focus target information to the controller through the camera interconnect,

wherein the application processor is packaged outside of the rigid housing.

2. The apparatus of claim 1, wherein the one or more focus regions include one or more pixel points.

3. The apparatus of claim 1, wherein the one or more focus regions include one or more pixel points with a radius and weight profile.

4. The apparatus of claim 3, wherein the weight profile includes a start point and an end point, with a predefined function to be applied between the start point and the end point.

5. The apparatus of claim 1, wherein the one or more focus regions include one or more pixel points with a weight mask.

6. The apparatus of claim 5, wherein one or more weights associated with the weight mask are used to establish a focus estimate.

7. The apparatus of claim 1, wherein first information from one or more focus pixels associated with the one or more focus regions is aggregated to generate a focus estimate for the one or more focus regions, the focus estimate based on the focus target information.

8. The apparatus of claim 1, wherein the focus target information is saved as metadata.

9. The apparatus of claim 1, wherein the one or more identified focus regions are selected by a user.

10. The apparatus of claim 1, wherein the image sensor is further configured to convert a second optical scene information to a second electrical representation, wherein the second electrical representation includes second focus pixel information, the lens assembly is further configured to adjust focus of the second optical scene information, and the controller is further configured to:

receive second focus target information;

receive the second electrical representation from the image sensor; and

continuously generate, by the focus control system, a second focus control signal based on the second focus target information and the second focus pixel information.

11. The apparatus of claim 10, wherein the application processor is configured to blend a first image based on the first optical scene information and the second optical scene information.

12. The apparatus of claim 10, wherein the focus target information corresponds with a first focus setting, and the second focus target information corresponds with a second focus setting.

13. The apparatus of claim 12, wherein the first focus setting is used to capture a first image, and the second focus setting is used to capture a second image, wherein the first image and the second image are blended to create a first blended image.

14. The apparatus of claim 1, wherein the controller is further configured to:

receive updated focus target information; and

in response to receiving the updated focus target information, adjust the lens assembly, based on the updated focus target information.

15. The apparatus of claim 1, wherein to continuously adjust focus, the controller is configured to:

generate a focus estimate based on the focus target information and second focus target information associated with a second optical scene;

generate a second focus signal based on the focus estimate; and

transmit the focus signal to the lens assembly to cause the lens assembly to adjust the focus.

16. The apparatus of claim 15, wherein the focus estimate is generated according to a weighted sum of two or more products, wherein each product is calculated by multiplying a focus pixel difference signal by a corresponding weight value.

17. The apparatus of claim 1, wherein the focus target information includes at least one coordinate and is represented as at least one of:

a list, an array, or a bit map.

18. The apparatus of claim 1, wherein the controller is further configured to translate at least one coordinate of the focus target information from at least one of:

a three-dimensional (3D) mapping to a two-dimensional (2D) mapping, an image sensor row and column mapping to an image plane mapping, or an XY image coordinate to a pixel image coordinate.

19. The apparatus of claim 1, wherein the one or more focus regions includes an object, and the application processor tracks the object such that when the object moves, the application processor updates the focus target information to maintain focus.

20. A method, comprising:

receiving, using a controller coupled to an image sensor and a lens assembly, focus target information, wherein the focus target information includes at least one of;

two dimensional (2D) coordinates, pixel image plane coordinates, integer pixel coordinates, normalized coordinates, or XY coordinates;

receiving, using the controller, a first electrical representation, including first focus pixel information from the image sensor;

continuously generating, using the controller, by a focus control system within the controller, a focus control signal, configured to adjust focus of a first optical scene information of the lens assembly, based on the focus target information and the first focus pixel information;

transmitting, using the controller, the focus control signal to the lens assembly;

generating, using the controller, by an array of analog-to-digital converter circuits, a digital representation of the first electrical representation;

receiving, using an application processor, the digital representation;

identifying, using the application processor, one or more focus regions based on the digital representation;

continuously generating, using the application processor, the focus target information based on the one or more focus regions; and

transmitting, using the application processor, the focus target information to the controller through a camera interconnect;

wherein the lens assembly, the image sensor and the controller are packaged together in a rigid housing, and the application processor is packaged outside of the rigid housing.