Continuous illumination of backlit display and of subject for image capture

• US 9,485,398 B2
• Filed: 04/21/2014
• Issued: 11/01/2016
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Patel, Parin
• Primary Examiner(s): Cutler, Albert
• Application Number:
  US14/257,702
• Publication Number:
  US 20140225980A1
• Time in Patent Office: 925 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G09G 2320/0666 for control of colour parameters, e.g. colour temperature
• CPC Class Codes: G09G 2360/14 Detecting light within display terminals, e.g. using a single or a plurality of photosensors
• CPC Class Codes: G09G 2360/144 the light being ambient light
• CPC Class Codes: G09G 2360/16 Calculation or use of calculated indices related to luminance levels in display data
• CPC Class Codes: G09G 3/3406 Control of illumination source illumination devices structurally associated with liquid crystal cells G02F1/1336
• CPC Class Codes: H04M 1/22 Illumination; Arrangements for improving the visibility of characters on dials
• CPC Class Codes: H04M 2250/52 including functional features of a camera
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 5/20 Circuitry for controlling amplitude response
• CPC Class Codes: H04N 7/141 between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085

Abstract

A device to capture an image includes a camera oriented to capture a first image of a subject and a display oriented to display a second image that is viewable by the subject. A backlight is coupled to the display to provide light that passes through the display toward the subject. An image processor is coupled to the camera and the backlight to adjust an amount of light provided by the backlight responsive to a quality of the first image. The light provided by the backlight may illuminate the subject and thereby improve the quality of the first image. The image processor may further adjust the amount of light provided by the backlight responsive to an ambient light level sensed by an ambient light sensor. The image processor may further adjust the second image to adjust an amount of light that passes through the display.

Claims

1. A method for capturing images of a user, the method comprising:

with an electronic device, receiving a first image captured by a first camera;

with a display in the electronic device, emitting light to display the first image and illuminate the user;

with a second camera in the electronic device, capturing at least a second image of the user;

with image processing circuitry in the electronic device and using at least the second image of the user, determining that the user is not properly illuminated by determining that the quality of at least the second image of the user is below a given threshold; and

in response to determining that the user is not properly illuminated, adjusting the light emitted by the display such that the user is properly illuminated, wherein adjusting the light emitted by the display comprises adjusting a brightness of the first image such that the user is properly illuminated, wherein adjusting the brightness of the first image such that the user is properly illuminated comprises increasing the brightness of dark areas of the first image relative to light areas of the first image to increase the amount of light that passes through the display.

2. The method defined in claim 1 wherein determining that the user is not properly illuminated comprises determining that the user is not sufficiently illuminated.

3. The method defined in claim 1 wherein determining that the user is not properly illuminated comprises determining that the user is over-illuminated and wherein adjusting the light emitted by the display comprises decreasing the amount of light emitted by the display.

4. The method defined in claim 1 further comprising:

with an ambient light sensor in the electronic device, sensing an ambient light level; and

based on the sensed ambient light level, further adjusting the light emitted by the display.

5. The method defined in claim 1 wherein the electronic device does not include an ambient light sensor.

6. The method defined in claim 1 wherein determining that the quality of at least the second image of the user is below the given threshold comprises determining that at least the second image has a signal-to-noise ratio below the given threshold.

7. The method defined in claim 1 further comprising:

after adjusting the light emitted by the display such that the user is properly illuminated and with the camera in the electronic device, capturing at least an additional image of the user with the second camera in the electronic device.

8. The method defined in claim 1 wherein determining that the user is not properly illuminated by determining that the quality of at least the second image of the user is below the given threshold comprises determining that at least the second image has an incorrect color balance and wherein adjusting the light emitted by the display such that the user is properly illuminated comprises adjusting a color balance of the display to compensate for the incorrect color balance of at least the second image such that subsequently-captured images of the user have a correct color balance.

9. A method for capturing images of a user, the method comprising:

with an electronic device, receiving a first image captured by a first camera;

with a display in the electronic device, emitting light to display the first image and illuminate the user;

with a second camera in the electronic device, capturing at least a second image of the user;

with image processing circuitry in the electronic device and using at least the second image of the user, determining that the user is not properly illuminated by determining that the quality of at least the second image of the user is below a given threshold; and

in response to determining that the user is not properly illuminated, adjusting the light emitted by the display such that the user is properly illuminated, wherein adjusting the light emitted by the display comprises adjusting a brightness of the first image such that the user is properly illuminated, wherein adjusting the brightness of the first image such that the user is properly illuminated comprises decreasing the brightness of dark areas of the first image relative to light areas of the first image to decrease the amount of light that passes through the display.

10. A method for using a display to provide illumination of a user of an electronic device during a videoconference, the method comprising:

displaying video on the display that illuminates the user;

with a camera in the electronic device, capturing additional video of the user during the videoconference;

with image processing circuitry, analyzing the additional video as it is being captured and, based on the analysis, determining that the user is not properly illuminated, wherein determining that the user is not properly illuminated comprises determining that the user is over-illuminated; and

while continuing to capture and analyze the additional video of the user, adjusting the video on the display until the user is properly illuminated, wherein adjusting the video on the display comprises decreasing a brightness of the video on the display, wherein adjusting the video on the display such that the user is properly illuminated comprises decreasing the brightness of dark areas of the display relative to light areas of the display to decrease the amount of light that passes through the display.

11. The method defined in claim 10 wherein determining that the user is not properly illuminated comprises determining that the user is not sufficiently illuminated and wherein adjusting the video on the display comprises increasing the brightness of the video on the display.

12. The method defined in claim 10 further comprising:

with an ambient light sensor in the electronic device, sensing an ambient light level; and

based on the sensed ambient light level, further adjusting the video on the display video on the display.

13. The method defined in claim 10 wherein the electronic device does not include an ambient light sensor.

14. The method defined in claim 10 wherein determining that the user is not properly illuminated comprises determining that the video has a signal-to-noise ratio below a given threshold.

15. The method defined in claim 10 wherein determining that the user is not properly illuminated comprises determining the additional video has an incorrect color balance and wherein adjusting the video on the display such that the user is properly illuminated comprises adjusting a color balance of the video on the display to compensate for the incorrect color balance such that later captured video of the user has a correct color balance.

16. A method for using a display to provide illumination of a user of an electronic device during a videoconference, the method comprising:

displaying video on the display that illuminates the user;

with a camera in the electronic device, capturing additional video of the user during the videoconference;

with image processing circuitry, analyzing the additional video as it is being captured and, based on the analysis, determining that the user is not properly illuminated, wherein determining that the user is not properly illuminated comprises determining that the user is over-illuminated; and

while continuing to capture and analyze the additional video of the user, adjusting the video on the display until the user is properly illuminated, wherein adjusting the video on the display comprises decreasing a brightness of the video on the display, and wherein adjusting the video on the display such that the user is properly illuminated comprises increasing the brightness of dark areas of the display relative to light areas of the display to increase the amount of light that passes through the display.