Light sensor beneath a dual-mode display

• US 10,062,322 B2
• Filed: 07/31/2017
• Issued: 08/28/2018
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: Essential Products, Inc.
• Original Assignee: Essential Products, Inc.
• Inventors: Evans, V, David John, Jiang, Xinrui, Rubin, Andrew E., Hershenson, Matthew, Miao, Xiaoyu, Tate, Joseph Anthony, Gagne-Keats, Jason Sean
• Primary Examiner(s): Sherman, Stephen G
• Application Number:
  US15/665,359
• Publication Number:
  US 20170330507A1
• Time in Patent Office: 393 Days
• Field of Search: 345207
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/1601 Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays constructional details related to flat displays integrated in a portable computer, e.g. laptop, handheld computer G06F1/1637; constructional details related to television receivers H04N5/64
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G09G 2360/144 the light being ambient light
• CPC Class Codes: G09G 3/03 specially adapted for displays having non-planar surfaces, e.g. curved displays
• CPC Class Codes: G09G 3/035 for flexible display surfaces
• CPC Class Codes: G09G 3/3208 organic, e.g. using organic light-emitting diodes [OLED]

Abstract

The technology disclosed here integrates a light sensor with a dual-mode display, thus increasing the size of the dual-mode display. The light sensor can be a camera, an ambient sensor, a proximity sensor, etc. The light sensor is placed beneath the dual-mode display and can detect incoming light while the dual-mode display is displaying a display image. The dual-mode display and the light sensor can operate at the same time. For example, a camera placed beneath the dual-mode display can record an image of the environment, while at the same time the dual-mode display is showing the display image. Further, multiple light sensors can be placed at various locations beneath the dual-mode display.

Claims

1. A method comprising:

obtaining a correspondence between a sensor placed beneath a selectively addressable region of a dual-mode display and the selectively addressable region;

causing the dual-mode display to become opaque when displaying a display image;

determining a portion of the display image visible to a user by making a measurement by the sensor while the dual-mode display is displaying the display image;

based on the determination of the portion of the display image visible to the user, reducing consumption of computational resources by displaying the portion of the display image visible to the user; and

during said making the measurement by the sensor, causing the selectively addressable region to stop displaying the display image, such that the selectively addressable region is substantially unnoticeable to the user.

2. The method of claim 1, said establishing the correspondence between the sensor and the selectively addressable region comprising:

configuring a first set of electrodes to activate and to deactivate the dual-mode display except the selectively addressable region; and

configuring a second set of electrodes to activate and to deactivate the selectively addressable region.

3. The method of claim 1, comprising:

recording by a plurality of cameras disposed beneath the dual-mode display a plurality of images;

based on the plurality of images recorded by the plurality of cameras, determining a user'"'"'s focus point on the dual-mode display; and

based on the user'"'"'s focus point, activating a camera in the plurality of cameras closest to the user'"'"'s focus point, said activating the camera comprising recording, by the camera, an image of the user.

4. The method of claim 1, comprising:

when the dual-mode display is not displaying the display image, recording, by the sensor comprising a camera, a calibration image;

when the dual-mode display is displaying the display image, recording, by the camera, an image; and

based on the calibration image, modifying the image to obtain a final image.

5. The method of claim 1, said establishing the correspondence between the sensor and the selectively addressable region comprising:

storing in a memory an identification associated with a plurality of light emitting elements corresponding to the selectively addressable region; and

sending the identification associated with the plurality of light emitting elements to a display controller.

6. The method of claim 1, comprising:

recording by a plurality of sensors disposed beneath the dual-mode display a plurality of measurements;

based on the plurality of measurements recorded by the plurality of sensors, determining a portion of the dual-mode display within a user'"'"'s field of view; and

displaying an image on the dual-mode display in the portion of the dual-mode display within the user'"'"'s field of view.

7. The method of claim 1, comprising:

determining a user'"'"'s gaze by the sensor within the dual-mode display disposed within a lens.

8. The method of claim 1, comprising:

determining an environment surrounding the user by a camera disposed in proximity to a user'"'"'s eye.

9. The method of claim 1, comprising:

determining a user'"'"'s focus point by the sensor within the dual-mode display disposed within a lens proximate to a user'"'"'s eye;

determining an environment surrounding the user by a camera disposed in proximity to the user'"'"'s eye; and

based on the determined user'"'"'s focus point and the determined environment surrounding the user, determining an object within the environment associated with the user'"'"'s focus point.

10. The method of claim 9, comprising:

displaying on the dual-mode display an information associated with the object.

11. The method of claim 10, the object comprising a barcode, and the information associated with the object comprising a different purchasing option of the object.

12. The method of claim 1, the dual-mode display comprising a curved display.

13. The method of claim 1, the dual-mode display comprising a lens disposed in proximity to a user'"'"'s eye.

14. The method of claim 1, the sensor comprising at least one of a camera, an ambient light sensor, or a proximity sensor.

15. A method comprising:

combining a dual-mode display with a lens, wherein the dual-mode display can be transparent or opaque;

placing a sensor beneath a selectively addressable region of the dual-mode display and the selectively addressable region;

configuring the dual-mode display to become opaque when displaying a display image;

determining a portion of the display image visible to a user by making a measurement by the sensor while the dual-mode display is displaying the display image;

based on the determination of the portion of the display image visible to the user, reducing consumption of computational resources by displaying the portion of the display image visible to the user; and

during said making the measurement by the sensor, causing the selectively addressable region to stop displaying the display image, such that the selectively addressable region is substantially unnoticeable to the user.

16. The method of claim 15, comprising:

determining a portion of the dual-mode display within a user'"'"'s field of view; and

displaying the display image on the dual-mode display in the portion of the dual-mode display within the user'"'"'s field of view.

17. The method of claim 15, comprising:

configuring a first set of electrodes to activate and to deactivate the dual-mode display except the selectively addressable region; and

configuring a second set of electrodes to activate and to deactivate the selectively addressable region.

18. The method of claim 15, comprising:

determining a user'"'"'s focus point by the sensor within the dual-mode display disposed within a lens proximate to a user'"'"'s eye;

determining an environment surrounding the user by the sensor disposed in proximity to the user'"'"'s eye; and

based on the determined user'"'"'s focus point and the determined environment surrounding the user, determining an object within the environment associated with the user'"'"'s focus point.

19. The method of claim 18, comprising:

displaying on the dual-mode display an information associated with the object.

20. The method of claim 19, the object comprising a barcode, and the information associated with the object comprising a different purchasing option of the object.