Integration of a light collection light-guide with a field sequential color display

• US 9,454,265 B2
• Filed: 09/23/2013
• Issued: 09/27/2016
• Est. Priority Date: 09/23/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Wyrwas, John Michael, Jacobs, Paul Eric, Gousev, Evgeni Petrovich, Gruhlke, Russell Wayne, Huang, Chung-Po, Seo, Hae-Jong, Cui, Xiquan, Maitan, Jacek
• Primary Examiner(s): Ghebretinsae, Temesghen
• Assistant Examiner(s): Ketema, Benyam
• Application Number:
  US14/034,374
• Publication Number:
  US 20150084927A1
• Time in Patent Office: 1,100 Days
• Field of Search: 345/173, 345/175
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G09G 2360/141 the light conveying information used for selecting or modulating the light emitting or modulating element
• CPC Class Codes: G09G 3/3433 using light modulating elements actuated by an electric field and being other than liquid crystal devices and electrochromic devices using liquid crystal devices G09G3/36; using electrochromic devices G09G3/38
• CPC Class Codes: G09G 3/346 based on modulation of the reflection angle, e.g. micromirrors micromirrors devices per se G02B26/0833

Abstract

This disclosure provides systems, methods and apparatus for touch and gesture recognition, using a field sequential color display. The display includes a processor, a lighting system, and an arrangement for spatial light modulation that includes an array of light modulators. Each light modulator is switchable between an open position that permits transmittance of light from the lighting system through a respective aperture and a shut position that blocks light transmission through the respective aperture. The processor switches the light modulators in accordance with a first modulation scheme to render an image and in accordance with a second modulation scheme to selectively pass object illuminating light through at least one of the respective apertures. A light sensor receives light resulting from interaction of the object illuminating with an object and outputs a signal to the processor. The processor recognizes, from the output of the light sensor, a characteristic of the object.

Claims

1. An apparatus comprising:

an interactive field sequential color (FSC) display having a front surface including a viewing area, the FSC display including a display lighting system and an arrangement for spatial light modulation, the arrangement for spatial light modulation including an array of light modulators and apertures, each light modulator configured to be switched between an open position that permits transmittance of light from the display lighting system through a respective one of the apertures to the front surface and a shut position that blocks light transmission through the respective aperture;

a transparent substrate, disposed between the display lighting system and the front surface and substantially parallel to the front surface, and having a periphery at least coextensive with the viewing area, the transparent substrate including a plurality of light turning elements selectively located in areas not directly above the apertures;

at least one light sensor disposed outside the periphery of the transparent substrate; and

a processor;

wherein;

the light modulators are switched in accordance with a first modulation scheme to render an image, the first modulation scheme adapted to utilize a persistence of vision phenomenon to render color images by sequentially alternating the color of visible light emitted by the display lighting system;

the transparent substrate is configured to pass light emitted by the display lighting system toward the front surface and to receive light reflected through the front surface from an object, the transparent substrate including a first light-turning arrangement that redirects a portion of the received light toward the at least one light sensor;

the light sensor is configured to output, to the processor, a signal representative of a characteristic of the received, redirected light; and

the processor is configured to;

switch the light modulators in accordance with a second modulation scheme to selectively pass object illuminating light from the display lighting system through at least one of the respective apertures, the object illuminating light being at least partially unrelated to the image; and

recognize, from the output of the light sensor, a characteristic of the object.

2. The apparatus of claim 1, wherein:

the display lighting system includes at least one infrared (IR) light emitter;

the at least one light sensor includes an IR light sensor;

the transparent substrate is configured to pass IR light emitted by the display lighting system toward the front surface and to receive IR light scattered through the display front surface from the object; and

the first light-turning arrangement redirects a portion of the received IR light toward the IR light sensor.

3. The apparatus of claim 2, wherein the transparent substrate includes a partial layer of a coating material, the coating material being opaque to visible light and substantially transparent to IR light.

4. The apparatus of claim 3, wherein the partial layer is selectively located in regions of the transparent substrate that are spatially removed from the corresponding apertures.

5. The apparatus of claim 1, wherein the second modulation scheme includes a sensing pattern interspersed between visible image patterns.

6. The apparatus of claim 5, wherein the sensing pattern includes a raster scan.

7. The apparatus of claim 1, wherein the array of light modulators includes a layer disposed on the rear surface of the transparent substrate.

8. The apparatus of claim 1, wherein the first light-turning arrangement is proximate to a front surface of the transparent substrate.

9. The apparatus of claim 1 wherein the processor is configured to recognize, from an output of the IR light sensor, a location of the object.

10. The apparatus of claim 9 wherein the display lighting system emits visible light during a first number of sub-frames and emits IR light during a second number of sub-frames.

11. The apparatus of claim 10 wherein a ratio of the first number to the second number is 10:

1.

12. The apparatus of claim 10 wherein the IR light emitter is flashed during a sub-frame where image data is being displayed.

13. The apparatus of claim 1, wherein the processor controls the display, responsive to the characteristic.

14. The apparatus of claim 1, wherein the characteristic is one or more of a location, or a motion of the object.

15. The apparatus of claim 1, wherein the first light-turning arrangement includes one or more of diffractive, refractive or holographic elements.

16. The apparatus of claim 1, wherein the object includes one or more of a hand, finger, hand held object, and other object under control of a user.

17. An apparatus comprising:

an interactive field sequential color (FSC) display having a front surface including a viewing area, the FSC display including a display lighting system and an arrangement for spatial light modulation, the arrangement for spatial light modulation including an array of light modulators and apertures, each light modulator configured to be switched between an open position that permits transmittance of light from the display lighting system through a respective one of the apertures to the front surface and a shut position that blocks light transmission through the respective aperture;

a transparent substrate, disposed between the display lighting system and the front surface and substantially parallel to the front surface, and having a periphery at least coextensive with the viewing area, the transparent substrate including a plurality of light turning elements selectively located in areas not directly above the apertures;

at least one light sensor disposed outside the periphery of the transparent substrate; and

means for recognizing, from the output of the light sensor, a characteristic of the object; and

means for switching the light modulators wherein;

the light modulators are switched in accordance with a first modulation scheme to render an image, and in accordance with a second modulation scheme to selectively pass object illuminating light from the display lighting system through at least one of the respective apertures, the object illuminating light being at least partially unrelated to the image, the first modulation scheme adapted to utilize a persistence of vision phenomenon to render color images by sequentially alternating the color of visible light emitted by the display lighting system;

the transparent substrate is configured to pass light emitted by the display lighting system toward the front surface and to receive light reflected through the front surface from an object, the transparent substrate including a first light-turning arrangement that redirects a portion of the received light toward the at least one light sensor;

the light sensor is configured to output, to the processor, a signal representative of a characteristic of the received, redirected light.

18. The apparatus of claim 17, wherein the second modulation scheme includes a sensing pattern interspersed between visible image patterns.

19. A method comprising:

switching, with a processor, one or more light modulators of an array of light modulators, wherein;

a field sequential color (FSC) display has a front surface including a viewing area, the FSC display including a display lighting system and an arrangement for spatial light modulation, the arrangement for spatial light modulation including an array of light modulators and apertures, each light modulator configured to be switched between an open position that permits transmittance of light from the display lighting system through a respective one of the apertures to the front surface and a shut position that blocks light transmission through the respective aperture, the transparent substrate including a plurality of light turning elements selectively located in areas not directly above the apertures;

a transparent substrate is disposed between the display lighting system and the front surface and substantially parallel to the front surface, and having a periphery at least coextensive with the viewing area;

at least one light sensor is disposed outside the periphery of the transparent substrate;

the light modulators are switched in accordance with a first modulation scheme to render an image, and in accordance with a second modulation scheme to selectively pass object illuminating light from the display lighting system through at least one of the respective apertures, the object illuminating light being at least partially unrelated to the image, the first modulation scheme adapted to utilize a persistence of vision phenomenon to render color images by sequentially alternating the color of visible light emitted by the display lighting system; and

the transparent substrate is configured to pass light emitted by the display lighting system toward the front surface and to receive light reflected through the front surface from an object, the transparent substrate including a first light-turning arrangement that redirects a portion of the received light toward the at least one light sensor;

outputting, from the light sensor to the processor, a signal representative of a characteristic of the received, redirected light; and

recognizing, from the output of the light sensor, a characteristic of the object.

20. The apparatus of claim 19, wherein the characteristic is one or more of a location, or a motion of the object.