Integrated near field sensor for display devices

• US 9,285,895 B1
• Filed: 03/28/2012
• Issued: 03/15/2016
• Est. Priority Date: 03/28/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Baldwin, Leo B.
• Primary Examiner(s): Yodichkas, Aneeta
• Application Number:
  US13/432,675
• Time in Patent Office: 1,448 Days
• Field of Search: 345/174, 345/175
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A transmissive element including a diffractive and/or refractive pattern can be used with a display screen of a computing device to allow the display screen to function as a near field optical sensor (NFOS). The NFOS can be used to detect shadows cast on the display screen, and/or bright objects detected near the display screen, which can be used to determine the relative position of one or more features with respect to the device. These features can be, for example, the fingers or thumbs of a user attempting to provide input to the computing device. The device can support motion or gesture detection using cameras of the device, and touch input using touch screen functionality, but the NFOS can enable the device to also track motion in the dead zone between the field of view of the cameras and the touch screen.

Claims

1. A method, comprising:

receiving light over a plurality of angles on a receiving surface of an optically transmissive sheet;

directing, by a diffraction pattern of the optically transmissive sheet, at least a portion of the light received on the receiving surface to a plurality of sensor elements configured to detect light;

analyzing variations in intensity of light detected by the plurality of sensor elements;

determining that at least one intensity value corresponding to one or more regions of the optically transmissive sheet is different than an intensity value corresponding to at least one other region of the optically transmissive sheet, the at least one intensity value indicative of one or more objects located within a detectable distance of the optically transmissive sheet; and

determining at least one position relative to the optically transmissive sheet for the one or more objects based at least in part on one or more characteristics of the one or more regions corresponding to the at least one intensity value.

2. The method of claim 1, wherein the one or more characteristics of the one or more regions includes at least one of a size, a shape, a location, a blur level, or an intensity of a shadow or an illuminated region detected by the computing device.

3. The method of claim 1, further comprising:

tracking the position of the at least one object over time; and

providing information for the position to an application executing on a computing device.

4. The method of claim 1, wherein the optically transmissive sheet is part of a display screen of a computing device including an active display layer and at least one light source for the active display layer, the plurality of sensor elements configured to detect light directed from the diffraction pattern during periods when the at least one light source is not active.

5. The method of claim 1, wherein each of the plurality of sensor elements corresponds to an associated region of the optically transmissive layer, and

wherein determining that the at least one intensity value corresponding to the one or more regions of the optically transmissive sheet is different than the intensity value corresponding to the least one other region, further includes analyzing intensity measurements for each of the plurality of sensor elements.

6. The method of claim 1, further comprising:

determining a direction of at least one light source for use in determining the position of the at least one object based at least in part upon one or more characteristics of the one or more regions.

7. The method of claim 1, further comprising:

capturing image information over a field of view at a distance from the optically transmissive sheet,

wherein determining the position of the at least one object includes analyzing the captured image information to determine a position of the at least one object when the at least one object is at least partially within the field of view.

8. The method of claim 7, wherein the position of the at least one object as determined from the captured image information is used as a reference point for tracking the position of the at least one object using the one or more shadows cast on the optically transmissive sheet.

9. The method of claim 7, wherein the captured image information is capable of being analyzed to determine at least one dimension the at least one object, the at least one dimension useful in determining the position based at least in part upon the one or more characteristics of the one or more shadows cast on the optically transmissive sheet.

10. The method of claim 1, wherein the at least one object includes at least one of a finger, thumb, hand, arm, pen, or stylus.

11. The method of claim 1, wherein the optically transmissive sheet is a plastic sheet, and wherein the diffractive pattern is embossed in the plastic sheet.

12. The method of claim 1, wherein the optically transmissive sheet is further operable to function as a brightness enhancing film.

13. The method of claim 1, further comprising:

determining a direction of at least one light source; and

using information about the direction of the at least one light source when determining the position of the at least one object based at least in part upon one or more characteristics of the one or more shadows cast on the optically transmissive sheet.

14. A computing device, comprising:

a device processor; and

a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the computing device to;

analyze intensity values for light detected by a plurality of sensor elements, the plurality of sensor elements configured to detect at least a portion of light that is received over a plurality of angles on a receiving surface of an optically transmissive sheet and directed by a diffraction pattern of the optically transmissive sheet;

determine that at least one intensity value corresponding to one or more regions of the optically transmissive sheet is different than an intensity value corresponding to at least one other region of the optically transmissive sheet, the at least one intensity value indicative of at least one object located near the computing device; and

determine a position relative to the optically transmissive sheet for the at least one object based at least in part upon a characteristic of the at least one intensity value.

15. The computing device of claim 14, wherein the instructions when executed further cause the computing device to:

track the position of the object over time; and

provide information for the position to an application executing on the computing device.

16. The computing device of claim 14, wherein the optically transmissive sheet is part of a display screen of the computing device including an active display layer and at least one light source for the active display layer, the plurality of sensor elements configured to detect light directed from the diffraction pattern during periods when the at least one light source is not active.

17. The computing device of claim 14, wherein the instructions when executed further cause the computing device to:

capture image information over a field of view of a camera of the computing device at a distance from the optically transmissive sheet,

wherein determining the position of the object includes analyzing the captured image information to determine the position of the object when the object is at least partially within the field of view.

18. The computing device of claim 17, wherein the position of the object as determined from the captured image information is used as a reference point for tracking the position of the object using the one or more shadows cast on the optically transmissive sheet, and

wherein the captured image information is capable of being analyzed to determine at least one dimension of the object, the at least one dimension useful in determining the position based at least in part upon the one or more characteristics of the one or more shadows cast on the optically transmissive sheet.

19. A non-transitory computer readable storage medium storing one or more sequences of instructions executable by one or more processors to perform a set of operations comprising:

analyzing variations in intensity values for light detected by a plurality of sensor elements, the plurality of sensor elements configured to detect at least a portion of light that is received over a plurality of angles on a receiving surface of an optically transmissive sheet and directed by a diffraction pattern of the optically transmissive sheet;

determining that at least one intensity value corresponding to one or more regions of the optically transmissive sheet is different than an intensity value corresponding to at least one other region of the optically transmissive sheet, the at least one different intensity value indicative of one or more objects located within a detectable distance of the optically transmissive sheet; and

determining at least one position relative to the optically transmissive sheet for the one or more objects based at least in part on one or more characteristics of the one or more regions of the optically transmissive sheet corresponding to the at least one intensity value.

20. The non-transitory computer readable storage medium of claim 19, further comprising instructions executed by the one or more processors to perform the operations of:

tracking the position of the at least one object over time; and

providing information for the position to an application executing on a computing device.

21. The non-transitory computer readable storage medium of claim 19, further comprising instructions executed by the one or more processors to perform the operations of:

determining a direction of at least one light source for use in determining the position of the at least one object based at least in part upon one or more characteristics of the one or more regions.

22. The non-transitory computer readable storage medium of claim 19, further comprising instructions executed by the one or more processors to perform the operations of:

capturing image information over a field of view at a distance from the optically transmissive sheet,

wherein determining the position of the at least one object includes analyzing the captured image information to determine a position of the at least one object when the at least one object is at least partially within the field of view, and wherein the position of the at least one object as determined from the captured image information is used as a reference point for tracking the position of the at least one object using the one or more shadows cast on the optically transmissive sheet.

23. The non-transitory computer readable storage medium of claim 19, further comprising instructions executed by the one or more processors to perform the operations of:

determining a direction of at least one light source; and

using information about the direction of the at least one light source when determining the position of the at least one object based at least in part upon one or more characteristics of the one or more shadows cast on the optically transmissive sheet.