Input method for surface of interactive display

• US 7,911,444 B2
• Filed: 08/31/2005
• Issued: 03/22/2011
• Est. Priority Date: 08/31/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yee, Dawson
• Primary Examiner(s): Xiao; Ke
• Application Number:
  US11/218,171
• Publication Number:
  US 20070046625A1
• Time in Patent Office: 2,029 Days
• Field of Search: 345/1.1, 345/1.3, 345156-184
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

An interactive display system configured for detecting an object or user input provided with an object. The system includes a display surface on which graphic images are displayed, one or more scanning light sources configured for scanning the interactive display surface, and a light detector configured for detecting light reflected from an object that is adjacent to or in contact with the interactive display surface. A computing system storing machine instructions is in communication with the scanning light source and the light detector. When executed, the machine instructions cause the computing system to illuminate the interactive display surface with the scanning light source, to detect light with the light detector that is reflected from an object after illumination with the light source, and to generate an output signal based on the detected light that has been reflected from an object on or adjacent to the interactive display surface.

Claims

1. A method for detecting an object or user input provided with an object, where the object is on or adjacent to an interactive display surface, using a scanning light source, comprising the steps of:

(a) illuminating an interactive display surface with a plurality of scanning light sources, wherein the interactive display surface has an external surface and an internal surface, wherein each of the scanning light sources is configured to emit light of a same predetermined wavelength and to simultaneously scan separate regions of the interactive display surface with the light of the predetermined wavelength, so that the entire interactive display surface is fully scanned over time, and wherein illuminating the interactive display surface includes the plurality of scanning light sources emitting the light through a photosensitive light surface toward the interactive display surface so as to illuminate an object on or adjacent to the external surface of the interactive display surface; and

(b) detecting light of the predetermined wavelength that is reflected from the object that is on or adjacent to the external surface of the interactive display surface, the light that is reflected being detected as at least one of the plurality of scanning light sources is illuminating the object with the light of the predetermined wavelength, and wherein the photosensitive light surface acts as a detector for the predetermined wavelength reflected from the object.

2. The method of claim 1, wherein the plurality of scanning light sources are configured for emitting light radiation of the predetermined wavelength in the infra-red spectrum.

3. The method of claim 1, wherein the step of detecting comprises the step of scanning the interactive display surface substantially in synchronization with the plurality of scanning light sources to detect light that is reflected from the object that is on or adjacent to the interactive display surface.

4. The method of claim 3, wherein the step of detecting comprises the step of coupling the scanning of the interactive light surface with the plurality of scanning light sources so that an area illuminated with the light of the predetermined wavelength is coincident with an area of detection in which light reflected from an object is detected.

5. The method of claim 1, wherein the photosensitive light surface is adjacent the internal surface of the interactive display surface and is the only light detector used in the step of detecting light that is reflected from the object that is adjacent to or in contact with the external surface of the interactive display surface.

6. The method of claim 1, further comprising the step of combining an output from the step of detecting light reflected in regions corresponding to each of the plurality of scanning light sources, so that a composite output is provided for the entire interactive display surface.

7. The method of claim 1, wherein the step of detecting comprises the steps of detecting at least one of:

(a) a specific object that is either adjacent to or in contact with the interactive display surface based on a characteristic of the object determined from the light reflected by the object;

(b) a size of the object;

(c) a shape of the object;

(d) a trajectory of the object over time;

(e) an orientation of the object; and

(f) a characteristic associated with the object based upon parameters of the light reflected from the object.

8. An interactive display system for detecting an object on or adjacent to a user interface surface, comprising:

(a) an interactive display surface on which graphic images are displayed, the interactive display surface having an internal surface and an external surface;

(b) a plurality of scanning light sources, the scanning light sources each being configured for scanning a region of the interactive display surface with light of the same wavelength and so that the full interactive display surface is fully scanned over time by the plurality of scanning light sources;

(c) a light detector configured for detecting light reflected from an object that is adjacent to or in contact with the interactive display surface, wherein the light detector is a photosensitive light surface adjacent the internal surface of the interactive display surface, and wherein the photosensitive light surface is configured to allow light from the plurality of scanning light sources to pass therethrough towards the interactive display surface and detect reflected light returning from the interactive display surface;

(d) a computing system in communication with the plurality of scanning light sources and the light detector, the computing system including a processor, and a memory in communication with the processor, the memory storing machine instructions that cause the processor to carry out a plurality of interactive display functions, including;

(i) illuminating the interactive display surface with the plurality of scanning light sources;

(ii) detecting light associated with the scanning light sources that is reflected from an object that is on or adjacent to the interactive display surface with the light detector while each of the scanning light sources is illuminating its region of the interactive display surface, wherein detecting light with the light detector includes detecting light using the photosensitive light surface after such light has passed through the photosensitive light surface and when the light is returned after being reflected from the object; and

(iii) generating an output signal based on the detected light that is reflected from an object that is on or adjacent to the interactive display surface.

9. The interactive display of claim 8, and wherein the light detector is one of:

(a) an area light detector;

(b) a linear light detector; and

(d) a point light detector.

10. The interactive display of claim 9, and wherein the light detector is integrated with at least one of the scanning light sources, so that the light detector and the at least one scanning light source scan together.

11. The interactive display of claim 10, wherein the light detector is a scanning light detector that is scanned coincident with an illumination light beam emitted by at least one of the scanning light sources.

12. The interactive display of claim 9, and wherein the area light detector is included within the photosensive light surface adjacent to the internal surface of the interactive display surface.

13. The interactive display of claim 12, and wherein the photosensitive layer is a photo transistor array.

14. The interactive display of claim 8, wherein each of the plurality of scanning light sources is configured for emitting light radiation of a wavelength in the infra-red spectrum.

15. The interactive display of claim 8, and wherein the interactive display surface further comprises a light diffusing layer.

16. The interactive display of claim 8, wherein the graphic image is formed on the interactive display surface with one of:

(a) a projector;

(b) a liquid crystal display panel; and

(c) an organic light emitting display panel.

17. The interactive display of claim 8, wherein each of the scanning light sources is a laser having a collimated light beam.

18. A method for receiving user input to an interactive display that is configured to detect objects on or adjacent to an interactive display surface, comprising the step of:

(a) associating each of a plurality of scanning light sources with one or more of a plurality of different surface area portions of the interactive display surface wherein a combination of the different surface area portions substantially describes a portion of the interactive display surface usable for receiving user input;

(b) illuminating each of the different surface area portions of the interactive display surface simultaneously with at least one of the plurality of scanning light sources within a predetermined time interval and with a same wavelength of light;

(c) detecting light that is reflected from one or more objects that are on or adjacent to at least one of the different surface area portions, wherein at least one of the plurality of scanning light sources provides a source for the light that is reflected when the scanning light source is providing illumination, and wherein detecting light includes triggering detection of the reflected light using a photosensitive light surface receiving light emitted from the plurality of scanning light sources toward the interactive display surface;

(d) generating a plurality of object detection signals based on the step of detecting light from one or more objects that are on or adjacent to each of the plurality of different surface area portions, wherein the plurality of light detection signals describes light reflected from the one or more objects; and

(e) processing the plurality of object detection signals to receive the user input, wherein the user input is associated with light reflected from one or more objects that are on or adjacent to the interactive display surface.

19. The method of claim 18, and further comprising:

(a) displaying an image on the interactive display surface while performing at least one of steps (a)-(e) of claim 18; and

(b) applying the user input to interact with the displayed image.

20. The method of claim 18, wherein:

illuminating each of the different surface area portions of the interactive display surface simultaneously comprises using a plurality of spatially small-sized cross-sectional beams concentrated in a small area rather than over an wide area;

the photosensitive light surface is adjacent the interactive display surface, and disposed between the interactive display surface and the plurality of scanning light sources;

the photosensitive light surface includes a light detector therein for detecting the reflected light; and

the scanning light sources are raster scanning emitter light sources with a collimated beam diameter, and approximately 152×

114 pixels in size, with a rate of about 60 times per second.