Retroreflection based multitouch sensor

• US 20080060854A1
• Filed: 08/02/2007
• Published: 03/13/2008
• Est. Priority Date: 08/03/2006
• Status: Active Grant

• Current Assignee: New York University
• Original Assignee: New York University
• Inventors: Perlin, Kenneth
• Granted Patent: US 9,348,463 B2
• Time in Patent Office: Days
• US Class Current: 178/18.03
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

A detection apparatus includes a multilocation sensor. The apparatus includes a sheet in communication with the sensor, which when a plurality of locations of the sheet are simultaneously activated, the sensor senses these locations, simultaneously. Alternatively, the apparatus includes a surface. The apparatus includes a sensor in communication with the surface that detects variable pressure regarding a touch on the surface. Alternatively, the apparatus includes a sensor in communication with the surface that is capable of detecting a touch on a surface and also a presence of an object near the surface but not touching the surface. Alternatively, the apparatus includes a lenslet array where the lenslets are diffractive. The apparatus includes a sensor for determining where the array is being touched. The sensor includes a first light source producing light at a first frequency, a second light source producing light at a second frequency, a camera which records images from the frequencies of light from the array, and a computer which determines where the surface is being touched from the images. Alternatively, the apparatus includes a sensor for determining where the surface is being touched, including a plurality of cameras that take images of the surface, a first array of lights positioned such that light from the first array is on a vertical line with one or more of the cameras, a second array of lights that are positioned such that light from the second array is on a horizontal line with one or more of the cameras and a computer connected to the cameras which determines where the surface is being touched from the images. Alternatively, the apparatus includes a sheet containing a plurality of elements which are at least one of refractive, diffractive, retroreflective, reflective, or focusing. The apparatus includes a sensor which determines where the sheet is touched based on light being one of refracted, diffracted, retroreflected, reflected or focused by the sheet to the sensor. Alternatively, the apparatus includes a sheet having no active components in, on, or adjoining the sheet. The apparatus includes active components separate and apart from the sheet which determine where the sheet is touched. The apparatus can include an autostereo display including a multitouch retroreflective surface and a projector. A method for detection. A software program stored on a computer 5 readable medium.

Claims

1. A detection apparatus comprising:

a multilocation sensor; and

a sheet in communication with the sensor, which when a plurality of locations of the sheet are simultaneously activated, the sensor senses these locations, simultaneously.

2. An apparatus as described in claim 1 wherein the multilocation sensor, when the plurality of locations of the sheets surface are touched simultaneously, the sensor senses these locations, simultaneously.

3. An apparatus as described in claim 2 wherein the sensor includes an imager which produces images of the surface.

4. An apparatus as described in claim 3 wherein the imager includes a camera and the sensor includes a computer in communication with the camera and receives the images from the camera.

5. An apparatus as described in claim 4 wherein the sheet includes a first lenslet array.

6. An apparatus as described in claim 5 wherein the surface is at the focal length of each lens of the lenslet array.

7. An apparatus as described in claim 6 wherein the array has a smooth side and a ridged side, and the array is positioned so the ridged side is facing the camera.

8. An apparatus as described in claim 5 wherein the array has lateral spacing between the lenses to allow the sheet to be looked through.

9. An apparatus as described in claim 5 including a second lenslet array.

10. An apparatus as described in claim 9 wherein the second lenslet array is oriented perpendicular to the first lenslet array, and together the first and second lenslet arrays act as a rear projector screen.

11. An apparatus as described in claim 10 including a projector in communication with the computer that displays an image generated by the computer.

14. An apparatus as described in claim 3 wherein the camera includes a shutter to reduce optical noise.

15. An apparatus as described in claim 2 wherein the surface is a thin-film comprising micro lenses.

16. An apparatus as described in claim 15 wherein the micro lenses are sparsely placed so the surface appears transparent.

17. An apparatus as described in claim 16 wherein the sheet includes a substrate and both sides of the substrate are coated with thin films comprising micro lenses.

20. An apparatus as described in claim 1 wherein the sensor uses retroreflection to determine the locations.

21. An apparatus as described in claim 1 wherein the sensor uses touch, pressure, or proximity to determine the locations.

33. An apparatus as described in claim 3 including a first light source and a second light source, and the computer subtracts pixel values from a second image formed when the second light source is on from corresponding pixel values of a first image formed when the first light source is on.

12. A detection apparatus comprising:

a surface; and

a sensor in communication with the surface that detects variable pressure regarding a touch on the surface.

13. A detection apparatus comprising:

a surface; and

a sensor in communication with the surface that is capable of detecting a touch on a surface and also a presence of an object near the surface but not touching the surface.

18. A detection apparatus comprising:

a lenslet array where the lenslets are diffractive; and

a sensor for determining where the array is being touched, the sensor includes a first light source producing light at a first frequency, a second light source producing light at a second frequency, a camera which records images from the frequencies of light from the array, and a computer which determines where the surface is being touched from the images.

19. A detection apparatus comprising:

a surface; and

a sensor for determining where the surface is being touched, including a plurality of cameras that take images of the surface, a first array of lights positioned such that light from the first array is on a vertical line with one or more of the cameras, a second array of lights that are positioned such that light from the second array is on a horizontal line with one or more of the cameras and a computer connected to the cameras which determines where the surface is being touched from the images.

22. A detection apparatus comprising:

a sheet containing a plurality of elements which are at least one of refractive, diffractive, retroreflective, reflective, or focusing; and

a sensor which determines where the sheet is touched based on light being one of refracted, diffracted, retroreflected, reflected or focused by the sheet to the sensor.

23. A detection apparatus comprising:

a sheet having no active components in, on, or adjoining the sheet; and

active components separate and apart from the sheet which determine where the sheet is touched.

24. A method for detection comprising the steps of:

activating simultaneously a plurality of locations of a sheet; and

sensing these locations simultaneously with a multilocation sensor in communication with the sheet.

34. A method as described in claim 24 wherein the sensing step includes the step of subtracting pixel values of a second image from corresponding pixel values of a first image.

25. A method for detection comprising the steps of:

touching a surface; and

detecting variable pressure regarding the touch on the surface.

26. A method for detection comprising the steps of:

detecting a touch on a surface with a sensor in communication with the surface; and

detecting a presence of an object near the surface but not touching the surface with the sensor.

27. A method for detection comprising the steps of:

determining where a surface is being touched with a sensor by taking images of the surface with a plurality of cameras, where a first array of lights is positioned such that light from the first array is on a vertical line with one or more of the cameras, and a second array of lights is positioned such that light from the second array is on a horizontal line with one or more of the cameras; and

determining with a computer connected to the cameras where the surface is being touched from the images.

28. A method for detection comprising the steps of:

touching a sheet containing a plurality of elements which are at least one of refractive, diffractive, retroreflective, reflective or focusing; and

determining with a sensor where the sheet is touched based on light being one of refracted, diffractive, (retro)reflected or focused by the sheet to the sensor.

29. A method for detection comprising the steps of:

touching a sheet having no active components in, on, or adjoining the sheet; and

determining where the sheet is touched with active components separate and apart from the sheet.

30. A detection apparatus comprising:

an autostereo display including a multitouch retroreflective surface and a projector.

31. A detection apparatus comprising:

a multilocation sensor; and

a sheet the first side and a second side in communication with the sensor, which when a location of the sheet is activated on the first side and the second side, the sensor senses the locations.

32. An apparatus as described in claim 31 wherein simultaneous activation of locations on the first side and the second side can be detected independently by the sensor.

35. A software program stored on a computer readable medium comprising the steps of:

activating simultaneously a plurality of locations of a sheet; and

sensing these locations simultaneously with a multilocation sensor in communication with the sheet.

36. A program as described in claim 35 including the step of measuring an area on contact of these locations.