Specular reflection reduction using multiple cameras

• US 7,630,002 B2
• Filed: 01/05/2007
• Issued: 12/08/2009
• Est. Priority Date: 01/05/2007
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Jenkins, David R.
• Primary Examiner(s): Tran; Sinh
• Assistant Examiner(s): Bemben; Richard M
• Application Number:
  US11/620,591
• Publication Number:
  US 20080165266A1
• Time in Patent Office: 1,068 Days
• Field of Search: 345/173, 345/175, 348/218.1, 348/207.99, 348/739
• US Class Current: 348/218.1
• CPC Class Codes: G06F 3/04182 Filtering of noise external to the device and not generated by digitiser components
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

An interactive display system that includes an interactive display screen. An illuminator is positioned to illuminate one of the inner or outer surfaces of the display screen. At least two cameras are placed so as to view the illuminated surface of the display screen. Each of at least one of the cameras are positioned such that specular reflections from the illuminator are received by the camera. The images from the different cameras are merged to form a merged image in which specular reflections are reduced or even cancelled.

Claims

1. An interactive display system comprising:

a display screen comprising one or more display layers;

an illuminator positioned at a single location so as to illuminate one of the inner surface or outer surface of the display screen from the single location, the illuminator causing at least a first specular reflection that is detected by a first camera and a second specular reflection that is detected by a second camera;

the first camera being positioned so as to view and capture a first image of at least a first area and a second area of the illuminated surface of the display screen, using light emitted from the illuminator, wherein the first image captured by the first camera includes;

a first image portion that corresponds to the first area of the illuminated surface and that includes the first specular reflection that is caused by light from the illuminator, and

a second image portion that corresponds to the second area of the illuminated surface and that is free from the first and second specular reflection caused by light from the illuminator;

the second camera being positioned so as to view and capture a second image of at least the first area and the second area of the illuminated surface of the display screen, using light emitted from the illuminator, wherein the second image captured by the second camera includes;

a third image portion that corresponds to the second area of the illuminated surface and that includes the second specular reflection that is caused by light from the illuminator, and

a fourth image portion that corresponds to the first area of the illuminated surface and that is free from the first and second specular reflection caused by light from the illuminator; and

an image merging mechanism configured to combine the first and second images from the first camera and from the second camera, respectively, in such a way that at least the first specular reflection is at least reduced in a merged image in comparison with the first specular reflection captured in the first image by the first camera.

2. The interactive display system in accordance with claim 1, further comprising:

a third camera positioned so as to view the illuminated surface of the display screen.

3. The interactive display system in accordance with claim 2, wherein the illuminator is positioned such that the third camera is enabled to capture a third image which is free from any specular reflection caused by the illuminator when the illuminator is illuminating.

4. The interactive display system in accordance with claim 3, wherein the illuminator is positioned off-center with respect to the one or more display layers.

5. The interactive display system in accordance with claim 2, wherein the third camera is positioned such that when the illuminator is illuminated, a third specular reflection occurs at a third area, which is detected by and captured by the third camera in a third image, the third area being different than the first and second areas.

6. The interactive display system in accordance with claim 5, further comprising:

a fourth and fifth camera positioned so as to view the illuminated surface of the display screen.

7. The interactive display system in accordance with claim 1, wherein the illuminator emits light primarily in the infrared range.

8. The interactive display system in accordance with claim 1, wherein the image merging mechanism is configured to combine the first and second images from the first and second cameras by:

accessing the second image portion taken by the first camera, the second image portion lacking any specular reflection even though the first camera receives the first specular reflection;

accessing the fourth image portion taken by the second camera, the fourth image portion lacking any specular reflection even though the second camera receives the second specular reflection; and

merging the second image portion from the first image and the fourth image portion from the second image.

9. The interactive display system in accordance with claim 8, wherein the second image portion and the fourth image portion, when combined, represent substantially all of the displayable area of the display screen.

10. The interactive display system in accordance with claim 8, wherein the first image portion and the second image portion are different is size and wherein a majority of the first image portion corresponds to an area in the second image in which there is a specular reflection and which corresponds to the fourth image portion.

11. The interactive display system in accordance with claim 10, wherein the first portion of the first image is rectangular.

12. The interactive display system of claim 8, wherein image merging mechanism is further configured to combine the first and second images from the first and second cameras by at least appending the merged image with a fifth image portion corresponding to an area of the display screen that is within photographic range of the second camera but which is outside of photographic range of the first camera.

13. The interactive display system in accordance with claim 1, wherein the illuminator emits light primarily in the ultraviolet spectrum.

14. The interactive display system in accordance with claim 1, wherein the illuminator is configured to illuminate during substantially all of the time that the interactive display is in an interactive mode of operation.

15. An image merging mechanism for use with an interactive display system that includes one or more display layers, a first camera positioned so as to view an illuminated surface of the one or more display layers at a first position such that when an illuminator that is positioned at a particular location is illuminated a specular reflection occurs at a specular reflection area of an image as captured by the first camera, a second camera positioned so as to view an illuminated surface of the one or more display layers at a second position such that when the illuminator is illuminated from the particular location the second camera is able to view and capture at least a portion of the specular reflection area of the first camera without capturing the specular reflection, the image merging mechanism configured to perform the following act:

an act of combining images from the first and second camera such that the specular reflection is at least reduced as compared to a level of the specular reflection captured by the first camera.

16. The image merging mechanism in accordance with claim 15, wherein the act of combining images comprises:

accessing a first portion of a first image taken by the first camera, the first portion of the first image lacking a specular reflection even though the first camera receives a specular reflection;

accessing a second portion of a second image taken by the second camera, the second portion of the second image lacking a specular reflection; and

merging the first portion of the first image and the second portion of the second image.

17. The image merging mechanism in accordance with claim 16, wherein the first portion of the first image and the second portion of the second image, when combined, represent substantially all of the displayable area of the one or more display layers.

18. The image merging mechanism in accordance with claim 16, wherein a majority of the first portion of the first image corresponds to an area in the second image in which there is a specular reflection.

19. An interactive display system comprising:

a display screen comprising one or more display layers;

an illuminator positioned at a particular location to illuminate one of the inner or outer surfaces of the display screen layers;

a first camera positioned so as to view the illuminated surface of the one or more display screen layers at a position such that when the illuminator is illuminated, a specular reflection occurs at an area of an image as captured by the first camera;

a second camera positioned so as to view the illuminated surface of the one or more display screen layers at a position such that when the illuminator is illuminated, a specular reflection occurs at a second portion of images captured by the second camera, the second camera capturing images reflected from the illuminator at a second region of the one or more display screen layers;

a third camera positioned so as to view the illuminated surface of the one or more display screen layers at a position such that the third camera captures images reflected from the illuminator at a third region of the one or more display screen layers;

a fourth camera positioned so as to view the illuminated surface of the one or more display screen layers at a position such that the fourth camera captures images reflected from the illuminator at a fourth region of the one or more display screen layers; and

a fifth camera, comprising an overlapping camera positioned so as to view the illuminated surface of the one or more display screen layers at a position such the overlapping camera captures images from areas of specular reflection for the first and second cameras, without itself capturing specular reflections for those corresponding areas.

20. The interactive display in accordance with claim 19, wherein the illuminator emits light primarily in the ultraviolet spectrum.

21. The interactive display in accordance with claim 19, wherein the illuminator emits light primarily in the infrared spectrum.