Projector and camera arrangement with shared optics and optical marker for use with whiteboard systems

• US 6,707,444 B1
• Filed: 08/18/2000
• Issued: 03/16/2004
• Est. Priority Date: 08/18/2000
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Wang, Xiping, Doany, Fuad Elias, Shae, Zon-Yin, Tseng, Belle L., Hendriks, Ferdinand
• Primary Examiner(s): Hjerpe, Richard
• Assistant Examiner(s): Nguyen, Jennifer T.
• Application Number:
  US09/644,980
• Time in Patent Office: 1,306 Days
• Field of Search: 345/87, 345/104, 345/207, 345/179, 345/159, 345/158, 345/169, 345/9, 345/156
• US Class Current: 345/156
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

Projector and camera arrangements are provided for use in electronic whiteboard systems. Specifically, the present invention provides projector and camera arrangements wherein the projector and camera share the same imaging optics. By sharing the same projection and camera optics, the distortions that affect the projection system are the same as those of the camera system. Thus, the calibration step required in conventional whiteboard systems where the projector and camera are separate, i.e., each having their own distinct optics and settings, is no longer needed. Further, the arrangements provided in accordance with the invention are self-aligning, even when lens distortions are large and even in the presence of strong perspective effects. The shared optics projector and camera arrangements of the invention also provide for dynamic zooming. In addition, various active and passive optical marker or lightpen designs are provided for use in electronic whiteboard systems.

Claims

1. An image capture and projection apparatus for use in an electronic whiteboard system, the apparatus comprising:

an image capture device, the image capture device tracking a position of at least one stylus associated with the electronic whiteboard system used to enter data in accordance with a surface associated with the electronic whiteboard system; and

an image projection device, the image projection device projecting an image which is viewable on the surface, in proximity of the position of the stylus, and representative of the data entered in accordance with the stylus, wherein the image capture device and the image projection device share at least one imaging lens for capturing and projecting one or more images, and the image capture device and the image projection device are physically integrated with one another to form an integrated image capture/projection device, and wherein an image capture portion of the integrated image capture/projection device comprises an infrared charge coupled device;

further wherein the image capture portion further comprises an infrared-reflecting, visible;

transmitting cube for;

(i) reflecting infrared light received through the imaging lens from the proximity of the stylus; and

(ii) passing visible light produced by an image projection portion of the integrated device through to the imaging lens for presentation on the surface of the electronic whiteboard surface.

2. The apparatus of claim 1, wherein the integrated image capture/projection device is a front-projection type device.

3. The apparatus of claim 1, wherein the integrated image capture/projection device is a rear-projection type device.

4. The apparatus of claim 1, wherein an image projection portion of the integrated image capture/projection device is a transmissive liquid crystal-based lightvalve type device.

5. The apparatus of claim 1, wherein an image projection portion of the integrated image capture/projection device is a reflective liquid crystal-based lightvalve type device.

6. The apparatus of claim 1, further wherein the image projection portion of the integrated image capture/projection device is a digital micromirror deflection type device.

7. The apparatus of claim 1, wherein the image capture portion further comprises an infrared-transmitting, visible-blocking filter for:

(i) passing infrared light reflected by the infrared-reflecting, visible-transmitting cube to the infrared charge coupled device; and

(ii) blocking visible light.

8. The apparatus of claim 1, wherein the image capture portion further comprises an infrared refocusing lens positioned between the infrared-reflecting, visible-transmitting cube and the infrared-transmitting, visible-blocking filter.

9. The apparatus of claim 7, wherein the infrared charge coupled device records the infrared light received from the proximity of the stylus via the imaging lens, the infrared-reflecting, visible-transmitting cube, the infrared refocusing lens and the infrared-transmitting, visible-blocking filter.

10. The apparatus of claim 1, wherein the integrated image capture/projection device is self-aligning.

11. The apparatus of claim 1, further wherein the integrated image capture/projection device is dynamically zoomable.

12. An image capture and projection apparatus for use in an electronic whiteboard system, the apparatus comprising:

an image capture device, the image capture device tracking a position of at least one stylus associated with the electronic whiteboard system used to enter data in accordance with a surface associated with the electronic whiteboard system; and

an image projection device, the image projection device projecting an image which is viewable on the surface, in proximity of the position of the stylus, and representative of the data entered in accordance with the stylus, wherein the image capture device and the image projection device share at least one imaging lens for capturing and projecting one or more images, and the image capture device and the image projection device are physically integrated with one another to form an integrated image capture/projection device, and wherein an image capture portion of the integrated image capture/projection device comprises an infrared charge coupled device;

further wherein an image projection portion and the image capture portion of the integrated device share a dichroic X-cube having an infrared-reflecting interface for;

(i) combining components of visible light produced by the image projection portion and passing them through to the imaging lens for presentation on the surface of the electronic whiteboard surface; and

(ii) reflecting infrared light received through the imaging lens from the proximity of the stylus.

13. The apparatus of claim 12, wherein an image projection portion and the image capture portion of the integrated device share a polarizing beam splitter having an infrared-reflecting coating for:

(i) polarizing a component of visible light produced by the image projection portion; and

(ii) reflecting infrared light received from the infrared reflecting interface of the dichroic X-cube.

14. The apparatus of claim 13, wherein the image capture portion further comprises an infrared-transmitting, visible-blocking filter for:

(i) passing infrared light reflected by the infrared-reflecting coating of the polarizing beam splitter to the infrared charge coupled device; and

(ii) blocking visible light.

15. The apparatus of claim 14, wherein the infrared charge coupled device records the infrared light received from the proximity of the stylus via the imaging lens, the infrare-dreflecting interface of the dichroic X-cube, the infrared-reflecting coating of the polarizing beam splitter and the infrared-transmitting, visible-blocking filter.

16. An image capture and projection apparatus for use in an electronic whiteboard system, the apparatus comprising:

an image capture device, the image capture device tracking a position of at least one stylus associated with the electronic whiteboard system used to enter data in accordance with a surface associated with the electronic whiteboard system; and

an image projection device, the image projection device projecting an image which is viewable on the surface, in proximity of the position of the stylus, and representative of the data entered in accordance with the stylus, wherein the image capture device and the image projection device share at least one imaging lens for capturing and projecting one or more images, and the image capture device and the image projection device are physically integrated with one another to form an integrated image capture/projection device, and wherein an image capture portion of the integrated image capture/projection device comprises an infrared charge coupled device;

further wherein an image projection portion and the image capture portion of the integrated device share a total internal reflection prism having an infrared-reflecting coating for;

(i) passing visible light produced by the image projection portion in accordance with a deflection micromirror device through to the imaging lens for presentation on the surface of the electronic whiteboard surface; and

(ii) reflecting infrared light received through the imaging lens from the proximity of the stylus.

17. The apparatus of claim 16, wherein the image capture portion further comprises an infrared-transmitting, visible-blocking filter for:

(i) passing infrared light reflected by the infrared-reflecting coating of the total internal reflection prism to the infrared charge coupled device; and

(ii) blocking visible light.

18. The apparatus of claim 17, wherein the infrared charge coupled device records the infrared light received from the proximity of the stylus via the imaging lens, the infrare-dreflecting coating of the total internal reflection prism and the infrared-transmitting, visible-blocking filter.

19. An electronic whiteboard system, the system comprising:

a data entry surface;

at least one stylus for entering data in accordance with the data entry surface;

an image capture device, the image capture device tracking a position of the stylus while the stylus is used to enter data in accordance with the data entry surface; and

an image projection device, the image projection device projecting an image which is viewable on the surface, in proximity of the position of the stylus, and representative of the data entered in accordance with the stylus, wherein the image capture device and the image projection device share at least one imaging lens for capturing and projecting one or more images, and the image capture device and the image projection device are physically integrated with one another to form an integrated image capture/projection device, and wherein an image capture portion of the integrated image capture/projection device comprises an infrared charge coupled device;

further wherein the image capture portion further comprises an infrared-reflecting, visible-transmitting cube for;

(i) reflecting infrared light received through the imaging lens from the proximity of the stylus; and

(ii) passing visible light produced by an image projection portion of the integrated device through to the imaging lens for presentation on the surface of the electronic whiteboard surface.