Projector as an input and output device

• US 6,764,185 B1
• Filed: 08/07/2003
• Issued: 07/20/2004
• Est. Priority Date: 08/07/2003
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Brinkman, Dirk, Forlines, Clifton L., Raskar, Ramesh, Beardsley, Paul A.
• Primary Examiner(s): Adams, Russell
• Assistant Examiner(s): Koval, Melissa J
• Application Number:
  US10/636,356
• Time in Patent Office: 348 Days
• Field of Search: 353/94, 353/69, 353/70, 353/121, 353/122, 353/42, 353/30, 702/152, 382/154, 348/383, 348/744, 345/32, 345/582
• US Class Current: 353/122
• CPC Class Codes: G03B 21/00 Projectors or projection-type viewers; Accessories therefor devices for changing pictures G03B23/00

Abstract

An interactive display system includes a sensor for sensing a relationship between a mobile coordinate frame fixed to a moving projector, and a stable coordinate frame fixed to a stationary display surface in the real world. An output image to be projected on the display surface is partitioned into a black image portion having a fixed relationship to the mobile coordinate frame, a stable image portion within the block image portion having a fixed relationship to the stable coordinate frame, and a mobile image portion within the stable image portion having a fixed relationship to the mobile coordinate frame. The mobile portion can be used as a pointer within the stable portion in a mouse-cursor like manner.

Claims

1. An interactive display system, comprising:

a sensor configured to sense a relationship between a mobile coordinate frame and a stable coordinate frame; and

means for partitioning a projected output image into a black image portion having a fixed relationship to the mobile coordinate frame, a stable image portion within the black image portion having a fixed relationship to the stable coordinate frame, and a mobile image portion within the stable image portion having a fixed relationship to the mobile coordinate frame.

2. The system of claim 1, further comprising:

a projector fixed relative to the mobile coordinate system; and

a display surface in the real world fixed relative to the stable coordinate frame.

3. The system of claim 1, in which the stable image portion moves as the mobile image portion, when the stable image portion is immediately adjacent to the mobile image portion.

4. The system of claim 1, in which the stable image portion moves as the black image portion, when the stable image portion is immediately adjacent to an outer boundary of the black image portion.

5. The system of claim 2, further comprising;

means for calibrating the projector.

6. The system of claim 2, in which the projector is moving while projecting the output image on the display surface.

7. The system of claim 6, in which the mobile image portion is a pointer.

8. The system of claim 7, in which the stable image portion includes a plurality of components, and further comprising;

means for initiating a computer operation according to a current location of the mobile image portion relative to one of the components of the stable image portion.

9. The system of claim 7, further comprising:

means for projecting a track of the pointer as the pointer is moving in the stable image portion.

10. The system of claim 9, in which the track is conformed to a predetermined shape.

11. The system of claim 9, in which the stable image portion includes a plurality of components, and further comprising;

means for initiating a computer operation according to a current location and shape of the track relative to one of the components of the stable image portion.

12. The system of claim 1, in which the sensor is a camera.

13. The system of claim 12, in which the mobile image portion is a pointer, and further comprising:

means for projecting a track of the pointer as the pointer is moving in the stable image portion, the track defining a region on a display surface; and

means for extracting the region from an input image acquired by the camera.

14. The system of claim 13, further comprising:

means for projecting the extracted region onto an arbitrary display surface.

15. The system of claim 11, further comprising:

means for initiating a computer processing on the extracted region.

16. The system of claim 15, in which the extracted region is a physical marker of distinct appearance on the display surface, and the computer processing identifies the physical marker, and the identity of the marker is used to initiate a computer operation.

17. The system of claim 1, in which the sensor is moved by a hand, and further comprising:

a projector and a display surface fixed relative to the stable coordinate frame.

18. The system of claim 1, in which the sensor is a camera acquiring images of a moving object to determine the mobile coordinate frame, and further comprising:

a projector and a display surface fixed relative to the stable coordinate frame.

19. The system of claim 7, in which the pointer is used to define a shape of the stable image portion.

20. The system of claim 11, in which the computer operation is applied to the stable image according to the current location and shape of the track, and the stable image includes an image acquired of a real-world scene.

21. An interactive display method, comprising:

sensing a relationship between a mobile coordinate frame and a stable coordinate frame; and

partitioning a projected output image into a black image portion having a fixed relationship to the mobile coordinate frame, a stable image portion within the block image portion having a fixed relationship to the stable coordinate frame, and a mobile image portion within the stable image portion having a fixed relationship to the mobile coordinate frame.