Interactive projected video image display system

• US 5,528,263 A
• Filed: 06/15/1994
• Issued: 06/18/1996
• Est. Priority Date: 06/15/1994
• Status: Expired due to Fees

• Current Assignee: Daniel Marcel Platzker
• Original Assignee: Daniel M. Platzker
• Inventors: Platzker, Daniel M., Ackner, Rueven, Agmon, Yoav
• Primary Examiner(s): BRIER, JEFFERY A
• Application Number:
  US08/259,887
• Time in Patent Office: 734 Days
• Field of Search: 345/145, 345/146, 345/156, 345/157, 345/173, 345/175, 345/179, 348/14, 348/18, 348/61, 348/552
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems

Abstract

An interactive projected image video display system which includes a computer for generating video images, a projection apparatus coupled to the video output port of the computer for projecting computer-generated video images onto a projection screen, and at least one video camera for visually recording the projected image. The computer includes a frame grabber board coupled to the video camera for capturing individual frames or series of frames from the video recording of projected image. The captured images are compared to the computer generated images to detect for the presence of recognizable features introduced onto the projected image by a user to instruct the computer to trigger a computer operation. The system can also simulates a mouse to be used with any off the shelf application that uses a regular mouse.

Claims

1. A method of providing interaction with a projected video image comprising the steps of:

using a computer to generate a video image, the video image containing at least one predetermined region having a controlled characteristic associated therewith;

projecting the video image on any surface capable of displaying a projected image;

introducing an object over the projected image for a predetermined minimum period of time;

visually recording the projected image;

capturing an image from the visual recording of the projected image which contains the object;

comparing the captured image to the computer generated video image to detect the presence of the object in relation to the at least one controlled characteristic; and

generating a control signal for actuating the computer in response to the presence of the object.

2. A method as in claim 1 wherein the at least one controlled characteristic includes a button and the image comparison step further comprises:

reading a block portion of the captured image corresponding to the location of the button; and

checking the block portion for the presence of the object.

3. A method as in claim 2 wherein the step of generating a control signal further comprises:

saving at least a portion of the captured image to a memory in the computer.

4. A method as in claim 1 wherein the step of generating a control signal further comprises:

triggering a computer operation in response to the location of the object being introduced over the projected image.

5. A method as in claim 4 wherein the step of modifying the video image further comprises:

generating a cursor corresponding to the location of the object being introduced over the projected image.

6. A method as in claim 5 in which an application program, is currently running on the computer, said application program is responsive to a driver for a computer input device and wherein said method further comprises the steps of:

selectively generating pop up menu windows adjacent a cursor on the projected image in response to user positioning of an object over the projected image, said menu windows contain user selectable buttons which represent driver options associated with a computer input device for triggering the operating system of the computer to send a desired computer input control message to said application program in order to perform a desired computer operation; and

selecting one of said buttons by placing an object thereover for a predetermined period of time.

7. A method as in claim 1 wherein the step of comparing the captured image to the computer generated image includes detecting a recognizable pattern created by the combination of the introduced object and the projected image.

8. A method as in claim 2 which includes the step of self testing to verify calibration of projected images with captured images, said step of self testing including the steps of:

storing a first image and a second image, said first and second stored images each having a common predetermined location associated therewith, said second stored image further including an object disposed over its respective predetermined location;

projecting said second stored image;

comparing the projected second stored image with said first stored image; and

triggering a predetermined computer operation if the respective predetermined locations of said first stored image and said second projected stored image are in coalignment during comparison with each other.

9. A method of providing interaction with a displayed video image comprising the steps of:

using a computer to generate a video image, the video image containing at least one predetermined region having a controlled characteristic associated therewith;

displaying the video image;

introducing an object over the displayed image for a predetermined minimum period of time;

visually recording the displayed image;

capturing an image from the visual recording of the displayed image which contains the object;

comparing the captured image to the computer generated video image to detect the presence of the object relative to the at least one controlled characteristic; and

generating a control signal for actuating the computer in response to the presence of the object.

10. A method as in claim 9 wherein the at least one controlled characteristic includes a button and the image comparison step further comprises:

reading a block portion of the captured image corresponding to the location of the button; and

checking the block portion for the presence of the object.

11. A method as in claim 10 wherein the step of generating a control signal further comprises:

saving at least a portion of the captured image to a memory in the computer.

12. A method as in claim 9 wherein the step of generating a control signal further comprises:

triggering a computer operation in response to the location of the object being introduced over the displayed image.

13. A method as in claim 12 wherein the step of modifying the video image further comprises:

generating a cursor corresponding to the location of the object being introduced over the displayed image.

14. A method as in claim 13 in which an application program, is currently running on the computer, said application program is responsive to a driver for a computer input device and wherein said method further comprises the steps of:

selectively generating pop up menu windows adjacent a cursor on the displayed image in response to user positioning of an object over the displayed image, said menu windows contain user selectable buttons which represent driver options associated with a computer input device for triggering the operating system of the computer to send a desired computer input control message to said application program in order to perform a desired computer operation; and

selecting one of said buttons by placing an object thereover for a predetermined period of time.

15. A method as in claim 9 wherein the step of comparing the captured image to the computer generated video image includes detecting a recognizable pattern created by the combination of the introduced object and the displayed image.

16. A method as in claim 10 which includes the step of self testing to verify calibration of displayed images with captured images, said step of self testing including the steps of:

storing a first image and a second image, said first and second stored images each having a common predetermined location associated therewith;

said second stored image further including an object disposed over its respective, predetermined location;

displaying said second stored image;

comparing the displayed second stored image with said first stored image; and

triggering a predetermined computer operation if the respective predetermined computer locations of said first and second stored images are in coalignment during comparison with each other.

17. An interactive projected video image display system comprising:

means for generating a video image, the video image containing at least one predetermined region having a controlled characteristic associated therewith;

means for projecting the video image onto any surface capable of displaying a projected image;

means for visually recording the projected image;

means for capturing an image from the visual recording of the projected image;

image discrimination means for comparing the captured image with the generated video image and for detecting the presence of an object introduced onto the projected image and its relation to the at least one controlled characteristic; and

means responsive to said image discrimination means for triggering a computer operation image when said introduced object is detected.

18. The system according to claim 17 wherein the projection means further comprises:

a liquid crystal display; and

an overhead projector.

19. The system according to claim 18 wherein:

the visual recording means comprises at least one video camera; and

the image capture means comprises video frame grabber.

20. The system according to claim 17 wherein:

the visual recording means comprises at least one video camera; and

the image capture means comprises video frame grabber.

21. A method of providing interaction with a projected video image comprising the steps of:

using a computer to generate a video image, the video image containing a button;

projecting the video image;

introducing an object over the projected image for a predetermined minimum period of time;

visually recording the projected image;

capturing an image from the visual recording of the projected image which contains the object;

comparing the captured image to the computer generated video image to detect the presence of the object, wherein the comparing of the capture image to the computer generated video image further includes;

reading a block portion of the captured image corresponding to the location of the button;

checking the block portion for the presence of the object;

generating a control signal for actuating the computer in response to the presence of the object; and

self testing to verify calibration of projected images with captured images, and wherein the self testing further includes;

storing a first image and a second image, said first and second stored images each having a common predetermined location associated therewith, said second stored image further including an object disposed over its respective predetermined location;

projecting said second stored image;

comparing the projected second stored image with said first stored image; and

triggering a predetermined computer operation in accordance with the relationship of the respective predetermined locations of said first stored image and said second projected stored image during comparison with each other.

22. A method of providing interaction with a displayed video image comprising the steps of:

using a computer to generate a video image, the video image containing a button;

displaying the video image;

introducing an object over the displayed image for a predetermined minimum period of time;

visually recording the displayed image;

capturing an image from the visual recording of the displayed image which contains the object;

comparing the captured image to the computer generated video image to detect the presence of the object;

wherein the comparing of the capture image to the computer generated image further includes;

reading a block portion of the captured image corresponding to the location of the button; and

checking the block portion for the presence of the object;

generating a control signal for actuating the computer in response to the presence of the object;

self testing to verify calibration of displayed images with captured images, and wherein the self testing includes;

storing a first image and a second image, said first and second stored images each having a common predetermined location associated therewith;

said second stored image further including an object disposed over its respective, predetermined location;

displaying said second stored image;

comparing the displayed second stored image with said first stored image; and

triggering a predetermined computer operation in accordance with the relationship of the respective predetermined computer locations of said first and second stored images during comparison with each other.