Interactive video display system

• US 20020186221A1
• Filed: 05/28/2002
• Published: 12/12/2002
• Est. Priority Date: 06/05/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Reactrix Systems Incorporated
• Inventors: Bell, Matthew
• Granted Patent: US 7,259,747 B2
• Time in Patent Office: Days
• US Class Current: 345/474
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G03B 21/26 Projecting separately subsidiary matter simultaneously with main image
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A device allows easy and unencumbered interaction between a person and a computer display system using the person'"'"'s (or another object'"'"'s) movement and position as input to the computer. In some configurations, the display can be projected around the user so that that the person'"'"'s actions are displayed around them. The video camera and projector operate on different wavelengths so that they do not interfere with each other. Uses for such a device include, but are not limited to, interactive lighting effects for people at clubs or events, interactive advertising displays, etc. Computer-generated characters and virtual objects can be made to react to the movements of passers-by, generate interactive ambient lighting for social spaces such as restaurants, lobbies and parks, video game systems and create interactive information spaces and art installations. Patterned illumination and brightness and gradient processing can be used to improve the ability to detect an object against a background of video images.

Claims

1. A system for detecting an object and generating a display in response, the system comprising

a first source, for outputing electromagnetic energy in a first wavelength range;

a detector for detecting a reflection of the first source of electromagnetic energy from an object;

a processor coupled to the detector for using the detected reflection to generate a display signal;

a second source, for outputing electromagnetic energy at a second wavelength range, wherein the second source generates a visible display in response to the display signal, wherein the first and second wavelength ranges are different.

2. The system of claim 1, wherein the first source outputs light that is not in the visible spectrum and wherein the second source outputs light that is in the visible spectrum.

3. The system of claim 2, wherein the first source outputs infrared light.

4. The system of claim 1, wherein the second source includes a video projector.

5. The system of claim 4, wherein the video projector projects images from above the object.

6. The system of claim 4, wherein the video projector projects images on a surface adjacent to the object.

7. The system of claim 6, wherein the surface is part of a rear-projection system.

8. The system of claim 6, wherein the surface is part of a front-projection system.

9. The system of claim 1, wherein the object includes a human user.

10. The system of claim 1, wherein the first source includes a pattern of illumination.

11. The system of claim 10, wherein the pattern includes a random dot pattern.

12. The system of claim 10, further comprising

an infrared light-emitting diode cluster for generating the pattern of illumination.

13. The system of claim 1, further comprising

a process for determining an influence image, wherein the influence image includes a region around an object image derived from the object.

14. The system of claim 13, wherein the visible display includes an item, the system further comprising

a process for determining a gradient of the influence image;

and

a process for using the gradient to determine interaction between the object and the item.

15. The system of claim 1, wherein the visible display includes a rendering of a character.

17. The method of claim 16, wherein multiple cameras are used.

18. The method of claim 17, wherein at least two cameras are used to produce a stereo effect.

19. The method of claim 16, wherein a plasma screen is used to generate the displayed image.

20. The method of claim 16, wherein the displayed image includes advertising.

21. The method of claim 16, wherein the displayed image is part of a video game.

22. The method of claim 16, further comprising

strobing the light at a first wavelength.

23. The method of claim 16, further comprising

interconnecting multiple systems so that information about image detection and displays can be transferred between the systems.

24. The method of claim 23, further comprising

using the transferred information to create a single display from two or more displays.

26. The system of claim 25, wherein the patterned illumination includes a random dot pattern.

27. The system of claim 25, wherein the patterned illumination includes a checkerboard pattern.

28. The system of claim 25, wherein the patterned illumination also illuminates the object.

16. A method for obtaining an image of a human user wherein the human user is adjacent to a displayed image, the method comprising

using light at a first wavelength to illuminate the human user;

using a camera responsive to the light at a first wavelength to detect the image of the human user; and

using light at a second wavelength, different from the first wavelength, to generate the displayed image.

25. A method for detecting an object in an image captured with a camera, the method comprising

using a patterned illumination to illuminate a background and not the object;

and

using a processing system to define the object apart from the background.

29. A method for detecting an object in an image captured with a camera, the method comprising

using a patterned background;

and

using a processing system to define the object apart from the background.

30. A method for computing an interaction of an object with a video item, the method comprising

using a processor to determine a gradient for the object;

using a processor to determine a boundary for the video item; and

identifying an interaction by using the gradient and the boundary.

31. The system of claim 30, further comprising

using a processor to determine a brightness of an area derived from the object;

and

identifying an interaction by using the brightness of the area and the boundary.

32. The system of claim 30, wherein the interaction is a person pushing the item.

33. The system of claim 30, wherein the interaction is a person touching the item.

34. The system of claim 30, wherein the interaction is a person deforming the item.

35. The system of claim 30, wherein the interaction is a person manipulating the item

36. A system for detecting an object and generating an output in response, the system comprising

a first source, for outputing electromagnetic energy in a first wavelength range;

a detector for detecting a reflection of the first source of electromagnetic energy from an object;

a processor coupled to the detector for using the detected reflection to generate a display signal;

a second source, for outputing electromagnetic energy at a second wavelength range, wherein the second source generates a visible display in response to the display signal, wherein the first and second wavelength ranges are different; and

audio output for outputing audio.