Video actuated interactive environment

• US 20040102247A1
• Filed: 11/05/2003
• Published: 05/27/2004
• Est. Priority Date: 11/05/2002
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Smoot, Lanny Starkes, McArdle, William Joseph
• Granted Patent: US 7,775,883 B2
• Time in Patent Office: Days
• US Class Current: 463/36
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/577 using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1062 being specially adapted to a type of game, e.g. steering wheel
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: A63F 2300/8023 the game being played by multiple players at a common site, e.g. in an arena, theatre, shopping mall using a large public display
• CPC Class Codes: A63F 2300/8088 involving concurrently several players in a non-networked game, e.g. on the same game console
• CPC Class Codes: G02B 26/0816 by means of one or more reflecting elements
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G07F 17/32 for games, toys, sports, or amusements

Abstract

A surface-based interactive environment that utilizes low cost equipment to create a highly interactive environment for participants. The game can include electronic versions of traditional games, such as air hockey, football, soccer, and the like. The game can also include one or more electronically generated characters with whom the participants can interact. The system utilizes a high-mounted projector to create the game environment and/or generate the characters. A remotely mounted infrared light source and infrared camera are used to detect the participants as they move around within the interaction area of the game.

Claims

1. A video-actuated system for creating an interactive environment, comprising:

a interaction surface;

at least one participant;

at least one energy source capable of generating electromagnetic energy in a relatively limited wavelength range, wherein the at least one energy source is positioned such that the energy from the at least one energy source strikes at least a portion of the interaction surface;

at least one electromagnetic imaging device, wherein the at least one electromagnetic imaging device is capable of detecting electromagnetic energy in at least the relatively limited wavelength range generated by the at least one energy source and converting the detected energy into an output signal, and wherein the at least one electromagnetic imaging device is positioned such that the at least one electromagnetic imaging device can detect the electromagnetic energy from at least a portion of the interaction surface;

at least one visible light projector having at least one input, wherein the at least one projector is capable of projecting visible light onto the interaction surface;

at least one computer; and

at least one computer program process operating on the at least one computer, capable of receiving the output signal from the electromagnetic imaging device, detecting blobs of infrared energy within the output signal, reporting information about the blobs, processing the blob information, and generating an input to the at least one visible light projector based on the processed blob information.

2. The system of claim 1, further comprising at least one relatively narrow wavelength pass filter, wherein the at least one pass filter allows electromagnetic energy substantially equivalent to the relatively narrow wavelength range generated by the energy source to pass through effectively unattenuated while attenuating electromagnetic energy at other wavelengths, and wherein the at least one pass filter is positioned substantially between the at least one imaging device and the interaction surface, such that substantially all electromagnetic energy entering the at least one imaging device passes through at least one pass filter.

3. The system of claim 1, further comprising at least one relatively narrow wavelength stop filter, wherein the at least one stop filter attenuates electromagnetic energy substantially equivalent to the relatively narrow wavelength range generated by the electromagnetic energy source while allowing other electromagnetic wavelengths to pass through effectively unattenuated, and wherein the at least one stop filters are positioned substantially between the at least one visible light projector and the surface, such that substantially all light from the at least one visible light projector passes through the at least one stop filter.

4. The system of claim 1, further comprising:

at least one relatively narrow wavelength pass filter, wherein the at least one pass filter allows electromagnetic energy substantially equivalent to the relatively narrow wavelength range generated by the energy source to pass through effectively unattenuated while attenuating electromagnetic energy at other wavelengths, and wherein the at least one pass filter is positioned substantially between the at least one imaging device and the interaction surface, such that substantially all electromagnetic energy entering the at least one imaging device passes through at least one pass filter; and

at least one relatively narrow wavelength stop filter, wherein the at least one stop filter attenuates electromagnetic energy substantially equivalent to the relatively narrow wavelength range generated by the electromagnetic energy source while allowing other electromagnetic wavelengths to pass through effectively unattenuated, and wherein the at least one stop filters are positioned substantially between the at least one visible light projector and the interaction surface, such that substantially all light from the at least one visible light projector passes through the at least one stop filter.

5. The system of claim 1, wherein the energy source generates electromagnetic energy with wavelengths in the infrared range.

6. The system of claim 1, wherein the interaction surface is comprised of a floor, and the energy source is positioned substantially above the interaction surface.

7. The system of claim 1, wherein the electromagnetic imaging device is a grayscale CCD camera.

8. The system of claim 1, wherein the interaction surface is comprised of a floor, and the imaging device is positioned substantially above the interaction surface.

9. The system of claim 1, wherein blob detection occurs on a frame-by-frame basis.

10. The system of claim 1, wherein blob detection is based on at least one customizable threshold brightness level.

11. The system of claim 1, wherein blob detection is based on at least one customizable blob size threshold.

12. The system of claim 11, wherein blob detection is based on at least one customizable threshold brightness level.

13. The system of claim 1, wherein blob detection is controllable based on a customizable granularity setting.

14. The system of claim 1, wherein a target is associated with the at least one participant.

15. The system of claim 14, wherein the target is a retro-reflective target.

16. The system of claim 1, wherein a plurality of targets are associated with each of the at least one participants.

17. The system of claim 16, wherein three targets of a known orientation are associated to each of the at least one participants.

18. The system of claim 17, wherein the blob information includes the size and location of each blob.

19. The system of claim 18, wherein the blob information is used to determine the orientation of each of the at least one the participants.

20. The system of claim 19, wherein the orientation of at least one of the at least one participants is used by the at least one computer program process to generate the input to the visual light projector.

21. The system of claim 17, wherein the blob information is used to determine the height of a prop above the interaction surface.

22. The system of claim 17, wherein the blob information is used to determine the scale of at least a portion of the image projected by the visual light projector.

23. The system of claim 1, wherein an electromagnetic energy source is assigned to each of the at least one participants.

24. The system of claim 1, wherein a pan and tilt mirror is placed in the optical path of the visible light projector.

25. The system of claim 1, wherein the visible light projector projects light encoded using a three dimensional display technique.

26. The system of claim 1, wherein the electromagnetic energy source projects electromagnetic energy onto at least a portion of the interaction surface from a position different from that from which the at least one electromagnetic imaging device detects the at least a portion of the interaction surface such that at least one of the at least one participants appears to cast a shadow on the at least a portion of the interaction surface when the at least a portion of the interaction surface is detected by the at least one electromagnetic imaging device.

27. The system of claim 26, wherein the blob detection algorithm determines the position of at least one participant shadow within the at least a portion of the interaction surface detected by the at least one electromagnetic imaging device.

28. The system of claim 27, wherein the at least one computer program process causes the at least one visible light projector to project an animated character in the region of the at least one participant shadow.

29. A video-actuated system for creating an interactive environment, comprising:

a interaction surface, comprised of a floor;

at least one participant, wherein at least one retro-reflective target is associated with the at least one participant;

an infrared energy source, wherein the infrared energy source is positioned substantially above the interaction surface and capable of projecting infrared energy onto the interaction surface;

at least one grayscale CCD camera, wherein the camera is capable of detecting the infrared energy generated by the infrared energy source and converting the detected energy into an output signal, and wherein the at least one camera is positioned substantially above the interaction surface;

at least one visible light projector having at least one input, wherein the at least one projector is capable of projecting visible light onto the interaction surface;

at least one relatively narrow wavelength pass filter, wherein the at least one pass filter allows infrared energy from the energy source to pass through effectively unattenuated while attenuating electromagnetic energy at other wavelengths, and wherein the at least one pass filter is positioned substantially between the at least one camera and the interaction surface, such that substantially all electromagnetic energy entering the at least one camera passes through at least one pass filter;

at least one relatively narrow wavelength stop filter, wherein the at least one stop filter attenuates infrared energy of a wavelength substantially equivalent to that generated by the electromagnetic energy source while allowing other electromagnetic wavelengths to pass through effectively unattenuated, and wherein the at least one stop filters are positioned substantially between the at least one visible light projector and the interaction surface, such that substantially all light from the at least one visible light projector passes through the at least one stop filter;

at least one computer; and

at least one computer program process operating on the at least one computer, capable of receiving the output signal from the camera, detecting blobs of infrared energy within the output signal, reporting information about the blobs, processing blob information, and generating an input to the at least one visible light projector based on the processed blob information.

30. The system of claim 29, wherein blob detection occurs on a frame-by-frame basis.

31. The system of claim 29, wherein blob detection is based on at least one customizable threshold brightness level.

32. The system of claim 29, wherein blob detection is based on at least one customizable blob size threshold.

33. The system of claim 32, wherein blob detection is based on at least one customizable threshold brightness level.

34. The system of claim 29, wherein blob detection is controllable based on a customizable granularity setting.

35. The system of claim 29, wherein a target is associated with the at least one participant.

36. The system of claim 35, wherein the target is a retro-reflective target.

37. The system of claim 29, wherein a plurality of targets are associated with each of the at least one participants.

38. The system of claim 37, wherein three targets of a known orientation are associated to each of the at least one participants.

39. The system of claim 38, wherein the blob information includes the size and location of each blob.

40. The system of claim 39, wherein the blob information is used to determine the orientation of each of the at least one the participants.

41. The system of claim 40, wherein the orientation of at least one of the at least one participants is used by the at least one computer program process to generate the input to the visual light projector.

42. The system of claim 29, wherein the visible light projector includes a pan and tilt mirror.

43. The system of claim 29, wherein the visible light projector projects light encoded using a three dimensional display technique.

44. A video-actuated system for creating an interactive environment, comprising:

a interaction surface;

a participant;

at least one imaging device, for generating an output signal representative of at least a portion of the interaction surface;

a visible light projector having at least one input, wherein the projector is capable of projecting visible light onto only a subset of the interaction surface;

a pan and tilt mirror system placed in the optical path of the projector, wherein the pan and tilt mirror system is capable of directing the output from the visible light projector to a desired position;

at least one computer;

at least one computer program process operating on the at least one computer, capable of receiving the output signal from the camera, detecting the position of the participant within the interaction surface, generating a signal for projecting a character in the proximity of the at least one participant, providing the signal to the input of the visible light projector, and providing approximately the detected position of the participant to the pan and tilt mirror system.

45. The system of claim 44, wherein the pan and tilt mirror system allows the generated character to follow the participant across the interaction surface.

46. A video-actuated system for creating an interactive environment, comprising:

a plurality of interaction surfaces;

at least one participant;

at least one energy source capable of generating electromagnetic energy in a relatively limited wavelength range, wherein the at least one energy source is positioned such that the energy from the at least one energy source strikes at least a portion of the interaction surface;

at least one electromagnetic imaging device, wherein the at least one electromagnetic imaging device is capable of detecting electromagnetic energy in at least the relatively limited wavelength range generated by the at least one energy source and converting the detected energy into an output signal, and wherein the at least one electromagnetic imaging device is positioned such that the at least one electromagnetic imaging device can detect the electromagnetic energy from at least a portion of the interaction surface;

a plurality of visible light projectors, wherein each of the plurality of visible light projectors has at least one input, and wherein each of the at least one projectors is capable of projecting visible light onto at least one of the plurality of interaction surfaces;

a plurality of computers; and

at least one computer program process operating on the plurality of computers, capable of receiving the output signal from at least one of the at least one electromagnetic imaging device;

detecting blobs of infrared energy within the output signal;

determining the position of the at least one participant in the plurality of interaction surfaces;

processing the determined participant position; and

generating an input to at least one of the plurality of visible light projectors based on the determined participant position.

47. The system of claim 46, wherein the generated input to the at least one of the plurality of visible light projectors causes at least one visible light projector to project a character near the determined participant position.

48. The system of claim 47, wherein the character can follow the participant as the participant moves among the plurality of interaction surfaces.

49. A video-actuated system for creating an interactive environment, comprising:

a plurality of interaction surfaces;

a participant;

at least one imaging device, wherein the at least one imaging device is capable of generating an output indicative of at least a portion of at least one of the plurality of interaction surfaces;

a plurality of visible light projectors, wherein each of the plurality of visible light projectors has at least one input, and wherein each of the at least one projectors is capable of projecting visible light onto at least one of the plurality of interaction surfaces;

a plurality of computers;

at least one computer program process operating on the plurality of computers, capable of receiving the output signal from at least one of the at least one imaging devices;

determining the position of the participant based on the output signal;

processing the determined participant position;

generating an input to at least one of the plurality of visible light projectors based on the determined participant position such that the at least one visible light projector projects a character near the determined participant position.

50. The system of claim 49, wherein the character follows the participant as the participant moves among the plurality of interaction surfaces.

51. A video-actuated system for creating an interactive environment, comprising:

an interactive surface;

a participant;

an energy source capable of generating electromagnetic energy in a relatively limited wavelength range, wherein the energy source is positioned such that the energy from the energy source strikes at least a portion of the interaction surface;

an electromagnetic imaging device, wherein the electromagnetic imaging device is capable of detecting electromagnetic energy in at least the relatively limited wavelength range generated by the energy source and converting the detected energy into an output signal, and wherein the electromagnetic imaging device is positioned such that the electromagnetic imaging device can detect the electromagnetic energy from at least a portion of the interaction surface;

a visible light projector having an input, wherein the visible light projector is capable of projecting visible light onto the interaction surface;

a video inverter, having an input connected to the output of the electromagnetic imaging device and an output connected to the input to the visible light projector, wherein the video inverter causes the visible light projector to project a visible light negative of the electromagnetic energy imaged by the electromagnetic imaging device.

52. The system of claim 51, further comprising a video subtractor, wherein the video subtractor is operatively connected between the output from electromagnetic imaging device and the input to the video inverter.