Virtual encounters

• US 9,948,885 B2
• Filed: 12/12/2003
• Issued: 04/17/2018
• Est. Priority Date: 12/12/2003
• Status: Active Grant

• Current Assignee: Beyond Imagination Inc.
• Original Assignee: Kurzweil Technologies, Inc.
• Inventors: Kurzweil, Raymond C.
• Primary Examiner(s): DUFFY, DAVID W
• Application Number:
  US10/735,595
• Publication Number:
  US 20050143172A1
• Time in Patent Office: 5,240 Days
• Field of Search: 463 30- 32, 463 34- 35, 463 40- 42, 600 38
• CPC Class Codes: A63F 13/21 characterised by their sensors, purposes or types
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/31 Communication aspects specific to video games, e.g. between several handheld game devices at close range
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: H04N 13/385 alternating rapidly the location of the left-right image components on the display screens for viewing without the aid of special glasses using time variant parallax barriers H04N13/315; displays for viewing with the aid of special glasses or head-mounted displays using temporal multiplexing H04N13/341
• CPC Class Codes: H04N 7/14 Systems for two-way working H04N7/12, H04N7/173 take precedence
• CPC Class Codes: H04N 7/141 between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085

Abstract

A virtual encounter system includes a mannequin coupled to a camera for receiving a video image. The camera sends the video image to a communications network. The virtual encounter system also includes a processor for overlaying a virtual environment over one or more portions of the video image to form a virtual scene and a set of goggles to render the virtual scene.

Claims

1. A virtual reality encounter system comprising:

a first mannequin;

a first camera supported by the first mannequin, the first camera for capturing a first image of a scene having a first environment in which the first mannequin is disposed;

a first processor receiving in real time the first image of the scene from the first camera supported by the mannequin, the first processor overlaying a virtual environment over one or more portions of the real-time received first image to form a first image of a virtual scene with the first image of the virtual scene including at least one remaining portion of the real-time received first image having the first environment, and with the overlaid virtual environment of the one or more portions, being a replacement background that has the scene that the first mannequin captures in the first image being replaced with a different environment from the first environment; and

sending the first image of the virtual scene including the at least one remaining portion of the real-time received first image in real time to a communications network; and

a first set of goggles to render a second image of a virtual scene from signals received from the communications network.

2. The system of claim 1, wherein the first mannequin is a first humanoid robot having tactile sensors positioned along the exterior of the first robot, the sensors sending first tactile signals to the communications network;

the system further including a body suit having tactile actuators, the tactile actuators receiving second tactile signals from the communications network.

3. The system of claim 2 further comprising:

wherein the first robot comprises a transmitter to wirelessly send the tactile signals and the first image to the communications network.

4. The system of claim 2, further comprising:

motion sensors positioned throughout the body suit, the motion sensors sending motion signals corresponding to movements of each sensor relative to a reference point, the motion signals transmitted to the communications network; and

a second humanoid robot, receiving, from the communications network, the motion signals from the motion sensors, the motion signals from the motion sensors causing a movement of the second robot that is correlated to a movement of the body suit.

5. The system of claim 4, wherein the second robot includes motion actuators corresponding to the motion sensors, the motion actuators causing the second robot to move.

6. The system of claim 4, wherein the second robot has life-like features, the second robot comprises:

a body; and

a microphone coupled to the body, the microphone for sending audio signals to the communications network.

7. The system of claim 6, wherein the first set of goggles further includes a first transducer to render the audio signals received from the microphone.

8. The system of claim 7, wherein the communications network comprises an interface having one or more channels for:

receiving the audio signals from the microphone;

receiving the first image from the first camera;

sending signals representing the second image of a virtual scene to the first set of goggles; and

sending the audio signals to the first transducer.

9. The system of claim 6, wherein the first set of the goggles and the first robot is at a first location and the first robot includes a first microphone for sending first audio signals to the communications network,

wherein the second humanoid robot is at a second location and the body of the second robot supports a second camera, the microphone coupled to the body of the second robot being a second microphone and the audio signals sent by the second microphone being second audio signals;

and

the system further comprising a second set of goggles to receive the first image of the virtual scene including at least one remaining portion of the real-time received first image and to receive the first audio signals from the first microphone.

10. The system of claim 9, wherein the communications network comprises:

a first communication gateway in the first location; and

a second communication gateway in the second location,

and the system further comprising a second processor connected to the first processor via a network.

11. The system of claim 9, wherein the body includes an eye socket and the second camera is positioned in the eye socket.

12. The system of claim 9, wherein the body includes an ear canal and the second microphone is positioned within the ear canal.

13. The system of claim 1, wherein the first set of goggles comprises a receiver to receive the second image of a virtual scene.

14. A method of having a virtual encounter, comprising:

receiving in real time a first video image of a scene in a first environment from a first camera coupled to a first mannequin, the first camera sending the first video image to a communications network;

sending first audio signals over the communications network, the first audio signals being produced from a first microphone coupled to the first mannequin;

overlaying a virtual environment over one or more portions of the first video image to form a first virtual scene, the first virtual scene including at least a remaining portion of the first video image and with the overlaid virtual environment of the one or more portions, being a replacement background that has the scene captured by the first camera on the first mannequin replaced with a different environment from the first environment;

rendering in real time the first virtual scene using a first set of goggles;

transducing the first audio signals received from the communications network using a transducer embedded in the first set of goggles;

sending second audio signals to the communications network from a second microphone coupled to a second mannequin having life-like features;

sending a second video image to the communications network from a second camera coupled to the second mannequin;

rendering the second image received from the communications network onto a monitor coupled to a second set of goggles; and

transducing the second audio signals received from the communications network using a second transducer embedded in the second set of goggles; and

sending from motion sensors positioned throughout a body suit motion actuating signals to first motion actuators over the communications network.

15. The method of claim 14, wherein the first mannequin is a first humanoid robot the body suit is a first body suit, and the method further comprises:

sending tactile signals received from tactile sensors positioned along an exterior of the first humanoid robot to the communications network; and

receiving the tactile signals from the communications network at a second body suit having tactile actuators.

16. The method of claim 15, further comprising:

sending motion signals from motion sensors positioned throughout the surface of a human, the motion signals corresponding to movements of each sensor relative to a reference point, the motion signals being transmitted to the communications network;

receiving, at the first humanoid robot, the motion signals sent by the motion sensors; and

causing a movement of the first humanoid robot that is correlated to a movement of the human based on the motion signals received from the motion sensors.

17. The method of claim 16, wherein receiving the motion signals comprises receiving motion signals from the motion sensors at corresponding motion actuators coupled to the first robot, causing a movement comprises the motion actuators causing the robot to move.

18. The method of claim 14, wherein the first set of goggles comprises a display to render the first virtual scene.

19. A method of having a virtual encounter, comprising:

receiving in real time a first video image of a first environment from a first camera coupled to a first mannequin, the first camera sending the first video image to a communications network;

sending first audio signals over the communications network, the first audio signals being produced from a first microphone coupled to the first mannequin;

overlaying a virtual environment over one or more portions of the first video image to form a first virtual scene, the first virtual scene including at least a remaining portion of the first video image, and with the overlaid virtual environment of the one or more portions, being a replacement background that replaces the first environment that the first mannequin captures in the first image with a different environment from the first environment; and

rendering in real time the first virtual scene using a first set of goggles.

20. The method of claim 19, further comprising:

sending second audio signals to the communications network from a second microphone coupled to a second mannequin having life-like features;

sending a second video image to the communications network from a second camera coupled to the second mannequin;

rendering the second image received from the communications network onto a monitor coupled to a second set of goggles; and

transducing the audio signals received from the communications network using a second transducer embedded in the second set of goggles.

21. The method of claim 20, wherein the second mannequin includes an eye socket and the second camera is positioned in the eye socket.

22. The method of claim 20, wherein the second mannequin includes an ear canal and further comprising positioning the second microphone within the ear canal.

23. The method of claim 20, wherein the second mannequin further comprises a transmitter to wirelessly send the second audio signals and the second video image to the communications network.

24. A virtual reality encounter system comprising:

a first mannequin including;

a first camera supported by the first mannequin, the first camera for capturing a first real-time image of a scene that encompasses a first environment in which the first mannequin is disposed;

a second mannequin including;

a second camera supported by the second mannequin, the second camera for capturing a second real-time image of a scene that encompasses the environment in which the second mannequin is disposed;

first motion sensors disposed over the second mannequin, the first motion sensors sending first motion signals over a communications network, and

first motion actuators disposed over the second mannequin, the first motion actuators receiving second, different motion signals from the communications network the second, different motion signals being different from the first motion signals;

a processor receiving in real time and processing the first real-time image and the second real-time image over the communications network, the processor configured to;

overlay a virtual environment over one or more portions of the received first real-time image to form a first image of a virtual scene with the first image of the virtual scene including at least one remaining portion of the first real-time image and with the overlaid virtual environment of the one or more portions, being a replacement background that has the scene captured by the first camera on the first mannequin replaced with a different environment from the first environment;

overlay a virtual environment over one or more portions of the received second real-time image to form a second image of a virtual scene with the second image of the virtual scene including at least one remaining portion of the second real-time image;

a set of goggles having a display, the set of goggles receiving and rendering on the display at least one of the first image of a virtual scene and the second image of a virtual scene from the communications network; and

a body suit having second motion sensors disposed over the body suit, the second motion sensors sending the second, different motion actuating signals to the first motion actuators over the communications network, the body suit further having second motion actuators disposed over the body suit, the second motion actuators receiving the first motion signals from the first motion sensors over the communications network.

25. The system of claim 24, wherein the first mannequin is a humanoid robot having tactile sensors and tactile actuators.

26. A virtual reality encounter system comprising:

a first humanoid robot having tactile sensors positioned along the exterior of the first robot, the sensors sending first tactile signals to a communications network;

the system further including a body suit, including

tactile actuators, the tactile actuators receiving second tactile signals from the communications network;

motion sensors positioned throughout the body suit, the motion sensors sending motion signals corresponding to movements of each sensor relative to a reference point, the motion signals transmitted to the communications network; and

a second humanoid robot, receiving, from the communications network, the motion signals from the motion sensors, the motion signals from the motion sensors causing a movement of the second robot that is correlated to a movement of the body suit;

a first camera supported by the first mannequin, the first camera for capturing a first image of a scene of a first environment that the first mannequin is disposed;

a first processor receiving in real time the first image of the scene from the first camera supported by the mannequin, the first processor overlaying a virtual environment over one or more portions of the real-time received first image to form a first image of a virtual scene with the first image of the virtual scene including at least one remaining portion of the real-time received first image and with the overlaid virtual environment of the one or more portions, being a replacement background that has the first environment captured by the first camera on the first mannequin replaced with a different environment from the first environment;

and sending the first image of the virtual scene including the at least one remaining portion of the real-time received first image in real time to a communications network; and

a first set of goggles to render a second image of a virtual scene from signals received from the communications network.

27. The system of claim 26, wherein the second robot includes motion actuators corresponding to the motion sensors, the motion actuators causing the second robot to move.

28. The system of claim 27, wherein the first set of the goggles and the first robot are at a first location, and the first robot includes a first microphone for sending first audio signals to the communications network, the second humanoid robot is at a second, different location and the body of the second robot supports a second camera and a second microphone coupled to the body of the second robot, being a second microphone and audio signals sent by the second microphone being second audio signals;

and

the system further comprising a second set of goggles to receive the first image of the virtual scene including at least one remaining portion of the real-time received first image and to receive the first audio signals from the first microphone.

29. The system of claim 27, wherein the communications network comprises:

a first communication gateway in the first location; and

a second communication gateway in the second location, and the system further comprises

a second processor connected to the first processor via the network.

30. The system of claim 27 wherein the first set of goggles comprises a receiver to receive the second image of a virtual scene;

the first processor overlaying the virtual environment overlays the virtual environment as a replacement background that has the scene that the first mannequin captures in the first image having the mannequin appear in a different environment; and

the second set of goggles receives the overlays the virtual environment as the replacement background.