Hypercatching in virtual reality (VR) system

• US 10,139,899 B1
• Filed: 11/30/2017
• Issued: 11/27/2018
• Est. Priority Date: 11/30/2017
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Niemeyer, Gunter D., Pan, Matthew K
• Primary Examiner(s): Chow, Jeffrey
• Application Number:
  US15/827,011
• Time in Patent Office: 362 Days
• Field of Search: None
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 13/816 Athletics, e.g. track-and-field sports
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 3/20 Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

A system designed to provide a dynamic physical interaction for a user during a virtual reality (VR) experience in a VR experience space. The system includes a VR system having a headset with a display screen and a VR rendering module generating a video output, and the display screen displays an image of a virtual world using the video output. The system includes an object delivery system delivering a physical object into the VR experience space to move toward or near to the user wearing the headset. The system includes, in the VR system, an interaction module generating a predicted trajectory for the physical object in the VR experience space. The image of the virtual world includes a virtual object corresponding to the physical object, and the image of the virtual object follows a virtual object trajectory in the virtual world differing from the predicted trajectory for the physical object.

Claims

1. A system for providing a dynamic physical interaction to a user during a virtual reality (VR) experience in a VR experience space, comprising:

a VR system including a headset with a display screen and a VR rendering module generating a video output, wherein the display screen displays an image of a virtual world based on the video output;

a physical object moving in the VR experience space toward the user wearing the headset; and

in the VR system, an interaction module generating a predicted trajectory for the physical object in the VR experience space,

wherein the image of the virtual world includes an image of a virtual object corresponding to the physical object,

wherein the image of the virtual object is rendered such that the image of the virtual object moves along a travel path in the virtual world,

wherein the image of the virtual object follows a virtual object trajectory as the image of the virtual object moves along the travel path in the virtual world that differs in at least one aspect from the predicted trajectory for the physical object in the VR experience space, and

wherein the virtual object trajectory is time synchronized with the predicted trajectory whereby a virtual interaction between the user and the virtual object in the virtual world occurs simultaneously with a physical interaction between the user and the physical object in the VR experience space.

2. The system of claim 1, further comprising an object delivery system moving the physical object in the VR experience space, wherein the interaction module includes a trajectory prediction algorithm generating the predicted trajectory based on design parameters and location of the object delivery system and based on physical characteristics of the physical object.

3. The system of claim 1, further including a tracking mechanism tracking movement of the physical object in the VR experience space and wherein the interaction module updates the predicted trajectory by processing the tracked movement.

4. The system of claim 1, wherein the VR system has a latency in generating the video output and wherein the video output is generated using forward rendering of the virtual object on the virtual object trajectory to correct for the latency.

5. The system of claim 1, wherein the headset has position mis-calibration of the displayed image and wherein the virtual object trajectory is generated to reposition the virtual object in the virtual world based on the position mis-calibration.

6. The system of claim 1, wherein the virtual object trajectory includes at least two velocities for the virtual object along the travel path and wherein at least one of the two velocities differs from a velocity for the physical object defined in the predicted trajectory for the physical object in the VR experience space.

7. The system of claim 1, wherein the VR system further includes a tracking mechanism for tracking a movement of a body part of the user or of a physical object worn or held by the user and wherein the virtual object trajectory is modified in response to the tracked movement, whereby the VR system operates to guide the user to interact with the physical object in the VR experience space.

8. The system of claim 2, wherein the object delivery system delivers by the physical object by pitching or dropping the physical object such that the physical object is untethered as it flies or falls along a travel path through the VR experience space.

9. A system for providing a dynamic physical interaction to a user during a virtual reality (VR) experience in a VR experience space, comprising:

a VR rendering module generating a video of a virtual world;

a headset, wearable by a participant in the VR experience, with a screen displaying the video of the virtual world;

a tracking system for tracking velocity and location of at least one portion of a body of the participant in a space;

a physical object moving in the space, wherein the VR rendering module operates to include an image of a virtual object in the virtual world based corresponding to the physical object and an image of a virtual body part corresponding to the at least one portion of the body of the participant based on the velocity and the location from the tracking system; and

an interaction module determining a predicted trajectory for the physical object in the space, wherein the VR rendering module generates the video of the virtual world with the virtual object following a virtual object trajectory that differs from the predicted trajectory as the virtual object is rendered to move along a travel path in the virtual world toward or away from the image of the virtual body part.

10. The system of claim 9, wherein the VR rendering module has a latency in generating the video of the virtual world and wherein the video of the virtual world is generated using forward rendering of the virtual object based on the virtual object trajectory to correct for the latency.

11. The system of claim 9, wherein the headset has position mis-calibration of the displayed image and wherein the virtual object trajectory is generated to reposition the virtual object in the virtual world based on the position mis-calibration.

12. The system of claim 9, wherein the virtual object trajectory includes a travel path for the virtual object in the virtual world that differs from a travel path for the physical object defined by the predicted trajectory for the physical object in the space.

13. The system of claim 9, wherein the virtual object trajectory includes at least two velocities for the virtual object along a travel path and wherein at least one of the two velocities differs from a velocity for the physical object defined in the predicted trajectory for the physical object in the space.

14. The system of claim 9, wherein the VR system further includes a tracking mechanism for tracking a movement of a body part of the participant or of a physical object worn or held by the participant and wherein the virtual object trajectory is modified in response to the tracked movement, whereby the system operates to guide the participant to interact with the physical object in the space.

15. The system of claim 9, wherein the virtual object trajectory is time synchronized with the predicted trajectory whereby a virtual interaction between the participant and the virtual object in the virtual world occurs simultaneously with a physical interaction between the participant and physical object in the space.

16. A system for providing a dynamic physical interaction to a user during a virtual reality (VR) experience in a VR experience space, comprising:

a VR system including a headset with a display screen and a VR rendering module generating a video output, wherein the display screen displays an image of a virtual world based on the video output;

an object delivery system delivering a physical object into the VR experience space to move toward the user wearing the headset; and

in the VR system, an interaction module generating a predicted trajectory for the physical object in the VR experience space,

wherein the image of the virtual world includes an image of a virtual object corresponding to the physical object,

wherein the image of the virtual object follows a virtual object trajectory in the virtual world as the image of the virtual object is rendered to move along a travel path in the virtual world,

wherein the object delivery system delivers by the physical object by launching or by dropping the physical object such that the physical object is untethered as it flies or falls along a travel path through the VR experience space,

wherein the travel path for the virtual object differs in at least one aspect from a travel path for the physical object defined by the predicted trajectory for the physical object in the VR experience space, and

wherein the virtual object trajectory is time synchronized with the predicted trajectory whereby a virtual interaction between the user and the virtual object in the virtual world occurs simultaneously with a physical interaction between the user and the physical object in the VR experience space.

17. The system of claim 16, wherein the VR system has a latency in generating the video output, wherein the video output is generated using forward rendering of the virtual object on the virtual object trajectory to correct for the latency, wherein the headset has position mis-calibration of the displayed image, and wherein the virtual object trajectory is generated to reposition the virtual object in the virtual world based on the position mis-calibration.

18. The system of claim 16, wherein the virtual object trajectory includes at least two velocities for the virtual object along a travel path and wherein at least one of the two velocities differs from a velocity for the physical object defined in the predicted trajectory for the physical object in the VR experience space.

19. The system of claim 16, wherein the VR system further includes a tracking mechanism for tracking a movement of a body part of the user or of a physical object worn or held by the user and wherein the virtual object trajectory is modified in response to the tracked movement, whereby the VR system operates to guide the user to interact with the physical object in the VR experience space.