System and method of simulating first-person control of remote-controlled vehicles

• US 9,996,978 B2
• Filed: 02/08/2016
• Issued: 06/12/2018
• Est. Priority Date: 02/08/2016
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Olson, Joseph Logan, Goslin, Michael P., Wong, Clifford, Panec, Timothy
• Primary Examiner(s): Brier, Jeffrey A
• Application Number:
  US15/018,764
• Publication Number:
  US 20170228932A1
• Time in Patent Office: 855 Days
• CPC Class Codes: A63F 2300/00 Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet

Abstract

Systems and methods of simulating first-person control of remoted-controlled vehicles are described herein. The system may include one or more of a remote-controlled (RC) vehicle, a display interface, an input interface, and/or other components. The RC vehicle may have an image capturing device configured to capture in-flight images. View information representing the captured images may presented on a display worn and/or otherwise accessible to user. The input interface may allow the user to provide control inputs for dictating a path of the RC vehicle. Augmented reality graphics may be overlaid on the view information presented to the user to facilitate gameplay and/or otherwise enhance a user'"'"'s experience.

Claims

1. A system configured to simulate first-person control of remoted-controlled vehicles used to play an augmented reality game, the system comprising:

a remote controlled vehicle comprising;

an image capturing device;

one or more radio frequency communication components; and

a hardware controller configured to;

receive control signals to dictate a path of the vehicle; and

obtain view information representing visual information captured by the image capturing device and effectuate transmission of the view information via the one or more radio frequency communication components;

a display interface including a graphics rendering component and a display, the graphics rendering component being configured to effectuate presentation of augmented reality images via the display for viewing by a user of the display interface, the augmented reality images including at least some of the view information;

an input interface, the input interface comprising one or more input sensors configured to generate sensor output in accordance with user input, the user input including gesture-based input; and

one or more physical processors configured by machine-readable instructions to;

receive the sensor output generated by the input sensors of the input interface;

determine the control signals from the sensor output generated by the input sensors of the input interface;

effectuate communication of the control signals to the one or more radio frequency communication components of the vehicle;

obtain the view information from the image capturing device;

determine augmented reality information that represents one or more augmented reality graphics to be overlaid on the view information, wherein determining the augmented reality information comprises;

identifying a first real-world object depicted in the view information, the first real-world object comprising a second vehicle controlled by a second user;

determining that a first augmented reality graphic is to be overlaid on the first real-world object, the first augmented reality graphic comprising a game entity of the augmented reality game;

determining one or both of a tilt or a rotation of the first real-world object; and

configuring the presentation of the first augmented reality graphic based on one or both of the tilt or the rotation of the first real-world object;

communicate the augmented reality information and the view information to the display interface, the augmented reality information and the view information representing the augmented reality images presented via the display,

wherein the communication of control signals and presentation of the augmented reality images via the display provides a first-person perspective of the vehicle maneuvering along the path to simulate first-person control by the user;

obtain user input from the user associated with gameplay actions user within the augmented reality game;

obtain second user input from the second user associated with second gameplay actions within the augmented reality game; and

wherein determining the augmented reality information further comprises;

configuring a presentation of a second augmented reality graphic based on the user input such that the second augmented reality graphic depicts the gameplay actions; and

configuring a presentation of a third augmented reality graphic based on the second user input such that the third augmented reality graphic depicts the second gameplay actions, and wherein the second gameplay actions are depicted as being performed by the game entity.

2. The system of claim 1, further comprising:

one or more wireless communications networks configured to facilitate communication of one or more of;

information between two or more of the input interface, the one or more physical processors, or the vehicle;

the view information between two or more of the vehicle, the display interface, or the one or more physical processors;

or

the augmented reality information between the one or more physical processors and the display interface.

3. The system of claim 1, further comprising a motion capture system, the motion capture system being disposed in an environment of the vehicle, the motion capture system comprising:

one or more motion capture devices configured to generate second view information representing second visual information captured by the one or more motion capture devices, the second view information conveying views of the vehicle within the environment; and

one or more physical processors configured by machine-readable instructions to;

obtain the second view information; and

determine, based on the second view information, position and/or orientation of the vehicle within the environment.

4. The system of claim 3, wherein the one or more physical processors of the motion capture system are further configured by machine-readable instructions to:

based on the position and/or orientation of the vehicle, effectuate communication of safety control signals to the vehicle, the safety control signals being configured to restrict the path of the vehicle.

5. The system of claim 1, wherein the display of the display interface is a head mounted display.

6. The system of claim 1, wherein vehicle is one or more of a flying vehicle, a terrestrial vehicle, a subsurface water vehicle, or a surface water vehicle.

7. The system of claim 1, further comprising a feedback interface device, the feedback interface device being configured to generate haptic feedback for a user of the feedback interface device;

wherein the vehicle further comprises one or more motion sensors, the one or more motion sensors being configured to generate output signals conveying motion information associated with the vehicle; and

wherein the one or more physical processors are further configured by machine-readable instructions to;

obtain the output signals generated by the one or more motion sensors of the vehicle; and

dictate control of the feedback interface device based on the output signals generated by the one or more motion sensors of the vehicle.

8. The system of claim 7, wherein the feedback interface device and input interface are disposed on one or more wearable articles.

9. The system of claim 1, wherein the one or more physical processors are further configured by machine-readable instructions such that determining the control signals from the sensor output generated by the input sensors of the input interface is based on control Specification, the control Specification specifying associations between sensor output generated by the input sensors of the input interface and control signals, such that the control Specification specifies that a first sensor output generated by the one or more input sensor of input interface is associated with a first control signal.

10. A method to simulate first-person control of remoted-controlled vehicles used to play an augmented reality game, the method comprising:

providing a remote controlled vehicle, the vehicle comprising;

an image capturing device;

one or more radio frequency communication components; and

a hardware controller;

receiving, via the hardware controller, control signals to dictate a path of the vehicle;

obtaining, via the hardware controller, view information representing visual information captured by the image capturing device and effectuating transmission of the view information via the one or more radio frequency communication components;

providing a display interface including a graphics rendering component and a display;

effectuating, via the graphics rendering component, presentation of augmented reality images via the display for viewing by a user of the display interface, the augmented reality images including at least some of the view information;

providing an input interface, the input interface comprising one or more input sensors configured to generate sensor output in accordance with user input, the user input including gesture-based input;

receiving, by one or more physical processors, the sensor output generated by the input sensors of the input interface;

determining, by the one or more physical processors, the control signals from the sensor output generated by the input sensors of the input interface;

effectuating, by the one or more physical processors, communication of the control signals to the one or more radio frequency communication components of the vehicle;

obtaining, by the one or more physical processors, the view information from the image capturing device;

determining, by the one or more physical processors, augmented reality information that represents one or more augmented reality graphics to be overlaid on the view information, wherein determining the augmented reality information comprises;

identifying a first real-world object depicted in the view information, the first real-world object comprising a second vehicle controlled by a second user;

determining that a first augmented reality graphic is to be overlaid on the first real-world object, the first augmented reality graphic comprising a game entity of the augmented reality game;

determining one or both of a tilt or a rotation of the first real-world object; and

configuring the presentation of the first augmented reality graphic based on one or both of the tilt or the rotation of the first real-world object;

communicating, by the one or more physical processors, the augmented reality information and the view information to the display interface, the augmented reality information and the view information representing the augmented reality images presented via the display,

wherein the communication of control signals and presentation of the augmented reality images via the display provides a first-person perspective of the vehicle maneuvering along the path to simulate first-person control by the user;

obtaining user input from the user associated with gameplay actions user within the augmented reality game;

obtaining second user input from the second user associated with second gameplay actions within the augmented reality game; and

wherein determining the augmented reality information further comprises;

configuring a presentation of a second augmented reality graphic based on the user input such that the second augmented reality graphic depicts the gameplay actions; and

configuring a presentation of a third augmented reality graphic based on the second user input such that the third augmented reality graphic depicts the second gameplay actions, and wherein the second gameplay actions are depicted as being performed by the game entity.

11. The method of claim 10, further comprising one or more of:

communicating, by one or more wireless communication networks, information between two or more of the input interface, the one or more physical processors, or the vehicle;

communicating, by the one or more wireless communication networks, the view information between two or more of the vehicle, the display interface, or the one or more physical processors;

or

communicating, by the one or more wireless communication networks, the augmented reality information between the one or more physical processors and the display interface.

12. The method of claim 10, further comprising:

providing a motion capture system within an environment of the vehicle, the motion capture system comprising one or more motion capture devices and one or more physical processors;

generating, by the one or more motion capture devices, second view information representing second visual information captured by the one or more motion capture devices, the second view information conveying views of the vehicle within the environment;

obtaining, by the one or more physical processors, the second view information; and

determining, by the one or more physical processors, position and/or orientation of the vehicle within the environment based on the second view information.

13. The method of claim 10, further comprising:

determining, by the one or more physical processors, safety control signals based on the position and/or orientation of the vehicle,

effectuating, by the one or more physical processors, communication of safety control signals to the vehicle, the safety control signals being configured to restrict the path of the vehicle.

14. The method of claim 10, wherein the display of the display interface is a head mounted display.

15. The method of claim 10, wherein vehicle is one or more of a flying vehicle, a terrestrial vehicle, a subsurface water vehicle, or a surface water vehicle.

16. The method of claim 10, further comprising:

providing a feedback interface device to generate haptic feedback for a user of the feedback interface device;

providing one or more motion sensors on the vehicle to generate output signals conveying motion information associated with the vehicle; and

obtaining, by one or more physical processor, the output signals generated by the one or more motion sensors of the vehicle; and

dictating, by one or more physical processors, control of the feedback interface device based on the output signals generated by the one or more motion sensors of the vehicle.

17. The method of claim 16, wherein the feedback interface device and input interface are disposed on one or more wearable articles.

18. The method of claim 10, wherein determining the control signals from the sensor output generated by the input sensors of the input interface is based on control Specification, the control Specification specifying associations between sensor output generated by the input sensors of the input interface and control signals, such that the control Specification specifies that a first sensor output generated by the one or more input sensor of input interface is associated with a first control signal.