System and method for dynamic in-vehicle virtual reality

• US 10,453,260 B2
• Filed: 03/09/2017
• Issued: 10/22/2019
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Alaniz, Arthur, Whinnery, Joseph, Murrish, Robert Wesley, Gleeson-May, Michael Eamonn
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): Robinson, Terrell M
• Application Number:
  US15/454,598
• Publication Number:
  US 20170186240A1
• Time in Patent Office: 957 Days
• Field of Search: 345633
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/216 using geographical information, e.g. location of the game device or player using GPS
• CPC Class Codes: A63F 13/332 using wireless networks, e.g. cellular phone networks
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: B60K 2360/55 Remote control arrangements
• CPC Class Codes: B60K 2360/589 Wireless data transfers
• CPC Class Codes: B60K 2360/77 Instrument locations other than the dashboard
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/213 Virtual instruments
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60K 35/80 Arrangements for controlling instruments
• CPC Class Codes: B60K 35/85 Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

A computer-implemented method for rendering views to an output device in a vehicle that is restricted to a fixed track that has a predetermined fixed path. Vehicle data about the vehicle is received from an off-vehicle device that is located remotely from the vehicle and along the predetermined fixed path. The vehicle data includes a velocity, an orientation, and a location of the vehicle. User data about a user in a vehicle is received that includes a velocity, an orientation, and a location. The method includes generating a view based on the vehicle data, the user data, and a model. The model includes one or more components that define the view. The method includes determining a difference between the vehicle data and the user data, and rendering the view to the output device by controlling the output device to display the view according to the determined difference.

Claims

1. A computer-implemented method for rendering views to an output device in a vehicle, wherein the vehicle is restricted to a fixed track that has a predetermined fixed path, comprising:

receiving vehicle data about the vehicle, the vehicle data including a velocity, an orientation, and a location of the vehicle, from an off-vehicle device that is located remotely from the vehicle and along the predetermined fixed path;

receiving user data about a user located in the vehicle, the user data including a velocity, an orientation, and a location of the user;

generating a view based on the vehicle data, the user data, and a model, wherein the model includes one or more components that define the view, wherein the one or more components are virtual reality components;

determining a difference between the vehicle data and the user data, wherein the one or more components of the model that define the view are augmented according to the difference between the vehicle data and the user data; and

rendering the view to the output device by controlling the output device to display the view based on the one or more augmented components of the model that define the view.

2. The computer-implemented method of claim 1, wherein the off-vehicle device obtains the vehicle data externally from the vehicle within a frame of reference of an exterior environment of the vehicle.

3. The computer-implemented method of claim 1, wherein the off-vehicle device is an imaging unit mounted to a stationary structure external to the vehicle and along the predetermined fixed path.

4. The computer-implemented method of claim 1, wherein determining the difference between the user data and the vehicle data includes determining a relative velocity of the vehicle relative to the user and a relative velocity of the user relative to the vehicle, and rendering the view to the output device includes augmenting the one or more components of the model that define the view according to the determined difference between the relative velocity of the vehicle and the relative velocity of the user.

5. The computer-implemented method of claim 1, wherein the vehicle data includes predictive vehicle data defining a velocity, an orientation, and a location of the vehicle at a future location of the vehicle on the fixed track.

6. The computer-implemented method of claim 5, including modifying the view by triggering an event in the view according to the predictive vehicle data.

7. The computer-implemented method of claim 5, including modifying a node class of the model to trigger an event in the view according to the predictive vehicle data.

8. The computer-implemented method of claim 1, wherein the user data includes health data associated with the user and rendering the view to the output device includes augmenting the one or more components of the model that define the view based on the health data to minimize sensory processing.

9. The computer-implemented method of claim 1, wherein the one or more components of the model includes a group of node classes, each node class of the group of node classes defining a property of the model, wherein rendering the view includes augmenting one or more nodes of the group of node classes based on the difference between the vehicle data and the user data.

10. A computing system for rendering views to an output device in a vehicle, wherein the vehicle is restricted to a fixed track that has a predetermined fixed path, comprising:

a processor operably connected for computer communication to the output device, the processor including;

a virtual reality data module receives vehicle data from an off-vehicle device that is located remotely from the vehicle and along the predetermined fixed path, wherein the vehicle data includes a velocity, an orientation, and a location of the vehicle, the virtual reality data module receives predictive vehicle data including a velocity, an orientation, and a location of the vehicle at a future point in time corresponding to a future point in location on the fixed track,

and the virtual reality data module receives user data the from the off-vehicle device, the user data including a velocity, an orientation and a location of a user within a frame of reference of the vehicle, wherein the user is located in the vehicle;

a dynamic virtual reality module generates a view based on the vehicle data, the predictive vehicle data, the user data, and a model, wherein the model includes one or more components that define the view, wherein the one or more components are virtual reality components; and

a rendering module determines a difference between the user data and the vehicle data, wherein the one or more components of the model that define the view are augmented according to the difference between the user data and the vehicle data, and the rendering module renders the view to the output device by controlling the output device to update display of the view based on the one or more augmented components of the model that define the view.

11. The computing system of claim 10, the rendering module determines the difference between a relative velocity of the vehicle relative to the user and a relative velocity of the user relative to the vehicle based on the vehicle data and the user data.

12. The computing system of claim 11, wherein the rendering module renders the view to the output device by controlling the output device to update display of the view according to the determined difference between the relative velocity of the vehicle relative to the user and the relative velocity of the user relative to the vehicle.

13. The computing system of claim 10, wherein the off-vehicle device is an imaging unit mounted to a stationary structure external to the vehicle and along the predetermined fixed path.

14. The computing system of claim 10, wherein the user data includes health data associated with the user, the health data indicating a sensory issue and the rendering module renders the view to the output device by augmenting the one or more components of the model that define the view based on the health data to minimize the sensory issue.

15. The computing system of claim 10, wherein the dynamic virtual reality module determines a vehicle occupant action associated with the user and based on the vehicle data and the user data, and modifies the view according to an event defined by the model and triggered by the vehicle occupant action.

16. A non-transitory computer-readable medium comprising instructions that when executed by a processor perform a method for rendering views to an output device in a vehicle, wherein the vehicle is restricted to a fixed track that has a predetermined fixed path, comprising:

receiving vehicle data about the vehicle, the vehicle data including a velocity, an orientation, and a location of the vehicle, from an off-vehicle device that is located remotely from the vehicle and along the predetermined fixed path;

receiving user data about a user located in the vehicle, the user data including a velocity, an orientation, and a location of the user;

generating a view based on the vehicle data, the user data, and a model, wherein the model includes one or more components that define the view, wherein the one or more components are virtual reality components;

determining a difference between the vehicle data and the user data, wherein the one or more components of the model that define the view are augmented according to the difference between the vehicle data and the user data; and

rendering the view to the output device by controlling the output device to display the view based on the one or more augmented components of the model that define the view.

17. The non-transitory computer-readable medium of claim 16, wherein determining the difference between the user data and the vehicle data includes determining a relative velocity of the vehicle relative to the user and a relative velocity of the user relative to the vehicle, and rendering the view to the output device includes augmenting the one or more components of the model that define the view according to the model and the determined difference between the relative velocity of the vehicle and the relative velocity of the user.

18. The non-transitory computer-readable medium of claim 16, wherein the vehicle data includes predictive vehicle data defining the velocity, the orientation, and the location of the vehicle within the frame of reference of the exterior environment of the vehicle at a future point in time, and rendering the view to the output device includes updating display of the view according to the determined difference and the predictive vehicle data.

19. The non-transitory computer-readable medium of claim 16, wherein the one or more components of the model includes a group of node classes, each node class of the group of node classes defining a property of the model.

20. The non-transitory computer-readable medium of claim 19, wherein rendering the view includes augmenting one or more nodes of the group of node classes based on the difference between the vehicle data and the user data.