System and method for dynamic in-vehicle virtual reality

• US 10,638,106 B2
• Filed: 01/30/2019
• Issued: 04/28/2020
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Alaniz, Arthur, Whinnery, Joseph, Murrish, Robert Wesley, Gleeson-May, Michael Eamonn
• Primary Examiner(s): Robinson, Terrell M
• Application Number:
  US16/262,814
• Publication Number:
  US 20190166352A1
• Time in Patent Office: 454 Days
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

A method for rendering views to an output device in a vehicle, including receiving vehicle data from a portable device. The vehicle data includes vehicle dynamics data and predictive vehicle data within a frame of reference of an exterior environment of the vehicle. The method also includes generating a view based on the vehicle data and a model. The model includes one or more components that define the view. The method also includes determining a difference between the vehicle dynamics data and the predictive vehicle data indicative of a perspective based on motion of the vehicle within the frame of reference of the exterior environment of the vehicle. The method further includes rendering the view to the output device by controlling the output device to update display of the view according to the model and the determined difference.

Claims

1. A computer-implemented method for rendering views to an output device in a vehicle, comprising:

receiving vehicle data from a portable device, wherein the vehicle data includes vehicle dynamics data and predictive vehicle data within a frame of reference of an exterior environment of the vehicle, wherein the vehicle dynamics data includes a current velocity, a current orientation, and a current location of the vehicle, and wherein the predictive vehicle data includes a future velocity, a future orientation, and a future location of the vehicle at a future time;

generating a view based on the vehicle data and a model, the model including one or more components that define the view;

determining a difference between the vehicle dynamics data and the predictive vehicle data indicative of a perspective based on motion of the vehicle within the frame of reference of the exterior environment of the vehicle; and

rendering the view to the output device by controlling the output device to update display of the view according to the model and the determined difference.

2. The computer-implemented method of claim 1, wherein the predictive vehicle data triggers an event to define the view.

3. The computer-implemented method of claim 1, wherein determining the difference between the vehicle dynamics data and the predictive vehicle data includes determining a relative velocity of the vehicle at a current time relative to the future time, and wherein rendering the view to the output device includes modifying the view according to the model and the relative velocity.

4. The computer-implemented method of claim 1, wherein the predictive vehicle data is calculated based on the vehicle data from the portable device.

5. The computer-implemented method of claim 1, wherein rendering the view includes augmenting the one or more components of the model based on the determined difference.

6. The computer-implemented method of claim 1, wherein the predictive vehicle data is calculated based on user data received from the portable device, and wherein the user data includes a velocity, an orientation and a location of a user within a frame of reference of the vehicle, wherein the user is located in the vehicle.

7. A computing system for rendering views to an output device in a vehicle that is moving, comprising:

a processor operatively connected for computer communication to one or more vehicle systems of the vehicle and the output device, the processor including;

a virtual reality data module receiving vehicle data from a portable device, wherein the vehicle data includes vehicle dynamics data and predictive vehicle data within a frame of reference of an exterior environment of the vehicle, wherein the vehicle dynamics data includes a current velocity, a current orientation, and a current location of the vehicle, and wherein the predictive vehicle data includes a future velocity, a future orientation, and a future location of the vehicle at a future time;

a dynamic virtual reality module generates a view based on the vehicle data and a model, the model including one or more components that define the view; and

a rendering module determines a difference between the vehicle dynamics data and the predictive vehicle data indicative of a perspective based on motion of the vehicle within the frame of reference of the exterior environment of the vehicle, and renders the view to the output device by controlling the output device to update display of the view according to the model and the determined difference.

8. The computing system of claim 7, wherein the predictive vehicle data triggers an event to define the view.

9. The computing system of claim 7, wherein the rendering module determines the difference between the vehicle dynamics data and the predictive vehicle data by determining a relative velocity of the vehicle at a current time relative to the future time, and renders the view to the output device by modifying the view according to the model and the relative velocity.

10. The computing system of claim 7, wherein the predictive vehicle data is calculated based on the vehicle data from the portable device.

11. The computing system of claim 7, wherein the rendering module augments one or more components of the model based on the determined difference.

12. A non-transitory computer-readable medium comprising instructions that when executed by a processor perform a method for rendering views to an output device in a vehicle, comprising:

receiving vehicle data from a portable device, wherein the vehicle data includes vehicle dynamics data and predictive vehicle data within a frame of reference of an exterior environment of the vehicle, wherein the vehicle dynamics data includes a current velocity, a current orientation, and a current location of the vehicle, and wherein the predictive vehicle data includes a future velocity, a future orientation, and a future location of the vehicle at a future time;

generating a view based on the vehicle data and a model, the model including one or more components that define the view;

determining a difference between the vehicle dynamics data at a current time and the predictive vehicle data indicative of a perspective based on motion of the vehicle within the frame of reference of the exterior environment of the vehicle at the future time; and

rendering the view to the output device by controlling the output device to update display of the view according to the model and the determined difference.

13. The non-transitory computer-readable medium of claim 12, wherein determining the difference between the vehicle dynamics data and the predictive vehicle data includes determining a relative velocity of the vehicle at the current time relative to the future time, and wherein rendering the view to the output device includes modifying the view according to the model and the relative velocity.

14. The non-transitory computer-readable medium of claim 12, wherein the predictive vehicle data is calculated based on the vehicle data from the portable device.

15. The non-transitory computer-readable medium of claim 12, wherein rendering the view includes augmenting the one or more components of the model based on the determined difference.

16. The non-transitory computer-readable medium of claim 12, wherein the predictive vehicle data is calculated based on user data received from the portable device, and wherein the user data includes a velocity, an orientation and a location of a user within a frame of reference of the vehicle, wherein the user is located in the vehicle.

17. The non-transitory computer-readable medium of claim 12, wherein the predictive vehicle data triggers an event to define the view.