System and method for dynamic in-vehicle virtual reality

• US 10,261,576 B2
• Filed: 01/31/2017
• Issued: 04/16/2019
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Alaniz, Arthur, Whinnery, Joseph, Murrish, Robert Wesley, Gleeson-May, Michael Eamonn
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): Robinson, Terrell M
• Application Number:
  US15/420,536
• Publication Number:
  US 20170139473A1
• Time in Patent Office: 805 Days
• Field of Search: 345633
• CPC Class Codes: B60R 2300/20 characterised by the type of display used
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A method for in-vehicle dynamic virtual reality includes receiving vehicle data from one or more vehicle systems of a vehicle, wherein the vehicle data includes vehicle dynamics data and receiving user data from a virtual reality device. The method includes generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data and the user data and rendering the virtual view to an output device by controlling the output device to update display of the virtual view according to the vehicle dynamics data.

Claims

1. A computer-implemented method for rendering views to an output device in a vehicle that is moving, comprising:

receiving vehicle data, the vehicle data including a velocity, an orientation, and a location of the vehicle within a first frame of reference, wherein the first frame of reference is defined by the vehicle relative to an exterior environment of the vehicle;

receiving user data, the user data including a velocity, an orientation and a location of a user within a second frame of reference different than the first frame of reference, wherein the second frame of reference is defined by the user located in the vehicle relative to the vehicle and independent of the exterior environment;

generating a view based on the vehicle data, the user data, and a model, the model including one or more components that define the view;

determining a difference between the user data from the second frame of reference and the vehicle data from the first frame of reference; and

rendering the view to the output device by controlling the output device to update display of the view according to the determined difference relative to the second frame of reference.

2. The computer-implemented method of claim 1, wherein comparing the user data and the vehicle data includes determining a relative velocity of the vehicle relative to the user and a relative velocity of the user relative to the vehicle, and rendering the view to the output device includes modifying the view according to the model and the determined difference between the relative velocity of the vehicle and the relative velocity of the user.

3. The computer-implemented method of claim 1, wherein the vehicle data includes predictive vehicle data defining the velocity, the orientation, and the location of the vehicle within the first frame of reference at a future point in time.

4. The computer-implemented method of claim 3, including modifying the view by triggering an event in the view according to the predictive vehicle data.

5. The computer-implemented method of claim 3, wherein determining the difference between the user data and the vehicle data includes determining a difference between the user data and the predictive vehicle data and rendering the view to the output device by controlling the output device to update display of the view according to the determined difference between the user data and the predictive vehicle data.

6. The computer-implemented method of claim 1, wherein the user data includes health data associated with the user and rendering the view to the output device includes modifying the view based on the health data to minimize sensory processing.

7. The computer-implemented method of claim 1, including determining a vehicle occupant action based on the vehicle data and the user data and modifying the view according to an event defined by the model and triggered by the vehicle occupant action.

8. The computer-implemented method of claim 1, including receiving output device data, the output device data including a velocity, an orientation and a location of the output device within the second frame of reference, wherein said output device is located in the vehicle and viewable by the user.

9. The computer-implemented method of claim 8, including determining a difference between the output device data and the vehicle data, and wherein rendering the view to the output device includes controlling the output device to update display of the view according to the determined difference between the user data and the vehicle data and the determined difference between the output device data and the vehicle data.

10. A computing system for rendering views to an output device in a vehicle that is moving, comprising:

a processor operatively connected for computer communication to one or more vehicle systems of the vehicle and the output device, the processor including;

a virtual reality data module receives vehicle data from the one or more vehicle systems of the vehicle, wherein the vehicle data includes a velocity, an orientation, and a location of the vehicle within a first frame of reference, wherein the first frame of reference is defined by the vehicle relative to an exterior environment of the vehicle, the virtual reality data module receives predictive vehicle data including a velocity, an orientation, and a location of the vehicle within the first frame of reference at a future point in time,

and the virtual reality data module receives user data the from the output device, the user data including a velocity, an orientation and a location of a user within a second frame of reference of the vehicle, wherein the second frame of reference is defined by the user located in the vehicle relative to the vehicle, and the user is in possession of the output device;

a dynamic virtual reality module generates a view based on the vehicle data, the predictive vehicle data, the user data, and a model, the model including one or more components that define the view; and

a rendering module determines a difference between the user data from the second frame of reference and the vehicle data from the first frame of reference, and the rendering module renders the view to the output device by controlling the output device to update display of the view according to the determined difference relative to the second frame of reference and the predictive vehicle data relative to the first frame of reference.

11. The computing system of claim 10, the rendering module determines the difference between a relative velocity of the vehicle relative to the user and a relative velocity of the user relative to the vehicle based on the vehicle data and the user data.

12. The computing system of claim 11, wherein the rendering module renders the view to the output device by controlling the output device to update display of the view according to the determined difference between the relative velocity of the vehicle relative to the user and the relative velocity of the user relative to the vehicle.

13. The computing system of claim 10, wherein the user data includes health data associated with the user, the health data indicating a sensory issue and the rendering module renders the view to the output device by modifying the view based on the health data to minimize the sensory issue.

14. The computing system of claim 10, wherein the dynamic virtual reality module determines a vehicle occupant action associated with the user and based on the vehicle data and the user data, and modifies the view according to an event defined by the model and triggered by the vehicle occupant action.

15. The computing system of claim 10, wherein the rendering module renders the view to the output device by controlling the output device to update display of the view to minimize a difference in motion between the user relative to the vehicle based on the determined difference between the user data and the vehicle data.

16. A non-transitory computer-readable medium comprising instructions that when executed by a processor perform a method for rendering views to an output device in a vehicle, comprising:

receiving vehicle data, the vehicle data including a velocity, an orientation, and a location of the vehicle within a first frame of reference, wherein the first frame of reference is defined by the vehicle relative to an exterior environment of the vehicle;

receiving user data, the user data including a velocity, an orientation and a location of a user within a second frame of reference different than the first frame of reference, wherein the second frame of reference is defined by the user located in the vehicle relative to the vehicle and independent of the exterior environment;

generating a view based on the vehicle data, the user data, and a model, the model including one or more components that define the view;

determining a difference between the user data from the second frame of reference and the vehicle data from the first frame of reference; and

rendering the view to the output device by controlling the output device to update display of the view according to the determined difference.

17. The non-transitory computer-readable medium of claim 16, wherein rendering the view to the output device by controlling the output device to update display of the view includes updating the view to minimize a difference in motion between the user relative to the vehicle based on the determined difference between the user data and the vehicle data.

18. The non-transitory computer-readable medium of claim 16, wherein the user data includes health data associated with the user and rendering the view to the output device includes modifying the view based on the health data to minimize sensory processing.

19. The non-transitory computer-readable medium of claim 16, including determining a vehicle occupant action based on the vehicle data and the user data and modifying the view according to an event defined by the model, the event triggered by the vehicle occupant action.

20. The non-transitory computer-readable medium of claim 16, wherein the vehicle data includes predictive vehicle data defining the velocity, the orientation, and the location of the vehicle within the first frame at a future point in time, and rendering the view to the output device includes updating display of the view according to the determined difference and the predictive vehicle data.