System and method for dynamic in-vehicle virtual reality

• US 10,850,744 B2
• Filed: 04/18/2018
• Issued: 12/01/2020
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Alaniz, Arthur, Whinnery, Joseph, Murrish, Robert Wesley, Gleeson-May, Michael Eamonn
• Primary Examiner(s): Robinson, Terrell M
• Application Number:
  US15/955,865
• Publication Number:
  US 20180229742A1
• Time in Patent Office: 958 Days
• Field of Search: 345633
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: B60R 2300/20 characterised by the type of display used
• CPC Class Codes: B60R 2300/80 characterised by the intended use of the viewing arrangement
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B62D 15/029 Steering assistants using warnings or proposing actions to the driver without influencing the steering system
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A computer-implemented method for rendering views to an output device and controlling a vehicle. The method includes determining a maneuver path for the vehicle within a spatial environment around the vehicle based on vehicle data from one or more vehicle systems of the vehicle. The method includes updating a view based on the spatial environment and the maneuver path, by augmenting one or more components of a model to provide a representation of the maneuver path virtually in the view as an available maneuver path or an unavailable maneuver path. The view is rendered to an output device and a vehicle maneuver request is generated based on the maneuver path. Further, the one or more vehicle systems are controlled to execute the vehicle maneuver request.

Claims

1. A computer-implemented method for rendering views for an immersive virtual reality or augmented reality to an output device and controlling a vehicle, comprising:

determining a maneuver path for the vehicle is an available maneuver path or an unavailable maneuver path based on vehicle data from one or more vehicle systems of the vehicle;

updating a view for the immersive virtual reality or augmented reality based on a spatial environment around the vehicle and the maneuver path by augmenting one or more components of a model that defines the view to provide an indicator corresponding to the available maneuver path thereby persuading a user input virtually in the view corresponding to the indicator and the available maneuver path;

rendering the view for the immersive virtual reality or augmented reality to the output device by controlling the output device to update display of the view according to the model on the output device;

in response to rendering the view, receiving the user input virtually in the view augmenting an object of the model displayed virtually in the view and corresponding to the indicator and the available maneuver path;

generating a vehicle maneuver request for the vehicle based on the maneuver path in the view and the user input; and

controlling the one or more vehicle systems of the vehicle to execute the vehicle maneuver request.

2. The computer-implemented method of claim 1, including determining an availability status of the vehicle maneuver request based on an updated spatial environment around the vehicle.

3. The computer-implemented method of claim 2, wherein upon determining the availability status of the vehicle maneuver request is invalid, transmitting an inter-vehicle request based on the vehicle maneuver request to one or more vehicles in the spatial environment.

4. The computer-implemented method of claim 3, wherein the inter-vehicle request includes a vehicle command that when executed by the one or more vehicles in the spatial environment thereby changes the availability status of the vehicle maneuver request to valid.

5. The computer-implemented method of claim 3, including upon receiving an authorization status in response to the inter-vehicle request, controlling the one or more vehicle systems of the vehicle to execute the vehicle maneuver request.

6. The computer-implemented method of claim 1, including updating the view based on the vehicle maneuver request by augmenting one or more components of the model to provide a representation of the vehicle maneuver request virtually in the view, and rendering the view to the output device by controlling the output device to update display of the view on the output device.

7. The computer-implemented method of claim 1, wherein the indicator persuades the user to control the object defined by of the model thereby resulting in the user input.

8. The computer-implemented method of claim 1, wherein the model is a virtual world model including the one or more components that define the view.

9. A computing system for rendering views for an immersive virtual reality or augmented reality to an output device in a vehicle, comprising:

a processor operably connected to the output device and the vehicle, the processor, comprising;

a virtual reality data module determines a maneuver path for the vehicle, within a spatial environment around the vehicle, is an available maneuver path or an unavailable maneuver path based on vehicle data from one or more vehicle systems of the vehicle;

a dynamic virtual reality module updates a view for the immersive virtual reality or augmented reality based on the spatial environment and the maneuver path, by augmenting one or more components of a model that defines the view to provide a persuasion indicator of the maneuver path virtually in the view based on the maneuver path being either the available maneuver path or the unavailable maneuver path, wherein the persuasion indicator is selected from an available indicator associated with the available maneuver path and an unavailable indicator associated with the unavailable maneuver path;

a rendering module renders the view for the immersive virtual reality or augmented reality to the output device by controlling the output device to update display of the view according to the model on the output device; and

the dynamic virtual reality module generates a vehicle maneuver request for the vehicle based on the maneuver path in the view and a user input virtually in the view persuaded by the persuasion indicator associated with the view,

and the dynamic virtual reality module controls one or more vehicle systems of the vehicle to execute the vehicle maneuver request.

10. The computing system of claim 9, wherein the dynamic virtual reality module generates the vehicle maneuver request by receiving a user input selecting the persuasion indicator.

11. The computing system of claim 9, wherein the dynamic virtual reality module determines an availability status of the vehicle maneuver request based on an updated spatial environment around the vehicle.

12. The computing system of claim 11, wherein upon determining the availability status of the vehicle maneuver request is invalid, the dynamic virtual reality module transmits an inter-vehicle request based on the vehicle maneuver request to one or more vehicles in the spatial environment.

13. The computing system of claim 12, wherein the inter-vehicle request includes a vehicle command that when executed by the one or more vehicles in the spatial environment thereby changes the availability status of the vehicle maneuver request to valid.

14. The computing system of claim 9, wherein the dynamic virtual reality module provides the available indicator as an incentive when the virtual reality data module determines the maneuver path is the available maneuver path.

15. The computing system of claim 9, wherein the dynamic virtual reality module provides the persuasion indicator as a deterrent when the virtual reality data module determines the maneuver path is the unavailable maneuver path.

16. A non-transitory computer-readable storage medium storing instructions that, when executed by a computer, causes the computer to perform a method comprising:

determining a maneuver path for a vehicle, within a spatial environment around the vehicle, is an available maneuver path or an unavailable maneuver path based on vehicle data from one or more vehicle systems of the vehicle;

updating a view for an immersive virtual reality or augmented reality based on the spatial environment and the maneuver path, by augmenting one or more components of a model that defines the view to provide a persuasion indicator with a representation of the maneuver path virtually in the view based on the maneuver path being either the available maneuver path or the unavailable maneuver path, wherein the persuasion indicator is selected from an available indicator associated with the available maneuver path and an unavailable indicator associated with the unavailable maneuver path;

rendering the view for the immersive virtual reality or augmented reality to an output device by controlling the output device to update display of the view according to the model on the output device;

generating a vehicle maneuver request for the vehicle based on the maneuver path in the view and a user input received virtually in the view; and

controlling the one or more vehicle systems of the vehicle to execute the vehicle maneuver request.

17. The non-transitory computer-readable storage medium of claim 16, including updating the view based on the vehicle maneuver request by augmenting one or more components of the model to provide a representation of the vehicle maneuver request virtually in the view, and rendering the view to the output device by controlling the output device to update display of the view on the output device.

18. The non-transitory computer-readable storage medium of claim 16, wherein generating the vehicle maneuver request includes receiving a user input selecting the persuasion indicator.

19. The non-transitory computer-readable storage medium of claim 16, including determining an availability status of the vehicle maneuver request based on an updated spatial environment around the vehicle.

20. The non-transitory computer-readable storage medium of claim 16, wherein the available indicator is an incentive defined by the model to persuade the user input when the maneuver path is the available maneuver path.