System and method for dynamic in-vehicle virtual reality

• US 10,819,966 B2
• Filed: 01/30/2019
• Issued: 10/27/2020
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Alaniz, Arthur, Whinnery, Joseph, Murrish, Robert Wesley, Gleeson-May, Michael Eamonn
• Primary Examiner(s): Robinson, Terrell M
• Application Number:
  US16/262,802
• Publication Number:
  US 20190166351A1
• Time in Patent Office: 636 Days
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

A method for rendering views to an output device in a vehicle, including receiving vehicle data from a portable device, and determining vehicle occupant action data based on the vehicle data associated with a vehicle occupant of the vehicle. The method including generating a view based on the vehicle data and a model, the model including one or more components that define the view, wherein generating the view includes modifying one or more components of the model according to the vehicle occupant action data. The method including rendering the view to the output device by controlling the output device to update display of the view according to the modified one more components of the model.

Claims

1. A computer-implemented method for rendering an immersive virtual reality or augmented reality to an output device in a vehicle, comprising:

receiving vehicle data from a portable device, wherein the vehicle data includes a change to a vehicle system of the vehicle;

determining vehicle occupant action data based on the change to the vehicle system associated with a vehicle occupant of the vehicle;

generating a view for the immersive virtual reality or augmented reality with the portable device, based on the vehicle data and a model, the model including one or more components that define the view, wherein generating the view includes modifying one or more components of the model defined by an event according to the vehicle occupant action data, and wherein the modification is based on the vehicle data; and

rendering the view to the output device by controlling the output device to update display of the view for the immersive virtual reality or augmented reality according to the modified one or more components of the model.

2. The computer-implemented method of claim 1, wherein the vehicle occupant action data is based on an interaction between the vehicle occupant and one or more vehicle systems that affect operation of the vehicle, and wherein the change to the vehicle system affects the operation of the vehicle.

3. The computer-implemented method of claim 1, wherein the vehicle occupant action data triggers the event defined by the model.

4. The computer-implemented method of claim 1, wherein the modification causes a related adjustment to one or more vehicle systems.

5. The computer-implemented method of claim 1, wherein the vehicle occupant action data is based on an interaction between the vehicle occupant and a first vehicle system, and wherein the vehicle occupant action data triggers an adjustment of a second vehicle system.

6. The computer-implemented method of claim 5, wherein the adjustment of the second vehicle system is based on the vehicle data.

7. A computing system for rendering views for an immersive virtual reality or augmented reality to an output device in a vehicle, comprising:

a processor operatively connected for computer communication to at least one vehicle system of the vehicle and the output device, the processor including;

a virtual reality data module receiving vehicle data associated with one or more vehicle systems from a portable device and identifying vehicle occupant action data based on a change to a vehicle system of the one or more vehicle systems associated with a vehicle occupant of the vehicle;

a dynamic virtual reality module generates a view for the immersive virtual reality or augmented reality with the portable device based on the vehicle data and a model, the model including one or more components that define the view and the dynamic virtual reality module modifies at least one of the one or more components of the model according to the vehicle occupant action data, wherein an event defines a modification to at least one component of the model, and wherein the modification is based on the vehicle data; and

a rendering module renders the view to the output device by controlling the output device to update display of the view according to the modified one more components of the model.

8. The computing system of claim 7, wherein the vehicle occupant action data is based on an interaction between the vehicle occupant and the one or more vehicle systems that affect operation of the vehicle.

9. The computing system of claim 7, wherein the vehicle occupant action data triggers the event defined by the model.

10. The computing system of claim 7, wherein the modification causes a related adjustment to one or more vehicle systems.

11. The computing system of claim 7, wherein the vehicle occupant action data is based on an interaction between the vehicle occupant and a first vehicle system, and wherein the vehicle occupant action data triggers an adjustment of a second vehicle system.

12. A non-transitory computer-readable medium comprising instructions that when executed by a processor perform a method for rendering views for an immersive virtual reality or augmented reality to an output device in a vehicle, comprising:

receiving vehicle data from a portable device;

determining vehicle occupant action data based on a change to a vehicle system of the one or more vehicle systems and the vehicle data associated with a vehicle occupant of the vehicle;

generating a view for the immersive virtual reality or augmented reality, via the portable device, based on the vehicle data and a model, the model including one or more components that define the view, wherein generating the view includes modifying one or more components of the model defined by an event according to the vehicle occupant action data, and wherein the modification is based on the vehicle data; and

rendering the view to the output device by controlling the output device to update display of the view according to the modified one or more components of the model.

13. The non-transitory computer-readable medium of claim 12, wherein the vehicle occupant action data is based on an interaction between the vehicle occupant and one or more vehicle systems that affect operation of the vehicle.

14. The non-transitory computer-readable medium of claim 12, wherein the vehicle occupant action data triggers the event defined by the model.

15. The non-transitory computer-readable medium of claim 12, wherein the modification causes a related adjustment to one or more vehicle systems.

16. The non-transitory computer-readable medium of claim 12, wherein the vehicle occupant action data is based on an interaction between the vehicle occupant and a first vehicle system, and wherein the vehicle occupant action data triggers an adjustment of a second vehicle system.

17. The non-transitory computer-readable medium of claim 16, wherein the adjustment of the second vehicle system is based on the vehicle data.