System and method for dynamic in-vehicle virtual reality

• US 9,599,819 B2
• Filed: 05/30/2014
• Issued: 03/21/2017
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Alaniz, Arthur, Whinnery, Joseph, Murrish, Robert Wesley, Gleeson-May, Michael Eamonn
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): ROBINSON, TERRELL M
• Application Number:
  US14/291,854
• Publication Number:
  US 20150097861A1
• Time in Patent Office: 1,026 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: B60R 2300/20 characterised by the type of display used
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A method for in-vehicle dynamic virtual reality includes receiving vehicle data from one or more vehicle systems of a vehicle, wherein the vehicle data includes vehicle dynamics data and receiving user data from a virtual reality device. The method includes generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data and the user data and rendering the virtual view to an output device by controlling the output device to update display of the virtual view according to the vehicle dynamics data.

Claims

1. A computer-implemented method for in-vehicle dynamic virtual reality, comprising:

receiving vehicle data from one or more vehicle systems of a vehicle, wherein the vehicle data includes vehicle dynamics data of the vehicle and an orientation and a location of the vehicle;

receiving user data from a portable device, the user data including at least an orientation and a location of a user and determining an orientation and a location of the user relative to the vehicle based on the vehicle data and the user data;

generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to the orientation and the location of the vehicle and the orientation and the location of the user relative to the vehicle;

determining a temporal-motion rendering speed based on the vehicle data and the user data, wherein the temporal-motion rendering speed is a correlation of differences between a motion, an orientation, and a location of the user relative to a motion, an orientation, and a location of the vehicle, wherein the motion, the orientation and the location of the user are determined based on the user data and the motion, the orientation, and the location of the vehicle is based on the vehicle data, thereby simulating real-time motion of the vehicle in the virtual view; and

rendering the virtual view to an output device by controlling the output device to update display of the virtual view according to the temporal-motion rendering speed.

2. The computer-implemented method of claim 1, including determining vehicle motion data based on the vehicle dynamics data.

3. The computer-implemented method of claim 2, including determining user motion data representing motion of the user relative to the vehicle based on the user data and the vehicle data.

4. The computer-implemented method of claim 3, wherein augmenting one or more components of the virtual world model is based on at least one of the user motion data and the vehicle motion data.

5. The computer-implemented method of claim 3, wherein determining the temporal-motion rendering speed is based on the vehicle motion data and the user motion data.

6. The computer-implemented method of claim 1, wherein the output device is the portable device.

7. The computer-implemented method of claim 1, wherein the virtual world model includes a group of node classes, the group of node classes including a view node class, a world structure node class, an eventsln node class, and an eventsOut node class, wherein generating the virtual view includes augmenting one or more properties of at least one node in the group of node classes according to the vehicle dynamics data, the orientation and the location of the vehicle, and the orientation and the location of the user relative to the vehicle.

8. The computer-implemented method of claim 1, wherein the user data includes health data of the user and further including modifying the temporal-motion rendering speed based on the health data.

9. A computing system for in-vehicle dynamic virtual reality, comprising:

a processor operatively connected for computer communication to one or more vehicle systems of a vehicle and a portable device, the processor including;

a virtual reality data module receives vehicle data from the one or more vehicle systems of the vehicle, wherein the vehicle data includes vehicle dynamics data of the vehicle, and receives user data from the portable device, the user data including at least orientation and location information about a user in possession of the portable device;

a dynamic virtual reality module generates the virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data and the user data; and

a rendering module determines user motion data representing motion of the user relative to a framework of the vehicle and relative to a speed of the vehicle, determines vehicle motion data based on the vehicle dynamics data, and determines a temporal-motion rendering speed based on the user motion data and the vehicle motion data, wherein the temporal-motion rendering speed is a correlation of differences between a motion, an orientation, and a location of the user relative to a motion, an orientation, and a location of the vehicle, wherein the motion, the orientation and the location of the user are determined based on the user data and the motion, the orientation, and the location of the vehicle is based on the vehicle data, wherein the rendering module, renders the virtual view from the dynamic reality module to an output device by controlling the output device to update display of the virtual view according to the temporal-motion rendering speed thereby minimizing a difference between the vehicle motion data and the user motion data.

10. The system of claim 9, wherein the dynamic virtual reality module augments one or more properties of one or more of the components of the virtual world model based on the vehicle data and the user data.

11. The system of claim 10, wherein the one or more properties includes a motion property defining a motion of the component.

12. The system of claim 9, wherein the dynamic virtual reality module augments one or more properties of one or more of the components of the virtual world model based on the temporal-motion rendering speed.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by a vehicle computer, causes the computer to perform the steps comprising of:

receiving vehicle data from one or more vehicle systems of a vehicle, wherein the vehicle data includes vehicle dynamics data of the vehicle and an orientation and a location of the vehicle;

receiving user data from a portable device, the user data including at least orientation and location information of a user and determining an orientation and a location of the user relative to the vehicle based on the vehicle data and the user data;

generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to the orientation and the location of the vehicle and the orientation and the location of the user relative to the vehicle;

determining a temporal-motion rendering speed based on the vehicle dynamics data and the user data, wherein the temporal-motion rendering speed is a correlation of differences between a motion, an orientation, and a location of the user relative to a motion, an orientation, and a location of the vehicle, wherein the motion, the orientation and the location of the user are determined based on the user data and the motion, the orientation, and the location of the vehicle is based on the vehicle data; and

rendering the virtual view to an output device by controlling the output device to update display of the virtual view according to the vehicle dynamics data and controlling the output device to update display of the virtual view based on the temporal-motion rendering speed.