System and method for dynamic in-vehicle virtual reality

• US 9,547,173 B2
• Filed: 02/11/2014
• Issued: 01/17/2017
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Alaniz, Arthur, Whinnery, Joseph, Murrish, Robert, Gleeson-May, Michael Eamonn
• Primary Examiner(s): Crawford, Jacinta M
• Assistant Examiner(s): ROBINSON, TERRELL M
• Application Number:
  US14/177,841
• Publication Number:
  US 20150097860A1
• Time in Patent Office: 1,071 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: B60R 2300/20 characterised by the type of display used
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A method for in-vehicle dynamic virtual reality includes receiving vehicle data from one or more vehicle systems of a vehicle, wherein the vehicle data includes vehicle dynamics data and receiving user data from a virtual reality device. The method includes generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data and the user data and rendering the virtual view to an output device by controlling the output device to update display of the virtual view according to the vehicle dynamics data.

Claims

1. A computer-implemented method for in-vehicle dynamic virtual reality, comprising:

receiving vehicle data from one or more vehicle systems of a vehicle, wherein the vehicle data includes vehicle dynamics data;

receiving user data from a head mounted display;

generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data and the user data; and

rendering the virtual view to the head mounted display by controlling the head mounted display to update display of the virtual view according to the vehicle dynamics data, wherein rendering the virtual view includes,

determining vehicle motion data, wherein the vehicle motion data is a direction of motion of the vehicle and a velocity of motion of the vehicle based on the vehicle dynamics data;

determining user motion data, wherein the user motion data is a direction of motion of the user relative to the vehicle and a velocity of motion of the user relative to the vehicle based on the user data and the vehicle data;

determining a temporal-motion rendering speed based on the vehicle motion data and the user motion data, wherein the temporal-motion rendering speed is determined as a correlation of differences between a motion, an orientation, and a location of the user relative to a motion, an orientation, and a location of the vehicle, wherein the motion, the orientation and the location of the user are determined based on the user data and the motion, the orientation, and the location of the vehicle is based on the vehicle data, and controlling the head mounted display to update display of the virtual view based on the temporal-motion rendering speed.

2. The method of claim 1, comprising determining the orientation and the location of the vehicle based on the vehicle data.

3. The method of claim 2, comprising determining the orientation and the location of the user relative to the vehicle based on the user data and the vehicle data.

4. The method of claim 3, wherein augmenting one or more components of the virtual world model is based on the orientation and the location of the vehicle and the orientation and the location of the user.

5. The method of claim 1, wherein determining the vehicle motion data includes determining predictive vehicle motion data, and determining the temporal-motion rendering speed is based on the vehicle motion data including the predictive vehicle motion data and the user motion data.

6. A vehicle computing system for in-vehicle dynamic virtual reality, comprising:

a data store storing a virtual world model, the virtual world model including one or more components that define a virtual view;

a processor operatively connected for computer communication to the data store, one or more vehicle systems of a vehicle and a virtual reality device the processor including;

a virtual reality data module receiving vehicle data from the one or more vehicle systems of the vehicle, wherein the vehicle data includes vehicle dynamics data, and receiving user data from the virtual reality device;

a dynamic virtual reality module generating the virtual view based on the vehicle data, the user data and the virtual world model from the data store, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data and the user data; and

a rendering module rendering the virtual view from the dynamic reality module to the virtual reality device by controlling the virtual reality device to update display of the virtual view according to vehicle motion data and user motion data, wherein the user motion data represents motion of a user relative to the vehicle and the rendering module determines the user motion data based on the user data and the vehicle dynamics data, wherein the rendering module determines a temporal-motion rendering speed as a correlation of differences between a motion, an orientation, and a location of the user relative to a motion, an orientation, and a location of the vehicle, wherein the motion, the orientation and the location of the user are determined based on the user data and the motion, the orientation, and the location of the vehicle is based on the vehicle data, and the temporal-motion rendering speed minimizes a difference between the vehicle motion data and the user motion data and the rendering module controls the virtual reality device to update display of the virtual view based on the temporal-motion rendering speed.

7. The system of claim 6, wherein the dynamic virtual reality module augments one or more properties of one or more of the components of the virtual world model based on the vehicle data and the user data.

8. The system of claim 7, wherein the one or more properties includes a motion property defining a motion of the component.

9. The system of claim 6, wherein the rendering module determines the vehicle motion data based on the vehicle dynamics data.

10. The system of claim 6, wherein the dynamic virtual reality module augments one or more properties of one or more of the components of the virtual world model based on the temporal-motion rendering speed.

11. A non-transitory computer-readable storage medium storing instructions that, when executed by a vehicle computer, causes the computer to perform the steps of:

receiving vehicle data from one or more vehicle systems of a vehicle, wherein the vehicle data includes vehicle dynamics data;

receiving user data from a head mounted display;

generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data and the user data;

determining a temporal-motion rendering speed as a correlation of differences between a motion, an orientation, and a location of the user relative to a motion, an orientation, and a location of the vehicle, wherein the motion, the orientation and the location of the user are determined based on the user data and the motion, the orientation, and the location of the vehicle is based on the vehicle; and

rendering the virtual view to the head mounted display by controlling the head mounted display to update display of the virtual view based on the temporal-motion rendering speed, wherein a rendering speed of the head mounted display is augmented based on the temporal-motion rendering speed.