System and method for dynamic in-vehicle virtual reality

• US 9,536,353 B2
• Filed: 07/10/2014
• Issued: 01/03/2017
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Alaniz, Arthur, Whinnery, Joseph, Murrish, Robert Wesley, Gleeson-May, Michael Eamonn
• Primary Examiner(s): Crawford, Jacinta M
• Assistant Examiner(s): ROBINSON, TERRELL M
• Application Number:
  US14/328,148
• Publication Number:
  US 20150097863A1
• Time in Patent Office: 908 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

A method for in-vehicle dynamic virtual reality, including receiving vehicle data from a portable device, the portable device operably connected for computer communication to an output device, the vehicle data including vehicle dynamics data, and receiving user data from at least one of the portable device or the output device. The method including generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data or the user data. The method including rendering the virtual view to the output device by controlling the output device to update display of the virtual view according to at least one of the vehicle data or the user data.

Claims

1. A computer-implemented method for in-vehicle dynamic virtual reality, comprising:

receiving vehicle data from a portable device, the portable device operably connected for computer communication to an output device, the portable device including position and motion sensors to obtain vehicle data independently from the vehicle, wherein the vehicle data is vehicle dynamics data of the vehicle and an orientation and a location of the vehicle;

receiving user data from at least one of the portable device or the output device, the user data including at least an orientation and a location of a user and determining an orientation and a location of the user relative to the vehicle based on the vehicle data and the user data;

generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to the orientation and the location of the vehicle and the orientation and the location of the user relative to the vehicle;

determining a temporal-motion rendering speed based on the vehicle data and the user data, wherein the temporal-motion rendering speed is a time-based correlation of differences between the orientation, the location and a motion of the user relative to the orientation, the location, and a motion of the vehicle, wherein the orientation, the location and the motion of the user are based on the user data and the orientation, the location, and the motion of the vehicle are based on the vehicle data; and

rendering the virtual view to the output device by controlling the output device to update display of the virtual view according to the temporal-motion rendering speed.

2. The computer-implemented method of claim 1, wherein the portable device is located within a motion frame of reference of the vehicle.

3. The computer-implemented method of claim 1, comprising determining vehicle motion data based on the vehicle dynamics data.

4. The computer-implemented method of claim 3, wherein the vehicle motion data comprises at least velocity and acceleration data.

5. The computer-implemented method of claim 3, comprising determining user motion data representing motion of the user relative to the vehicle based on the user data and the vehicle data.

6. The computer-implemented method of claim 5, wherein augmenting one or more components of the virtual world model is based on at least one of the user motion data and the vehicle motion data.

7. The computer-implemented method of claim 1, wherein one or more properties or one or more components of the virtual world model are augmented based on the temporal-motion rendering speed.

8. A system for in-vehicle dynamic virtual reality, comprising:

a portable device including position and motion sensors for gathering vehicle data about a vehicle independently from the vehicle, thereby providing a proxy for vehicle sensors of the vehicle, the portable device operably connected to an output device and a processor, the processor includes;

a virtual reality data module receives vehicle data from the portable device, wherein the vehicle data includes vehicle dynamics data of the vehicle, and receives user data from at least one of the portable device or the output device, wherein the user data includes an orientation and a location of a user and the virtual reality data module determines an orientation and a location of the user relative to the vehicle based on the vehicle data and the user data;

a dynamic virtual reality module generates the virtual view based on the vehicle data, the user data and a virtual world model, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data and the user data, the user data including the orientation and the location of the user relative to the vehicle; and

a rendering module determines a temporal-motion rendering speed based on the vehicle motion data and the user motion data and renders the virtual view from the dynamic reality module to the output device by controlling the output device to update display of the virtual view according to the temporal-motion rendering speed, wherein the temporal-motion rendering speed is a time-based correlation of differences between the orientation, the location and a motion of the user relative to the orientation, the location, and a motion of the vehicle, wherein the orientation, the location and the motion of the user are based on the user data and the orientation, the location, and the motion of the vehicle are based on the vehicle data.

9. The system of claim 8, wherein the portable device is attached to the vehicle in a location within a motion reference frame of the vehicle.

10. The system of claim 8, wherein the rendering module determines vehicle motion data based on the vehicle dynamics data.

11. The system of claim 10, wherein the rendering module determines user motion data representing motion of the user relative to the vehicle based on the user data and the vehicle data.

12. The system of claim 8, wherein the dynamic virtual reality module augments one or more properties of one or more of the components of the virtual world model based on the temporal-motion rendering speed.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by a vehicle computer, causes the computer to perform the steps of:

receiving vehicle data from a portable device, the portable device operably connected for computer communication to an output device, the vehicle data including vehicle dynamics data of the vehicle and an orientation and a location of the vehicle, wherein the portable device obtains the vehicle data independently from the vehicle, wherein the portable device includes position and motion sensors, and gathers the vehicle data from the sensors;

receiving user data from at least one of the portable device or the output device, the user data including at least orientation and location information of a user and determining an orientation and a location of the user relative to the vehicle based on the vehicle data and the user data;

generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to the orientation and the location of the vehicle and the orientation and the location of the user relative to the vehicle;

determining a temporal-motion rendering speed based on the vehicle data and the user data, wherein the temporal-motion rendering speed is a time-based correlation of differences between the orientation, the location and a motion of the user relative to the orientation, the location, and a motion of the vehicle, wherein the orientation, the location and the motion of the user are based on the user data and the orientation, the location, and the motion of the vehicle are based on the vehicle data; and

rendering the virtual view to the output device by controlling the output device to update display of the virtual view according to the temporal-motion rendering speed.

14. The non-transitory computer-readable storage medium of claim 13, wherein one or more properties or one or more components of the virtual world model are augmented based on the temporal-motion rendering speed.