HEAD MOUNTED DISPLAY PRESENTATION ADJUSTMENT

• US 20150294505A1
• Filed: 04/13/2015
• Published: 10/15/2015
• Est. Priority Date: 04/14/2014
• Status: Active Grant

• Current Assignee: iOnRoad Technologies Limited (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Atsmon, Dan
• Granted Patent: US 9,928,653 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Embodiments are disclosed for adjusting a presentation on a head-mounted display (HMD). In one or more example embodiments, a method of dynamically orienting a presentation of a HMD includes gathering HMD sensor data via at least one HMD sensor that is installed on an HMD worn by a driver of the vehicle and gathering vehicle sensor data via at least one vehicle mounted sensor mounted to the vehicle. The example method further includes performing an analysis of the HMD sensor data and of the vehicle sensor data to identify a difference between the HMD sensor data and the vehicle sensor data, and calculating, based on the difference, an orientation of the HMD device in relation to the vehicle. The method further includes adjusting a presentation of data on a display of the HMD device based on the orientation.

Claims

1. A method of dynamically orienting a presentation of a head-mounted display (HMD), comprising:

gathering HMD sensor data via at least one HMD sensor installed on an HMD worn by a driver of a vehicle;

gathering vehicle sensor data via at least one vehicle-mounted sensor mounted on the vehicle;

performing an analysis of said HMD sensor data and of said vehicle sensor data to identify a difference between said HMD sensor data and said vehicle sensor data;

calculating, based on said difference, an orientation of said HMD device in relation to said vehicle; and

adjusting a presentation of data on a display of said HMD device based on said orientation.

2. The method of claim 1, wherein the HMD sensor data comprises an HMD image and the vehicle sensor data comprises a vehicle image.

3. The method of claim 2, wherein the HMD image includes a driver control area and a scenery seen through at least part of a windshield of the vehicle, wherein the vehicle image includes at least a portion of the scenery imaged from a perspective of the vehicle sensor, and wherein identifying the difference between the HMD sensor data and the vehicle sensor data comprises identifying a difference in an apparent position of an object located in the scenery and included in each of the HMD image and the vehicle image.

4. The method of claim 2, wherein the HMD image includes a driver control area comprising a set of size-known markers arranged in a predetermined arrangement, and where performing the analysis of the HMD sensor data comprises identifying an apparent size of each member of the set of size-known markers in the HMD image.

5. The method of claim 4, wherein identifying the apparent position and size of each member of the set of size-known markers comprises detecting the apparent position and size of each member of the set of size-known markers using a Scale-invariant feature transform (SIFT) process.

6. The method of claim 2, further comprising combining the HMD image and the vehicle image to create a stereoscopic image.

7. The method of claim 1, wherein the HMD sensor data comprises a current measured HMD device motion determined via at least one HMD device motion detector, wherein the vehicle sensor data comprises a current measured vehicle motion determined via at least one vehicle motion detector, and wherein performing an analysis of the HMD sensor data and the vehicle sensor data comprises identifying a difference between the current HMD device motion and the current vehicle motion.

8. The method of claim 7, further comprising capturing an HMD image, via at least one HMD image sensor installed on the HMD, of a driver control area and a scenery seen through part of a windshield of the vehicle, capturing a vehicle image, via at least one vehicle imaging device mounted on the vehicle, of at least a portion of the scenery, wherein performing an analysis of the HMD sensor data and the vehicle sensor data further comprises identifying a difference between an apparent position of an object located in the scenery and imaged in both the HMD image and the vehicle image, and wherein calculating the orientation is further based on the apparent position.

9. The method of claim 7, further comprising capturing an HMD image, via at least one HMD image sensor installed on the HMD, of a driver control area comprising a set of size-known markers arranged in a predetermined arrangement, wherein performing an analysis of the HMD sensor data further comprises identifying an apparent position and size of each member of the set of size-known markers in the HMD image, and wherein calculating the orientation is further based on the apparent position.

10. A system for dynamically orienting a presentation of a head-mounted display (HMD), comprising:

at least one HMD image sensor which is installed on a HMD worn by a driver of a vehicle that captures a HMD image of a driver control area and a part of a windshield and a scenery seen through said part of said windshield and in front of said vehicle, the scenery including an object having a first apparent position from a perspective of the at least one HMD image sensor;

at least one device image sensor of an imaging device mounted on a base fixated to said vehicle that captures a vehicle image of said scenery, the object in the scenery having a second apparent position from a perspective of the at least one device image sensor; and

an analysis and calculation module comprising instructions executable by a processor of the HMD to;

perform an analysis of said HMD image and of said vehicle image to identify a difference in the first and second apparent positions of the object located in said scenery and depicted both in said HMD image and in said vehicle image;

calculate, based on said difference, an orientation of said HMD device in relation to said vehicle; and

adjust a presentation of data on a display of said HMD device based on said orientation.

11. The system of claim 10, wherein adjusting the presentation of data on the display of the HMD device based on the orientation comprises moving a location of a virtual object by an amount based on the difference between the first and second apparent positions of the object responsive to determining that the data on the display comprises a virtual object tied to a real-world object inside of the vehicle.

12. The system of claim 10, wherein the HMD image includes a set of size-known markers arranged in a predetermined arrangement, and where performing the analysis of the HMD image comprises identifying an apparent position and size of each member of the set of size-known markers in the HMD image.

13. The system of claim 12, wherein identifying the apparent position and size of each member of the set of size-known markers comprises detecting the apparent position and size of each member of the set of size-known markers using a Scale-invariant feature transform (SIFT) process.

14. The system of claim 10, further comprising at least one HMD device motion detector configured to determine a current HMD device motion and at least one vehicle motion detector configured to determine a current vehicle motion, and wherein the instructions are further executable to calculate the orientation of the HMD device relative to the vehicle based on a difference between the current HMD device motion and the current vehicle motion.

15. The system of claim 10, wherein the instructions are further executable to combine the HMD image and the vehicle image to create a stereoscopic image.

16. A system for dynamically orienting a presentation of a head-mounted display (HMD), comprising:

a driver control area comprising a set of size known markers arranged in a known arrangement;

at least one HMD image sensor which is installed on a HMD worn by a driver of a vehicle that captures a HMD image of a driver control area and a part of a windshield and a scenery seen through said part of said windshield and in front of said vehicle; and

captures a marking image of said a driver control area;

at least one HMD device motion detector installed in said HMD device that measures a current HMD device motion;

at least one device image sensor of an imaging device mounted on a base fixated to said vehicle that captures a vehicle image of said scenery;

at least one vehicle motion detector which is installed in said vehicle that measures a current vehicle motion; and

an analysis and calculation module comprising instructions executable by a processor of at least one of the HMD and an in-vehicle computing system to;

perform an analysis to identify;

a difference in apparent position of an object located in said scenery and depicted both in said HMD image and in said vehicle image;

a difference between said current vehicle motion and said current HMD device motion; and

an apparent position and size of each member of said set of size known markers in said marking image;

calculate, based on said differences and said apparent position, an orientation of said HMD device in relation to said vehicle; and

adjust a presentation of data on a display of said HMD device based on said orientation.

17. The system of claim 16, wherein adjusting the presentation of data on the display of the HMD device based on the orientation comprises moving a location of a virtual object by an amount based on the differences between the apparent position of the object and the current vehicle and HMD device motion responsive to determining that the data on the display comprises a virtual object tied to a real-world object inside of the vehicle.

18. The system of claim 16, wherein the HMD image includes a set of size-known markers arranged in a predetermined arrangement, and where performing the analysis comprises identifying an apparent position and size of each member of the set of size-known markers in the HMD image.

19. The system of claim 18, wherein identifying the apparent position and size of each member of the set of size-known markers comprises detecting the apparent position and size of each member of the set of size-known markers using a Scale-invariant feature transform (SIFT) process.

20. The system of claim 16, wherein the instructions are further executable to combine the HMD image and the vehicle image to create a stereoscopic image.