Alert generation correlating between head mounted imaging data and external device

• US 10,217,343 B2
• Filed: 11/09/2017
• Issued: 02/26/2019
• Est. Priority Date: 01/06/2014
• Status: Active Grant

• Current Assignee: iOnRoad Technologies Limited (Samsung Electronics Co. Ltd.)
• Original Assignee: iOnRoad Technologies Limited (Samsung Electronics Co. Ltd.)
• Inventors: Akiva, Ohad, Atsmon, Dan, Dayan, Ram
• Primary Examiner(s): Wu, Zhen Y
• Application Number:
  US15/808,782
• Publication Number:
  US 20180075727A1
• Time in Patent Office: 474 Days
• Field of Search: 340575
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: B60C 9/00 Reinforcements or ply arrangement of pneumatic tyres inserts having reinforcing means B60C5/08; bead structure, e.g. turnup or overlap construction, B60C15/00
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: B60W 2040/0872 Driver physiology
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2520/00 Input parameters relating to overall vehicle dynamics
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A method of alerting a vehicle'"'"'s driver. The method comprises receiving an azimuth value indicating a gazing direction of a head of the driver from a primary device placed on the driver, receiving a vehicle bearing value indicating a vehicle moving direction of the vehicle driven by the driver, calculating an angular difference between the azimuth value and the vehicle bearing value, generating an alert when the angular difference deviates from an angular threshold, and automatically presenting the alert to the driver in response to the angular difference.

Claims

1. A method of alerting a vehicle'"'"'s driver, comprising:

receiving a first set of image data capturing images from a driver'"'"'s perspective, the first set of image data captured by at least one image sensor coupled to a primary device placed on a driver and directed outwardly from the driver;

receiving a second set of image data, the second set of image data captured by a second device mounted on a vehicle driven by the driver capturing images from a perspective directed to a movement direction of the vehicle;

processing the first and second sets of image data to determine a comparison result, the comparison result determined according to image similarity between the first and second sets of image data by comparing objects captured in images of the first set of image data and objects captured in images of the second set of image data;

generating an alert according to the comparison result when the image similarity deviates from an image similarity threshold; and

automatically presenting the alert to the driver.

2. The method of claim 1, wherein the generating is performed when an angular difference deviates from an angular threshold, the angular difference is between an azimuth value indicating a horizontal orientation facing direction of a head of the driver and a vehicle bearing value indicating a current vehicle moving direction of the vehicle driven by the driver.

3. The method of claim 2, wherein the horizontal orientation facing direction is calculated based on head movements of the driver measured by the primary device.

4. The method of claim 2, wherein the angular threshold selected from a plurality of predefined angular thresholds according to a traffic condition.

5. The method of claim 2, wherein the generating further comprises generating the alert when the angular difference deviates from the angular threshold and the image similarity deviates from the image similarity threshold;

and

automatically presenting the alert to the driver.

6. The method of claim 2, further comprising:

receiving a driver activity notification from the second device, the driver activity notification indicating whether the driver is interacting with the second device;

generating the alert when the driver activity notification indicates that the driver is interacting with the second device and the angular difference deviates from the angular threshold; and

automatically presenting the alert to the driver.

7. The method of claim 3, wherein the angular threshold is dynamically modified based on data received.

8. The method of claim 2, wherein the vehicle bearing value is generated by a geographical positioning system (GPS);

and

wherein the vehicle bearing value is transmitted by one or more of a circuit coupled to the vehicle and a mobile device to the primary device placed on the driver.

9. The method of claim 2, further comprising:

automatically presenting the alert to the driver in response to the angular difference while the angular difference is still present.

10. The method of claim 1, further comprising receiving three-dimensional orientation information from the primary device, the three-dimensional orientation information indicating a horizontal orientation facing direction of the head of the driver, wherein the generating is performed when the horizontal orientation facing direction is determined to be different from a current vehicle moving direction.

11. The method of claim 1, wherein the second device is oriented toward a current vehicle moving direction of the vehicle.

21. The method of claim 1, wherein the comparing is conducted by an image processing mechanism that is a member of the group consisting of image subtraction, use of correlation algorithms, image registrations, and point mapping registrations.

12. A system of alerting a driver of a vehicle, comprising:

a wearable mount configured to be placed on a head of the driver for measuring head movements of the driver;

at least one first image sensor coupled to the wearable mount, the first image sensor directing outwardly from the driver to capture images from a driver'"'"'s perspective;

at least one second image sensor coupled to the vehicle and capturing images from a perspective directed to a movement direction of the vehicle;

a processor configured to execute code instructions comprising;

code instructions for receiving a first set of image data captured by the first image sensor;

code instructions for receiving a second set of image data captured by the second image sensor;

code instructions for processing the first and second sets of image data to determine a comparison result, the comparison result determined according to image similarity between the first and second sets of image data by comparing objects captured in images of the first set of image data and objects captured in images of the second set of image data;

code instructions for generating an alert according to the comparison result; and

code instructions for automatically presenting the alert to the driver.

13. The system of claim 12, wherein the generating is performed when an angular difference deviates from an angular threshold, the angular difference is between an azimuth value indicating a horizontal orientation facing direction of a head of the driver and a vehicle bearing value indicating a current vehicle moving direction of a vehicle driven by the driver.

14. The system of claim 12, wherein the generating is performed when the comparison result deviates from an image similarity threshold.

15. The system of claim 12, wherein the code instructions for generating the alert further comprise:

code instructions for receiving a plurality of primary device orientation values over a period of time;

code instructions for determining an angular velocity of the head of the driver by utilizing the plurality of primary device orientation values;

wherein the generating is performed when the angular velocity deviates from an angular velocity threshold and the comparison result deviates from an angular threshold.

16. The system of claim 12, wherein the alert is presented as a sound using a mobile phone.

17. The system of claim 12, wherein:

the wearable mount is a pair of glasses with an electronic display; and

the code instructions further comprise code instructions for presenting the alert on the electronic display.

18. The system of claim 13, wherein:

the wearable mount is a pair of glasses with an electronic display and a GPS; and

the code instructions further comprise code instructions for receiving the vehicle bearing value from the GPS.

19. The system of claim 16, wherein the processor is configured to execute code instructions for instructing the vehicle to gain driving control according to the comparison result.

20. The system of claim 16, wherein the processor is configured to execute code instructions for receiving an additional set of image data and processing the additional set of image data to determine an eyes gazing direction result;

and

wherein the generating is performed when the eyes gazing direction result deviates from an eyes gazing direction threshold.