MM-wave radar driver fatigue sensor apparatus

• US 9,862,271 B2
• Filed: 04/15/2015
• Issued: 01/09/2018
• Est. Priority Date: 04/14/2014
• Status: Active Grant

• Current Assignee: Novelic D.O.O.
• Original Assignee: Novelic D.O.O.
• Inventors: Brankovi, Veselin, Gruji, Du{hacek over (s)}an, Jovanovi, Pavle, Mihajlovi, Veljko, Savi, Milan, Tasovac, Darko
• Primary Examiner(s): PHAM, TOAN NGOC
• Application Number:
  US15/303,237
• Publication Number:
  US 20170036541A1
• Time in Patent Office: 1,000 Days
• Field of Search: 340575, 3405731
• CPC Class Codes: A61B 2503/22 Motor vehicles operators, e.g. drivers, pilots, captains
• CPC Class Codes: A61B 2562/0228 Microwave sensors
• CPC Class Codes: A61B 5/0507 using microwaves or terahertz waves
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: A61B 5/7225 Details of analog processing, e.g. isolation amplifier, gain or sensitivity adjustment, filtering, baseline or drift compensation A61B5/30, A61B5/316 take precedence; transformers for specific applications H01F38/00; amplifiers H03F
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: G01S 7/415 Identification of targets based on measurements of movement associated with the target
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: H01Q 1/2283 mounted in or on the surface of a semiconductor substrate as a chip-type antenna or integrated with other components into an IC package
• CPC Class Codes: H01Q 1/3233 particular used as part of a sensor or in a security system, e.g. for automotive radar, navigation systems
• CPC Class Codes: H01Q 1/3291 mounted in or on other locations inside the vehicle or vehicle body
• CPC Class Codes: H01Q 21/0006 Particular feeding systems
• CPC Class Codes: H01Q 9/285 Planar dipole H01Q9/065 takes precedence; patch antenna H01Q9/0407

Abstract

The present invention discloses a mm-wave radar sensor to be deployed in the vehicles for sensing driver fatigue. The key system relevant components are utilization of mm-wave integrated radar, with specific planar high gain antenna radiation pattern, by analyzing at least two major biometric parameters of the drives simultaneously: heartbeat and respiratory dynamics. The method of operation calculates probability of the fatigue event. In case that probability is above a predefined threshold, the interaction with vehicle control system is initiated, using typical arbitrary automotive interfaces. Corresponding predefined actions are taken in that case. The predefined actions could be one or combination of the following: driver safety belt pulling, audio signal alerts to driver, vibration alert to driver, inside cabin light condition changes, engine operation condition change, corresponding communication using arbitrary wireless means to outside vehicle environment. Optionally, the system is utilizing additional driver imposed parameters like acceleration sensor information. Preferably, the system is using 60 GHz or 77-79 GHz integrated radar front end working in Doppler operation mode, with 4×4 Tx and Rx planar radiation elements, with physical size typically in the range 4×2×1 cm, or smaller.

Claims

1. Mm-wave Radar Driver Fatigue Sensor Apparatus and Method of Operation, where mm-wave declares operation between 30 and 300 GHz, including:

high-gain planar antenna for transmitting mm-wave radio signals, where the high-gain planar antenna has at least two radiation elements;

high-gain planar antenna for receiving mm-wave radio signals, where the high-gain planar antenna has at least two radiation elements;

integrated mm-wave radio front end, implemented in arbitrary semiconductor technology, having on-chip integrated mm-wave voltage control oscillator, mm-wave power amplifier, mm-wave low noise amplifier, mm-wave mixer, digital control interface, power supply; and

PLL analog to digital conversion entity;

digital processing functionality including controlling functionality and calculation and memory capacity for performing digital signal processing by arbitrary type of the realization options;

optional interface to vehicle infrastructure, including one or more standardized automotive wired interfaces;

supporting circuitry, including mechanical interface to vehicle infrastructure and supporting electronic circuitry for power supply of apparatus;

where the apparatus is integrated in a vehicle steering wheel, facing a driver, with direct line-of-sight operation and where Method of Operation includes;

transmission of mm-wave signals generated in integrated mm-wave radio front end using high-gain planar antenna for transmitting mm-wave radio signals;

receiving mm-wave signals reflected from driver body using high-gain planar antenna for receiving mm-wave radio signals;

amplification of the reflected signal in integrated mm-wave radio front end;

down-conversion of the signals by mixing with the same signal of the same frequency as the transmitted signal in integrated mm-wave radio front end;

amplification of a converted signal after mixer in integrated mm-wave radio front end;

analog filtering of the signals after amplification in integrated mm-wave radio front end;

signal conditioning in integrated mm-wave radio front end for subsequent analog to digital conversion performed by analog to digital conversion entity;

digital processing of the signal in digital processing functionality, by;

extracting a heartbeat rate from the previous arbitrary processed signal;

extracting a rate of change of the heartbeat rate from the previous arbitrary processed signal;

extracting a respiratory rate from the previous arbitrary processed signal;

extracting a rate of change of the respiratory rate from the previous arbitrary processed signal;

digital processing in driver fatigue event decision functionality which includes the following steps;

evaluation if the heartbeat rate is within a specified range;

evaluation if the respiratory rate is within a specified range;

evaluation if the rate of change of the heartbeat rate is within the specified range;

evaluation if the rate of change of the respiratory rate is within the specified range;

statistical evaluation of a driver heartbeat rate data history;

statistical evaluation of a driver respiratory rate data history;

time information entity which provides information on continuous driving duration, total driving duration in last period of specified duration and current local time information;

provision of a current heartbeat rate by the entity of evaluation if the heartbeat rate being within the specified range and the current rate of change of the heartbeat rate by the entity of evaluation if the rate of change of the heartbeat rate being within the specified range to driver statistic heartbeat rate model entity of the statistical evaluation of the driver heartbeat rate data history;

provision of a current respiratory rate by the entity of evaluation if the respiratory rate being within the specified range and the current rate of change of the respiratory rate by the entity of evaluation if the rate of change of the respiratory rate being within the specified range to driver statistic respiratory rate model entity of the statistical evaluation of the driver respiratory rate data history;

digital processing in Driver fatigue event calculation decision functionality is performed, which;

calculates a score by processing the information provided through entities of evaluation if the heartbeat rate is within the specified range;

evaluation if the respiratory rate is within the specified range;

evaluation if the rate of change of the heartbeat rate is within specified range;

evaluation if the rate of change of the respiratory rate is within specified range;

statistical evaluation of the driver heartbeat rate data history statistical evaluation of the driver respiratory rate data history and time information entity weighted by a specified coefficients, where the score is related to a probability of the driver fatigue event;

in case that the calculated score is above predefined threshold, decision on positive driver fatigue event is made;

in case of a positive driver fatigue event the entity of digital processing in Driver fatigue event calculation decision functionality sends the decision information and the corresponding score to the entity of evaluation if the respiratory rate being within the specified range;

in case of the positive driver fatigue event the entity of evaluation if the respiratory rate being within the specified range initiates appropriate specified actions of the entity of providing interface to vehicle infrastructure by using typical vehicle wired interfaces and/or entity of containing acceleration sensors and gyroscopes.

2. System according to claim 1, in which apparatus is integrated in a vehicle chassis above the driver'"'"'s visual field, facing the driver, with direct line-of-sight operation.

3. System according to claim 1, in which further comprises entity of providing information about vehicle dynamics to driver fatigue event calculation entity, and where this information influences driver fatigue event score calculation in method of operation, in case that the reduction of vehicle speed is detected by MEMS based acceleration sensors.

4. System according to claim 1, in which further comprises entity provides of providing information about vehicle dynamics to driver fatigue event calculation entity, and where this information influences driver fatigue event score calculation in Method of operation, in case repeatable corrections of a vehicle direction are detected by MEMS based acceleration sensors.

5. System a according to claim 1, in which optional information from vehicle cabin gas sensor is provided to driver fatigue event calculation entity, and where this information influences driver fatigue event score calculation in method of operation, in case that increased CO₂ concentration is detected.

6. System according to claim 1, in which the supporting circuitry contains audio and/or visual alerting capabilities of arbitrary realization, which are activated in case the driver fatigue event is detected by the entity of initiating predefined actions.

7. System according to claim 1, in which the sensor apparatus and method of operation has only one high gain antenna for transmitting and receiving mm-wave radio signals, where the high-gain planar antenna has at least two radiation elements and isolator functionality being released by plurality of the realization option, providing isolation between Rx and Tx chains, and related Rx and Tx connection to high gain antenna.

8. System according to claim 1, in which the sensor apparatus and method of operation has entity of integrated mm-wave radio front end having instead of mm-wave down conversion mixer, an IQ Demodulator, and two signal conditioning chains instead of one.