Vehicle occupant head positioning system

• US 10,349,032 B2
• Filed: 09/30/2016
• Issued: 07/09/2019
• Est. Priority Date: 09/30/2016
• Status: Active Grant

• Current Assignee: Arriver Software LLC
• Original Assignee: Veoneer US Inc. (SSW Partners LP)
• Inventors: Jemander, Thorbjorn, Murray, Shane
• Primary Examiner(s): Hallenbeck-Huber, Jeremiah C
• Application Number:
  US15/282,329
• Publication Number:
  US 20180096475A1
• Time in Patent Office: 1,012 Days
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/54 Audio sensitive means, e.g. ultrasound
• CPC Class Codes: G01S 13/582 adapted for simultaneous range and velocity measurements
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 15/58 Velocity or trajectory determination systems; Sense-of-movement determination systems velocity measurement in imaging systems G01S15/8979
• CPC Class Codes: G01S 15/86 Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G08B 29/188 Data fusion; cooperative systems, e.g. voting among different detectors
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Described are occupant positioning systems, and methods of use thereof, which combine image capture and radar or ultrasonic sensors, determine the head position and/or velocity of a vehicle occupant'"'"'s head in three dimensions for use in a driver monitoring application. The driver monitoring applications may include features such as driver drowsiness estimation and indication, driver attention monitoring, driver gaze direction and driver gaze positioning, driver identification, head-up display adjustment and automatic sun blocking. These are features that can improve the operational safety of the vehicle.

Claims

1. In a motor vehicle having an occupant, a vehicle occupant head positioning system comprising:

wave sensing assembly comprising at least one transmitter for transmitting waves towards a head of an occupant in the vehicle, and at least one receiver for receiving waves reflected off the occupant head;

image capture assembly comprising at least one camera directed at the occupant head for capturing images, and at least one light emitting device for illuminating the occupant head when the images are being captured; and

one or more sensor fusion processor(s) configured to

process the waves received by the at least one receiver to compute the radial range with respect to the at least one receiver,

process the images captured by the at least one camera to determine an image position of the occupant head, and

determine from the radial range and the image position a three dimensional position of the occupant head by;

providing a head state model of a positional state of the occupant head;

determining a fused range based on the radial range and a range determined from frame-to-frame changes in image coordinates in the images captured by the image capture assembly;

converting the fused range and image coordinates of the captured images to three dimensional coordinates based on corresponding focal length(s) of the at least one camera;

determining a best match between the head state model and the three dimensional coordinates; and

updating the head state model with the three dimensional coordinates and an occupant head track position predicted by the head state model.

2. The vehicle occupant head positioning system of claim 1, wherein the wave sensing assembly transmits and receives radar or ultrasonic waves.

3. The vehicle occupant head positioning system of claim 1, further comprising corresponding attachment mechanisms for fixably securing the wave sensor assembly and image capture assembly to the vehicle.

4. The vehicle occupant head positioning system of claim 1, further comprising a driver monitoring system configured to receive the three dimensional position output from the sensor fusion assembly.

5. The vehicle occupant head positioning system of claim 4, wherein the driver monitoring system includes functionality selected from drowsiness estimation and alert, driver inattention to traffic alert, driver gaze direction estimation, driver gaze position estimation, driver identification, head-up display adjustment, and automatic sun-blocking.

6. The vehicle occupant head positioning system of claim 1, wherein the at least one illuminating element emits infrared light.

7. The vehicle occupant head positioning system of claim 6, wherein the at least one camera detects infrared light.

8. The vehicle occupant head positioning system of claim 1, wherein the wave sensor assembly computes at least one of azimuth or elevation of the occupant head.

9. The vehicle occupant head positioning system of claim 1, wherein supplemental measurements, in the form of optical flow from the image capture assembly, are used to feed a time filtering component to improve the accuracy or reliability of the position or velocity estimation.

10. The vehicle occupant head positioning system of claim 1, wherein supplemental measurements, in the form of Doppler information from the wave sensing assembly are used to feed a time filtering component to improve the accuracy or reliability of the position or velocity estimation.