Blind-spot monitoring using machine vision and precise FOV information

• US 9,931,985 B1
• Filed: 05/07/2015
• Issued: 04/03/2018
• Est. Priority Date: 05/07/2015
• Status: Active Grant

• Current Assignee: Ambarella International LP (Ambarella Incorporated)
• Original Assignee: Ambarella Incorporated
• Inventors: Pertsel, Shimon, Fink, Alexander
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Noh, Jae N
• Application Number:
  US14/706,222
• Time in Patent Office: 1,062 Days
• Field of Search: 348148
• CPC Class Codes: B60R 1/081 avoiding blind spots, e.g. by using a side-by-side association of mirrors
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/202 displaying a blind spot scene on the vehicle part responsible for the blind spot
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/8026 in addition to a rear-view mirror system
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An apparatus comprising a first sensor, an interface, a second sensor and a processor. The first sensor may be configured to generate a video signal based on a targeted view of a driver. The interface may be configured to receive status information about one or more components of a vehicle. The second sensor may be configured to detect an object within a predetermined radius of the second sensor. The processor may be configured to determine a field of view of the driver based on the status information. The processor may be configured to generate a control signal in response to a cross reference between (i) a field of view of the driver and (ii) the detected object. The control signal may be used to alert the driver if the detected object is not in the field of view of the driver.

Claims

1. An apparatus comprising:

a first sensor configured to generate a video signal based on a targeted view of a driver;

an interface configured to receive status information about one or more components of a vehicle;

a second sensor configured to generate a proximity signal in response to detecting an object within a predetermined radius of said second sensor; and

a processor configured to

(A) determine a location of said object with respect to said vehicle in response to said proximity signal,

(B) calculate current location coordinates of eyes of said driver detected in said video signal,

(C) determine a field of view of said driver at a time when said proximity signal is received based on (i) said status information and (ii) said current location coordinates of said eyes,

(D) perform a cross reference between (i) said field of view of said driver and (ii) said location of said object from said proximity signal, and

(E) generate a control signal, wherein (i) said cross reference is configured to determine whether said object is within said field of view using said current location coordinates of said eyes at said time said proximity signal is received, (ii) said current location coordinates of said eyes are determined based on an analysis of one or more video frames of said video signal to determine (a) one or more of a first location coordinate and a second location coordinate of said eyes and (b) a depth coordinate representing a distance of said eyes from said first sensor, (iii) said depth coordinate is determined based on a comparison of a reference number of pixels of a vehicle component in a reference video frame to a current number of pixels of said vehicle component in said video frames, (iv) said vehicle component is capable of relative movement with respect to said first sensor and (v) said control signal is used to alert said driver when said location of said object is not in said field of view of said driver.

2. The apparatus according to claim 1, wherein said first sensor is implemented as at least one of (i) a component of a camera and (ii) a video capturing device.

3. The apparatus according to claim 2, wherein said second sensor is implemented as at least one of a radar device, an array of radars, a sonar, an array of sonars, a lidar device, an array of lidar devices, an ultra-sound device and an array of ultra-sound devices.

4. The apparatus according to claim 2, wherein said second sensor is implemented as at least one of a single camera and a camera surround view system.

5. The apparatus according to claim 1, wherein said status information comprises an orientation of one or more mirrors of said vehicle.

6. The apparatus according to claim 1, wherein said status information comprises a position of a seat of said driver.

7. The apparatus according to claim 6, wherein said position of said seat of said driver comprises a height, depth, recline and an angle of a bottom cushion of said seat.

8. The apparatus according to claim 1, wherein said field of view of said driver is determined based on (i) data from said video signal and (ii) a distance of said driver from said first sensor.

9. The apparatus according to claim 1, wherein said field of view of said driver comprises one of a number of fields of view, each corresponding to a respective number of mirrors of said vehicle.

10. The apparatus according to claim 1, wherein said field of view of said driver comprises a forward view of said driver.

11. The apparatus according to claim 1, wherein said video signal comprises a plurality of digital images.

12. The apparatus according to claim 1, wherein (i) said object is another vehicle and (ii) said apparatus implements blind-spot monitoring based on said field of view.

13. The apparatus according to claim 1, wherein a represented version of at least one of said detected object and said field of view are presented in said vehicle.

14. The apparatus according to claim 13, wherein said represented version is presented on a display unit of said vehicle.

15. The apparatus according to claim 13, wherein said presentation of said represented version is configured to display said detected object outside said field of view in a first color and said detected object inside said field of view in a second color.

16. The apparatus according to claim 13, wherein said presentation of said represented version is configured to implement visual effects corresponding to (i) a velocity of said detected object and (ii) a status of said detected object.

17. The apparatus according to claim 1, wherein (i) said processor is further configured to determine an intention of said driver and (ii) said generation of said control signal is based on said intention of said driver.

18. The apparatus according to claim 1, wherein said processor is further configured to determine if said location of said object is in an actual blind-spot of said driver.

19. The apparatus according to claim 1, wherein said processor is further configured to generate said control signal when said object is not present in said field of view corresponding to a visibility of said driver through a mirror of said vehicle.