Vehicular imaging system with camera misalignment correction and capturing image data at different resolution levels dependent on distance to object in field of view

• US 9,018,577 B2
• Filed: 02/25/2013
• Issued: 04/28/2015
• Est. Priority Date: 08/17/2007
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Lu, Yuesheng, Higgins-Luthman, Michael J., Jeyaraj, Antony V., Phirke, Manoj R.
• Primary Examiner(s): Le, Que T
• Assistant Examiner(s): BENNETT, JENNIFER D
• Application Number:
  US13/776,094
• Publication Number:
  US 20130169812A1
• Time in Patent Office: 792 Days
• Field of Search: 250/208.1, 250/214.1, 250/214.R, 250/206, 250/559.4, 250/559.1, 250/559.29, 250/559.3, 250/222.1, 250/221, 340/937, 340/936, 340/939, 340/934, 340/938, 340/540, 340/436, 340/437, 340/440, 340/441, 340463-467, 340/435, 348/118, 348/148, 348/143, 348/294, 382/104, 382/106, 382/274, 382/266, 356/3.13, 356/4.01, 356/3.01, 356/139.04, 356/139.09, 356/399, 356/400
• US Class Current: 250/221
• CPC Class Codes: B60Q 1/085 due to special conditions, e.g. adverse weather, type of road, badly illuminated road signs or potential dangers B60Q1/10, B60Q1/12, B60Q1/1423 take precedence
• CPC Class Codes: B60Q 1/1423 Automatic dimming circuits, i.e. switching between high beam and low beam due to change of ambient light or light level in road traffic
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A vehicular imaging system comprises a single imaging sensor having a plurality of photosensor elements. The imaging sensor captures image data of a scene exterior of the vehicle within its field of view. Image data captured by the imaging sensor is provided to a control. The control also receives via a communication bus at least one of vehicle pitch information, vehicle yaw information and vehicle steering information. The control detects two substantially straight and converging road features along the road the vehicle is travelling and determines a point of intersection where they would converge using, at least in part, the received vehicle information and the control automatically corrects for misalignment of the imaging sensor mounted at the vehicle.

Claims

1. A vehicular imaging system comprising:

an imaging device comprising a single imaging sensor;

said imaging sensor comprising a plurality of photosensor elements;

said imaging device mounted at a vehicle and having a field of view exterior of said vehicle;

wherein said field of view exterior of said vehicle encompasses a scene forward of said vehicle;

wherein said imaging device is disposed in said vehicle behind the windshield of said vehicle and views through a portion of the windshield;

wherein said portion of the windshield comprises a portion cleaned by a windshield wiper of said vehicle;

wherein said imaging device comprises a CMOS imaging device;

said single imaging sensor of said imaging device capturing image data within said field of view;

a control receiving said image data;

said control also receiving, via a communication bus of said vehicle, at least one of (i) vehicle pitch information, (ii) vehicle yaw information and (iii) vehicle steering information;

said control detecting converging road features along the road said vehicle is travelling;

said control determining a point of intersection where said converging road features would converge;

said control automatically correcting for misalignment of said imaging device mounted at said vehicle;

wherein said automatic correction for misalignment of said imaging device mounted at said vehicle comprises, at least in part, said determination by said control of where said converging road features would converge;

wherein at least one of (i) said control is operable to process a reduced image data set more than other image data and (ii) said control detects an object present exteriorly of said vehicle within a target zone;

wherein said control comprises an image processor, said image processor processing captured image data at at least two resolution levels, said at least two resolution levels comprising a higher resolution level of processing of captured image data by said image processor and a lower resolution level of processing of captured image data by said image processor;

wherein said image processor processes captured image data at different resolution levels at least in part in response to a distance between the subject vehicle and a detected light source present in said field of view; and

wherein said image processor processes captured image data at said lower resolution level responsive to the distance being below a threshold level and wherein said image processor processes captured image data at said higher resolution level responsive to the distance being above the threshold level.

2. The vehicular imaging system of claim 1, wherein said control automatically compensates for an initial misalignment.

3. The vehicular imaging system of claim 1, wherein said control corrects for at least one of (i) a temporary vehicle condition and (ii) a reinstallation misalignment.

4. The vehicular imaging system of claim 1, wherein said control is operable to detect that at least a portion of said photosensor array is blocked.

5. The vehicular imaging system of claim 1, wherein said control is operable to automatically correct for misalignment of up to at least approximately +/−

4 degrees from an aligned condition.

6. The vehicular imaging system of claim 1, wherein said control compares a location of an object detected exteriorly of said vehicle to an expected location of the object and, wherein, based at least in part on said comparison, said control corrects for misalignment of said imaging device.

7. The vehicular imaging system of claim 1, wherein at least one of (i) said photosensor array comprises a cell phone imager and (ii) said converging road features comprise converging lane markers.

8. The vehicular imaging system of claim 1, wherein at least one of (i) said control is operable to selectively process data from said photosensor array with at least two different resolutions and (ii) said control is operable to encode signals into the light output of a light of said vehicle to allow an imaging system of another vehicle to detect said encoded signal.

9. The vehicular imaging system of claim 1, wherein said control is operable to detect and ameliorate a bad photosensor element of said plurality of photosensor elements.

10. The vehicular imaging system of claim 1, wherein at least one of (i) said control utilizes image data captured while said vehicle travels in a curve, (ii) said control determines the driving side of the road said vehicle is travelling on, (iii) said control monitors the trajectory of said vehicle and (iv) said control monitors the trajectory of said vehicle in order to confirm the road said vehicle is travelling on is at least one of substantially flat and substantially straight.

11. A vehicular imaging system comprising:

an imaging device comprising a single imaging sensor;

said imaging sensor comprising a plurality of photosensor elements;

said imaging device mounted at a vehicle and having a field of view exterior of said vehicle;

wherein said field of view exterior of said vehicle encompasses a scene forward of said vehicle;

wherein said imaging device is disposed in said vehicle behind the windshield of said vehicle and views through a portion of the windshield;

wherein said portion of the windshield comprises a portion cleaned by a windshield wiper of said vehicle;

wherein said imaging device comprises a CMOS imaging device;

said single imaging sensor of said imaging device capturing image data within said field of view;

a control receiving said image data;

said control also receiving, via a communication bus of said vehicle, at least one of (i) vehicle pitch information, (ii) vehicle yaw information and (iii) vehicle steering information;

said control detecting converging road features along the road said vehicle is travelling;

said control determining a point of intersection where said converging road features would converge;

said control automatically correcting for misalignment of said imaging device mounted at said vehicle;

wherein said automatic correction for misalignment of said imaging device mounted at said vehicle comprises, at least in part, said determination by said control of where said converging road features would converge;

wherein said control corrects for at least one of (i) a temporary vehicle condition and (ii) a reinstallation misalignment;

wherein at least one of (i) said control utilizes image data captured while said vehicle travels in a curve, (ii) said control determines the driving side of the road said vehicle is travelling on, (iii) said control monitors the trajectory of said vehicle and (iv) said control monitors the trajectory of said vehicle in order to confirm the road said vehicle is travelling on is at least one of substantially flat and substantially straight;

wherein said control comprises an image processor, said image processor processing captured image data at at least two resolution levels, said at least two resolution levels comprising a higher resolution level of processing of captured image data by said image processor and a lower resolution level of processing of captured image data by said image processor;

wherein said image processor processes captured image data at different resolution levels in response to a distance between the subject vehicle and a detected light source present in said field of view; and

wherein said image processor processes captured image data at said lower resolution level responsive to the distance being below a threshold level and wherein said image processor processes captured image data at said higher resolution level responsive to the distance being above the threshold level.

12. The vehicular imaging system of claim 11, wherein at least one of (i) said control is operable to process a reduced image data set more than other image data and (ii) said control detects an object present exteriorly of said vehicle within a target zone.

13. The vehicular imaging system of claim 11, wherein said control compares a location of an object detected exteriorly of said vehicle to an expected location of the object and, wherein, based at least in part on said comparison, said control corrects for misalignment of said imaging device.

14. The vehicular imaging system of claim 11, wherein at least one of (i) said photosensor array comprises a cell phone imager, (ii) said converging road features comprise converging lane markers and (iii) said control is operable to automatically correct for misalignment of up to at least approximately +/−

4 degrees from an aligned condition.

15. A vehicular imaging system comprising:

an imaging device comprising a single imaging sensor;

said imaging sensor comprising a plurality of photosensor elements;

said imaging device mounted at a vehicle and having a field of view exterior of said vehicle;

wherein said field of view exterior of said vehicle encompasses a scene forward of said vehicle;

wherein said imaging device is disposed in said vehicle behind the windshield of said vehicle and views through a portion of the windshield;

wherein said portion of the windshield comprises a portion cleaned by a windshield wiper of said vehicle;

wherein said imaging device comprises a CMOS imaging device;

said single imaging sensor of said imaging device capturing image data within said field of view;

a control receiving said image data;

said control also receiving, via a communication bus of said vehicle, at least one of (i) vehicle pitch information, (ii) vehicle yaw information and (iii) vehicle steering information;

said control detecting converging road features along the road said vehicle is travelling;

said control determining a point of intersection where said converging road features would converge;

said control automatically correcting for misalignment of said imaging device mounted at said vehicle;

wherein said automatic correction for misalignment of said imaging device mounted at said vehicle comprises, at least in part, said determination by said control of where said converging road features would converge;

wherein at least one of (a) said photosensor array comprises a cell phone imager, (b) said converging road features comprise converging lane markers, (c) said control corrects for a temporary vehicle condition, (d) said control corrects for a reinstallation misalignment and (e) said control compares a location of an object detected exteriorly of said vehicle to an expected location of the object and, wherein, based at least in part on said comparison, said control corrects for misalignment of said imaging device.

wherein said control comprises an image processor, said image processor processing captured image data at at least two resolution levels, said at least two resolution levels comprising a higher resolution level of processing of captured image data by said image processor and a lower resolution level of processing of captured image data by said image processor;

wherein said image processor processes captured image data at different resolution levels in response to a distance between the subject vehicle and a detected light source present in said field of view; and

wherein said image processor processes captured image data at said lower resolution level responsive to the distance being below a threshold level and wherein said image processor processes captured image data at said higher resolution level responsive to the distance being above the threshold level.

16. The vehicular imaging system of claim 15, wherein at least one of (i) said control is operable to process a reduced image data set more than other image data and (ii) said control detects an object present exteriorly of said vehicle within a target zone.

17. The vehicular imaging system of claim 15, wherein said control is operable to automatically correct for misalignment of at least approximately +/−

4 degrees from an aligned condition.

18. The vehicular imaging system of claim 15, wherein said control is operable to detect and ameliorate a bad photosensor element of said plurality of photosensor elements.

19. The vehicular imaging system of claim 15, wherein at least one of (i) said control utilizes image data captured while said vehicle travels in a curve, (ii) said control determines the driving side of the road said vehicle is travelling on, (iii) said control monitors the trajectory of said vehicle and (iv) said control monitors the trajectory of said vehicle in order to confirm the road said vehicle is travelling on is at least one of substantially flat and substantially straight.

20. The vehicular imaging system of claim 15, wherein at least one of (i) said control is operable to selectively process data from said photosensor array with at least two different resolutions and (ii) said control is operable to encode signals into the light output of a light of said vehicle to allow an imaging system of another vehicle to detect said encoded signal.