Vehicle-based imaging system function diagnosis and validation

• US 8,605,949 B2
• Filed: 11/30/2011
• Issued: 12/10/2013
• Est. Priority Date: 11/30/2011
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zhang, Wende, Litkouhi, Bakhtiar Brian
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): BEG, SAMAH A
• Application Number:
  US13/307,236
• Publication Number:
  US 20130136309A1
• Time in Patent Office: 741 Days
• Field of Search: None
• US Class Current: 382/104
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 2207/30264 Parking
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method of determining functionality of a vision-based imaging system for a vehicle includes capturing images by the vision-based imaging system. The images include landmarks that are identified in multiple images for distinguishing displacement of landmarks between the images. Edge maps for each of the images are generated. Like regions of at least two edge maps are compared for distinguishing displacement of a landmark between the at least two edge maps. Each pixel location of a first edge map is compared with a same pixel location of a second edge map for determining whether each compared pixel location has a different intensity value. An aggregate number of pixel locations having a different intensity value between the at least two edge maps is determined and compared to a threshold. An error message is generated in response to the aggregate number being less than the threshold.

Claims

1. A method of determining functional validity of a vision-based imaging system for a vehicle, the method comprising the steps of:

capturing successive images by the vision-based imaging system mounted to a vehicle, the images including landmarks that are identified in multiple images for distinguishing displacement of landmarks between the images;

generating an edge map for each of the images by a processing module;

comparing like regions of at least two edge maps for distinguishing displacement of a landmark between the at least two edge maps, each edge map being constructed of pixels having varying intensity values, wherein each pixel location of a first edge map is compared with a same pixel location of a second edge map, wherein each of the same pixel locations of the at least two edge maps are compared within one another for determining whether each compared pixel location has a different intensity value; and

determining an aggregate number of pixel locations having a different intensity value between the at least two edge maps;

determining whether the aggregate number is greater than a threshold for indicating displacement of the landmarks between the at least two edge maps;

generating an error message to a driver in response to the aggregate number being less than the threshold;

validating a functionality of the vision-based imaging system in response to the aggregate number being greater than a threshold; and

generating an error message by an output device in response to determining invalidity of the functionality of the vision-based imaging system.

2. The method of claim 1 wherein generating an error message in response to the aggregate number being less than the threshold comprises:

determining whether the vehicle is moving in response to the aggregate number being less than the threshold; and

generating an error message by an output device only in response to the determination that the vehicle is moving.

3. The method of claim 2 wherein the error message indicates that an imaging signal transmitted within vision-based imaging system is paused.

4. The method of claim 2 wherein the error message indicates that an image captured by the vision-based imaging system is blocked by an object.

5. The method of claim 1 wherein validating a functionality of the vision-based imaging system includes validating an identification of a lane marker of the road captured by the vision-based imaging system.

6. The method of claim 5 wherein validation of the vision-based imaging system comprises:

determining whether the vehicle is moving;

identifying a lane marker by the vision-based imaging system in response to a determination that the vehicle is moving;

comparing the identified lane marker in at least two of the images;

determining a characteristic of the lane marker in the first image for comparison with the characteristic of the lane marker in the second image; and

invalidating the functionality of the vision-based imaging system in response to the characteristic of the lane marker of the first image not matching the characteristic of the lane marker of the second image.

7. The method of claim 6 wherein the characteristic of the lane marker includes a position of the lane marker within the images.

8. The method of claim 6 wherein the characteristic of the lane marker includes an orientation of the lane marker within the images.

9. The method of claim 5 wherein validation of the vision-based imaging system further comprises:

identifying a lane marker by the vision-based imaging system;

identifying the lane marker by a secondary sensing system of the vehicle;

comparing the lane marker captured by the vision-based imaging system with the lane marker captured by the secondary sensing system; and

invalidating vision-based imaging system in response to the lane marker captured by the vision-based imaging system not matching the lane marker captured by the secondary sensing system.

10. The method of claim 9 wherein the vision-based imaging system includes a rear vision-based imaging device, and wherein identifying the lane marker by a secondary sensing system includes identifying the lane marker using a forward vision-based imaging device.

11. The method of claim 9 wherein the vision-based imaging system includes a rear vision-based imaging device, and wherein identifying the lane marker by a secondary sensing system includes identifying the lane marker using a side view vision-based imaging device.

12. The method of claim 10 wherein identifying the lane marker by a secondary sensing system includes identifying the lane marker using a global positioning-based system.

13. The method of claim 10 wherein identifying the lane marker by a secondary sensing system includes identifying the lane marker using a lidar-based sensing system.

14. The method of claim 1 wherein the error message indicates that the vision-based imaging system is uncalibrated.

15. The method of claim 1 wherein the error message indicates that a video connection is incorrectly connected.

16. The method of claim 1 wherein the error message indicates that the vision-based imaging system is inconsistent.

17. The method of claim 1 wherein generating an error message by an output device includes generating an audio message identifying the vision-based imaging system error.

18. The method of claim 1 wherein generating an error message by an output device includes generating a visual message identifying the vision-based imaging system error.

19. The method of claim 18 wherein the visual message identifying the vision-based imaging system error is displayed on a vehicle vision-based imaging output display screen.

20. The method of claim 1 wherein generating an error message by an output device includes generating a haptic response for identifying the vision-based imaging system error.