Vehicle vision system with image classification

• US 9,619,716 B2
• Filed: 08/11/2014
• Issued: 04/11/2017
• Est. Priority Date: 08/12/2013
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Pflug, Goerg
• Primary Examiner(s): PARK, SOO JIN
• Application Number:
  US14/456,164
• Publication Number:
  US 20150042808A1
• Time in Patent Office: 974 Days
• Field of Search: None
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G06V 10/85 Markov-related models; Markov random fields
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/586 of parking space

Abstract

A vision system of a vehicle includes a camera disposed at a vehicle and having a field of view exterior of the vehicle. The camera includes an imaging array having a plurality of photosensing elements arranged in a two dimensional array of rows and columns. The imaging array includes a plurality of sub-arrays comprising respective groupings of neighboring photosensing elements. An image processor is operable to perform a discrete cosine transformation of captured image data, and a Markov model compares at least one sub-array with a neighboring sub-array. The image processor is operable to adjust a classification of a sub-array responsive at least in part to the discrete cosine transformation and the Markov model.

Claims

1. A vision system of a vehicle, said vision system comprising:

a camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises an imaging array having a plurality of photosensing elements arranged in a two dimensional array of rows and columns;

wherein said imaging array comprises a plurality of sub-arrays comprising respective groupings of neighboring photosensing elements;

an image processor operable to process image data captured by said camera;

wherein said image processor is operable to perform a discrete cosine transformation of captured image data;

wherein a result of the discrete cosine transformation is put into a K-Means classifier;

wherein a Markov model compares at least one sub-array with a neighboring sub-array; and

wherein said image processor is operable to adjust a classification of a sub-array responsive at least in part to the discrete cosine transformation and said Markov model.

2. The vision system of claim 1, wherein said image processor determines a probability that a sub-array is sensing a portion of a scene that belongs to a particular class, and wherein, responsive to the probability determination, said image processor assigns a classification to that sub-array.

3. The vision system of claim 2, wherein said Markov model compares the assigned classification of at least one sub-array with the classification of a neighboring sub-array.

4. The vision system of claim 2, wherein the particular class comprises one of (i) nature, (ii) manmade, (iii) street, (iv) sky, (v) threshold between manmade and street and (vi) sidewalk.

5. The vision system of claim 2, wherein a plausibilification algorithm is applied subsequently to raise the accuracy level of the classification.

6. The vision system of claim 5, wherein said plausibilification algorithm applies plausible rules to diminish falsely classified segments.

7. The vision system of claim 6, wherein said plausible rules include at least one of (i) a ‘

street’

classification cannot be applied to a surface above the level of the horizon and (ii) a ‘

street’

classification cannot be surrounded by portions classified as ‘

sky’

.

8. The vision system of claim 1, wherein said Markov model comprises three sequential stages.

9. The vision system of claim 8, wherein each of said stages of said Markov model compares the sub-arrays consecutively.

10. The vision system of claim 9, wherein the classification of each sub-array is an input parameter to a processing of a neighboring sub-array.

11. The vision system of claim 10, wherein said Markov model compares each sub-array to neighboring sub-arrays to determine whether the classification of each sub-array matches the classification of neighboring sub-arrays.

12. The vision system of claim 11, wherein said Markov model is operable to change a classification of a sub-array responsive to determining a classification that best matches the sub-array'"'"'s originally assigned classification and a neighboring sub-array'"'"'s classification.

13. A vision system of a vehicle, said vision system comprising:

a camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises an imaging array having a plurality of photosensing elements arranged in a two dimensional array of rows and columns;

wherein said imaging array comprises a plurality of sub-arrays comprising respective groupings of neighboring photosensing elements;

an image processor operable to process image data captured by said camera;

wherein said image processor is operable to perform a discrete cosine transformation of captured image data;

wherein said image processor determines a probability that a sub-array is sensing a portion of a scene that belongs to a particular class, and wherein, responsive to the probability determination, said image processor assigns a classification to that sub-array;

wherein a Markov model compares the classification of at least one sub-array with a classification of a neighboring sub-array;

wherein said image processor is operable to adjust a classification of a sub-array responsive at least in part to the discrete cosine transformation and said Markov model; and

wherein said Markov model is operable to change a classification of a sub-array responsive to determining a new classification that best matches the sub-array'"'"'s originally assigned classification and a neighboring sub-array'"'"'s classification.

14. The vision system of claim 13, wherein the particular class comprises one of (i) nature, (ii) manmade, (iii) street, (iv) sky, (v) threshold between manmade and street and (vi) sidewalk.

15. The vision system of claim 13, wherein a plausibilification algorithm is applied subsequently to raise the accuracy level of the classification, and wherein said plausibilification algorithm applies plausible rules to diminish falsely classified segments, wherein said plausible rules include at least one of (i) a ‘

street’

classification cannot be applied to a surface above the level of the horizon and (ii) a ‘

street’

classification cannot be surrounded by portions classified as ‘

sky’

.

16. The vision system of claim 13, wherein said Markov model comprises three sequential stages, and wherein each of said stages of said Markov model compares the sub-arrays consecutively, and wherein the classification of each sub-array is an input parameter to a processing of a neighboring sub-array.

17. A vision system of a vehicle, said vision system comprising:

a camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises an imaging array having a plurality of photosensing elements arranged in a two dimensional array of rows and columns;

wherein said imaging array comprises a plurality of sub-arrays comprising respective groupings of neighboring photosensing elements;

an image processor operable to process image data captured by said camera;

wherein said image processor is operable to perform a discrete cosine transformation of captured image data;

wherein said image processor determines a classification for the portion of the scene imaged by a sub-array and assigns that classification to that sub-array;

wherein a Markov model compares the classification of at least one sub-array with a classification of a neighboring sub-array;

wherein said image processor is operable to adjust a classification of a sub-array responsive at least in part to the discrete cosine transformation and said Markov model;

wherein said Markov model compares each sub-array to neighboring sub-arrays to determine whether the classification of each sub-array matches the classification of neighboring sub-arrays; and

wherein said Markov model is operable to change a classification of a sub-array responsive to determining a new classification that best matches the sub-array'"'"'s originally assigned classification and a neighboring sub-array'"'"'s classification.

18. The vision system of claim 17, wherein the particular classification comprises one of (i) nature, (ii) manmade, (iii) street, (iv) sky, (v) threshold between manmade and street and (vi) sidewalk.

19. The vision system of claim 17, wherein a plausibilification algorithm is applied subsequently to raise the accuracy level of the classification, and wherein said plausibilification algorithm applies plausible rules to diminish falsely classified segments, wherein said plausible rules include at least one of (i) a ‘

street’

classification cannot be applied to a surface above the level of the horizon and (ii) a ‘

street’

classification cannot be surrounded by portions classified as ‘

sky’

.

20. The vision system of claim 17, wherein said Markov model comprises three sequential stages, and wherein each of said stages of said Markov model compares the sub-arrays consecutively, and wherein the classification of each sub-array is an input parameter to a processing of a neighboring sub-array.