Method of processing an image of a visual scene

• US 9,330,321 B2
• Filed: 10/22/2013
• Issued: 05/03/2016
• Est. Priority Date: 07/26/2004
• Status: Active Grant

• Current Assignee: Joyson Safety Systems Acquisition, LLC
• Original Assignee: TK Holdings Incorporated (Ningbo Junsheng Investment Group Co., Ltd.)
• Inventors: Schamp, Gregory G., Demro, James C., Davies, Owen A.
• Primary Examiner(s): Bali, Vikkram
• Application Number:
  US14/059,838
• Publication Number:
  US 20140044310A1
• Time in Patent Office: 924 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B60R 21/0134 responsive to imminent contact with an obstacle , e.g. using radar systems
• CPC Class Codes: B60R 21/34 Protecting non-occupants of a vehicle, e.g. pedestrians B60R19/02 takes precedence
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An image of a visual scene, comprising a plurality of pixels. is acquired and an associated range map is either determined therefrom or separately acquired. Elements of the range map comprise distances from the camera for each pixel of the image. In one aspect, either the image or the range map is processed with a connected-components sieve filter that locates clusters of pixels or elements that are connected to one another along either adjacent rows, columns or diagonally. In another aspect, a cross-range value of range-map element is compared with a down-range-responsive cross-range threshold of a boundary of a collision-possible space and the pixel or element is nulled or ignored if associated with a location that is not in the collision-possible space. The collision-possible space is responsive to an operating condition of a vehicle from which the image is acquired.

Claims

1. A method of processing an image of a visual scene, comprising:

a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels, and said plurality of pixels are arranged in a first array of pixels comprising a first plurality of rows and a second plurality of columns;

b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; and

c. processing said at least one pixel of at least one of said at least one image or said range map with a connected-components sieve filter, wherein said connected-components sieve filter is adapted to locate at least one cluster of pixels or elements within said at least one of said at least one image or said range map such that said at least one cluster of pixels or elements contains one or more said pixels or elements that are connected to at least one other pixel or element of said pixels or elements either along a same one of said first plurality of rows of said at least one of said at least one image or said range map, a same one of said second plurality of columns of said at least one of said at least one image or said range map, or diagonally adjacent along both said first plurality of rows of said at least one of said at least one image or said range map and said second plurality of columns of said at least one of said at least one image or said range map; and

d. comparing a count of pixels or elements contained within at least one said cluster of pixels or elements with a threshold, and if said count of pixels or elements contained within said at least one said cluster of pixels or elements is less than said threshold, then nulling or ignoring said pixels or elements within said at least one said cluster of pixels or elements.

2. A method of processing an image of a visual scene as recited in claim 1, further comprising comparing said at least one distance value of said range map with at least one threshold for at least one said pixel, and operating on said at least one pixel of said at least one of said at least one image or said range map responsive to the operation of comparing said at least one distance value with said at least one threshold.

3. A method of processing an image of a visual scene as recited in claim 2, further comprising nulling said at least one pixel if said at least one distance value exceeds a first threshold.

4. A method of processing an image of a visual scene as recited in claim 3, wherein said at least one distance value comprises at least one of a range value or a down-range value.

5. A method of processing an image of a visual scene as recited in claim 2, further comprising nulling said at least one pixel if said at least one distance value is less than a second threshold.

6. A method of processing an image of a visual scene as recited in claim 5, wherein said at least one distance value comprises a height value.

7. A method of processing an image of a visual scene as recited in claim 1, wherein said at least one image of said visual scene comprises at least one pair of stereo images of said visual scene, and the operation of acquiring or determining said range map comprises determining said range map from said at least one pair of stereo images of said visual scene.

8. A method of processing an image of a visual scene as recited in claim 1, further comprising compressing a bit length of said at least one distance value associated with said at least one pixel of said plurality of pixels.

9. A method of processing an image of a visual scene, comprising:

a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels;

b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image, and said at least one distance value comprises a down-range value and a cross-range value;

c. comparing said cross-range value associated with said at least one pixel with a cross-range threshold representative of a boundary of a collision-possible space, wherein said cross-range threshold representative of said boundary of said collision-possible space is responsive to said down-range value; and

d. nulling or ignoring said at least one pixel or a corresponding element of said range map if the operation of comparing said cross-range value with said cross-range threshold indicates that said at least one pixel is associated with a location that is not in said collision-possible space, wherein said collision-possible space is responsive to an operating condition of a vehicle from which said at least one image is acquired.

10. A method of processing an image of a visual scene as recited in claim 9, wherein said collision-possible space is responsive to a velocity of said vehicle.

11. A method of processing an image of a visual scene as recited in claim 9, wherein said collision-possible space is responsive to at least one of a steering angle or a turn rate of said vehicle.

12. A method of processing an image of a visual scene as recited in claim 9, wherein said collision-possible space is shaped responsive to a projected path of said vehicle.

13. A method of processing an image of a visual scene as recited in claim 9, wherein said at least one image of said visual scene comprises at least one pair of stereo images of said visual scene, and the operation of acquiring or determining said range map comprises determining said range map from said at least one pair of stereo images of said visual scene.

14. A method of processing an image of a visual scene as recited in claim 9, further comprising:

a. tracking at least one object in said at least one image of said visual scene;

b. estimating a velocity of said at least one object responsive to the operation of tracking said at least one object; and

c. nulling or ignoring either at least one said pixel corresponding to said at least one object or nulling or ignoring a corresponding element of said range map if said velocity of said at least one object exceeds a corresponding velocity threshold, or if responsive to the operation of tracking, said at least one object is predicted to not enter said collision possible space.

15. A method of processing an image of a visual scene as recited in claim 14, further comprising determining a type of said at least one object, wherein said corresponding velocity threshold is responsive to said type of said at least one object.

16. A method of processing an image of a visual scene as recited in claim 14, wherein said corresponding velocity threshold comprises a maximum possible velocity of a vulnerable road user (VRU).