Vulnerable road user protection system
First Claim
Patent Images
1. A method of processing an image of a visual scene, comprising:
- a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels;
b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image;
c. calculating a range histogram from said range map, wherein said range histogram accounts for every pixel of said at least one image, said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value;
d. associating each element of said range map counted by said element of said vector with at least one of a corresponding pixel of said plurality of pixels;
e. operating on said range histogram; and
f. operating on at least one of said at least one of said corresponding pixel of said plurality of pixels associated with said element of said range map responsive to the operation of operating on said range histogram.
6 Assignments
0 Petitions
Accused Products
Abstract
A range map of a visual scene generated by a stereo vision and associate image processing system, and is filtered to remove objects beyond a region of interest and for which a collision is not possible, and to remove an associated road surface. Objects clustered in range bins are separated by segmentation. A composite range map is generated using principale components analysis and processed with a connected-components sieve filter. Objects are identified using one or more of a harmonic profile and other features using an object recognition processor using a combination of inclusive, exclusive and harmonic networks to generate a classification metric.
-
Citations
41 Claims
-
1. A method of processing an image of a visual scene, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram accounts for every pixel of said at least one image, said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. associating each element of said range map counted by said element of said vector with at least one of a corresponding pixel of said plurality of pixels; e. operating on said range histogram; and f. operating on at least one of said at least one of said corresponding pixel of said plurality of pixels associated with said element of said range map responsive to the operation of operating on said range histogram. - View Dependent Claims (2)
-
-
3. A method of processing an image of a visual scene, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. associating each element of said range map counted by said element of said vector with at least one of a corresponding pixel of said plurality of pixels or said range map; e. operating on said range histogram; and f. operating on at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram; g. further comprising decreasing a resolution of said range map prior to calculating said range histogram by reducing a digital bit resolution of said range map prior to calculating said range histogram, so as to provide for smoothing said range map prior to calculating said range histogram.
-
-
4. A method of processing an image of a visual scene, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. associating each element of said range map counted by said element of said vector with at least one of a corresponding pixel of said plurality of pixels or said range map; e. operating on said range histogram; and f. operating on at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram, wherein the operation of operating on said range histogram comprises searching said range histogram for at least one object in said visual scene responsive to a corresponding at least one cluster of elements of said range histogram, and the operation of operating on said at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram comprises nulling or ignoring each pixel of said plurality of pixels not associated with said at least one object and converting each other pixel of said plurality of pixels associated with said at least one object to a uniform value indicative of an active pixel so as to form a binary segmentation image of an entirety of said at least one image or said range map. - View Dependent Claims (5, 6)
-
-
7. A method of processing an image of a visual scene, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. associating each element of said range map counted by said element of said vector with at least one of a corresponding pixel of said plurality of pixels or said range map; e. operating on said range histogram; and f. operating on at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram; g. further comprising generating a list that for each said element of said vector provides a location for each corresponding element of said range map contributing to said count of said elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value so as to facilitate the operation of operating on said at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram. - View Dependent Claims (8)
-
-
9. A method of processing an image of a visual scene comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. associating each element of said range map counted by said element of said vector with at least one of a corresponding pixel of said plurality of pixels or said range map; e. operating on said range histogram; and f. operating on at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram, wherein the operation of operating on said range histogram comprises searching said range histogram for at least one object in said visual scene responsive to a corresponding at least one cluster of elements of said range histogram, and the operation of operating on said at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram comprises nulling or ignoring each pixel of said plurality of pixels not associated with said at least one object and converting each other pixel of said plurality of pixels associated with said at least one object to a uniform value indicative of an active pixel so as to form a binary segmentation image of said at least one object; and g. iteratively isolating laterally-spaced attached objects in said visual scene from one another in said binary segmentation image of said at least one object by searching for a best-fit rectangle in a subset of said binary segmentation image of said at least one object, having an aspect ratio associated with a vulnerable road user (VRU). - View Dependent Claims (10, 11)
-
-
12. A method of processing an image of a visual scene, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. associating each element of said range map counted by said element of said vector with at least one of a corresponding pixel of said plurality of pixels or said range map; e. operating on said range histogram; and f. operating on at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram so as to provide for locating at least one object within said at least one image or within said range map; g. extracting at least one profile of at least one object from an associated boundary thereof within said at least one image or said range map, or a binary segmentation image thereof; h. determining a corresponding center of said at least one profile; i. determining a measurement vector comprising a plurality of distances from said corresponding center to said at least one profile at a plurality of angles relative to and through said corresponding center, wherein said plurality of angles is in excess of four; j. comparing said measurement vector with at least one stored vector representative of at least one type of reference object; and k. determining whether or not said at least one object corresponds to at least one said reference object responsive to the operation of comparing said measurement vector with said at least one stored vector. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. A method of processing an image of a visual scene, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. associating each element of said range map counted by said element of said vector with at least one of a corresponding pixel of said plurality of pixels or said map; e. operating on said range histogram; and f. operating on at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram so as to provide for locating at least one object within said at least one image or within said range map; g. extracting at least one profile of at least one object from an associated boundary thereof within said at least one image or said range map, or a binary segmentation image thereof; h. determining a corresponding center of said at least one profile; i. determining a measurement vector comprising a plurality of distances from said corresponding center to said at least one profile at a plurality of angles relative to and through said corresponding center, wherein each distance of said of plurality of distances is determined by first radially searching radially outwards along a search direction from said corresponding center until reaching a point on said at least one profile, and then calculating a corresponding distance of said plurality of distances, from said corresponding center to said point on said at least one profile; j. comparing said measurement vector with at least one stored vector representative of at least one type of reference object; and k. determining whether or not said at least one object corresponds to at least one said reference object responsive to the operation of comparing said measurement vector with said at least one stored vector; l. further comprising limiting the search along said search direction to a boundary of a best fit rectangle within said at least one image or said at least one image responsive thereto.
-
-
23. A method of processing an image of a visual scene,
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels: -
b. acquiring or determining range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map for which, said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. associating each element of said range map counted by said element said vector with at least one of a corresponding pixel of said plurality of pixels or said range map; e. operating on said range histogram; and f. operating on at least one of said at least one of said corresponding pixel of said plurality of pixels or said range map associated with said element of said range map responsive to the operation of operating on said range histogram so as to provide for locating at least one object within said at least one image or within said range map; g. extracting at least one profile of at least one object from an associated boundary thereof within said at least one image or said range map, or a binary segmentation image thereof; h. determining a corresponding center of said at least one profile; i. determining a measurement vector comprising a plurality of distances from said corresponding center to said at least one profile at a plurality of angles relative to and through said corresponding center; j. comparing said measurement vector with at least one stored vector representative of at least one type of reference object; and k. determining whether or not said at least one object corresponds to at least one said reference object responsive to the operation of comparing said measurement vector with said at least one stored vector; l. further comprising deriving said at least one stored vector from a plurality of associated measurement vectors from a corresponding plurality of different images of a corresponding said at least one said reference object, wherein the operation of deriving said at least one stored vector further comprises filtering the data from said plurality of associated measurement vectors with a central moving average filter prior to storing said at least one stored vector. - View Dependent Claims (24, 25, 26, 27, 28, 29)
-
-
30. A method of processing an image of a visual scene, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram accounts for every pixel of said at least one image, said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map, for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. forming at least one cluster of a plurality of associated range bins of said range histogram corresponding to at least one corresponding object in said visual scene; and e. generating a response responsive to said at least one cluster. - View Dependent Claims (31, 32, 33, 34, 35, 36, 37, 38)
-
-
39. A method of processing an image of a visual scene, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map, for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. forming at least one cluster of a plurality of associated range bins of said range histogram corresponding to at least one corresponding object in said visual scene; and e. generating a response responsive to said at least one cluster, wherein the operation of generating said response responsive to said at least one cluster comprises separating and isolating said at least one corresponding object in said visual scene from at least one other object in said visual scene responsive to range information associated with said at least one cluster, wherein the operation of separating and isolating said at least one corresponding object in said visual scene from said at least one other object in said visual scene is responsive to a composite range map generated from a plurality of sequential images of said visual scene.
-
-
40. A method of processing an image of a visual scene, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map, for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. forming at least one cluster of a plurality associated range bins of said range histogram corresponding to least one corresponding object in said visual scene; and
;e. generating a response responsive to said at least one cluster, wherein the operation of generating said response responsive to said at least one cluster cormprises separating and isolating said at least one corresponding object in said visual scene from at least one other object in said visual scene responsive to range information associated with said at least one cluster, wherein the operation of separating and isolating said at least one corresponding object in said visual scene from said at least one other object in said visual scene is responsive to a range spread of said at least one cluster, and said range spread of said at least one cluster is responsive to a difference between maximum and minimum ranges associated with said at least one cluster; further comprising; f. generating a list that for each said element of said vector provides a location for each corresponding element of said range map contributing to said count of said elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value so as to provide for isolating objects in said range map that have negligible differential range with respect to one another; and g. processing said at least one pixel of said range map with a connected-components sieve filter, h. wherein said connected-components sieve filter is adapted to locate at least one cluster of pixels or elements within said range map such that at least one isolated portion of said at least one cluster contains at least one pixel or element of said pixels or elements that is connected to at least one other pixel or element of said pixels or elements either along a same one of a first plurality of rows said range map, a same one of a second plurality of columns of said range map, or diagonally along both said same one of said first plurality of rows of said range map and said same one of said second pluralities of columns of said range map.
-
-
41. A method of processing an image of a visual scenes, comprising:
-
a. acquiring at least one image of a visual scene, wherein said at least one image comprises a plurality of pixels; b. acquiring or determining a range map comprising at least one distance value associated with at least one pixel of said plurality of pixels, wherein said at least one distance value comprises a measure of distance relative to an associated at least one camera used to acquire said at least one image; c. calculating a range histogram from said range map, wherein said range histogram comprises a vector, each element of said vector is associated with a different corresponding range of values of said at least one distance value, and each said element of said vector comprises a count of elements of said range map, for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value; d. forming at least one cluster of a plurality of associated range bins of said range histogram corresponding to least one corresponding object in said visual scene; and
;e. generating a response responsive to said at least one cluster;
further comprising;f. decreasing a resolution of said range map prior to calculating said range histogram; and g. generating a list that for each said element of said vector provides a location for each corresponding element of said range map contributing to said count of said elements of said range map for which said at least one distance value of said range map is within said corresponding range of values of said at least one distance value so as to provide for isolating objects in said range map that have negligible differential range with respect to one another.
-
Specification