Obstacle detection having enhanced classification

• US 20100013615A1
• Filed: 03/31/2005
• Published: 01/21/2010
• Est. Priority Date: 03/31/2004
• Status: Abandoned Application

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Hebert, Martial, Stentz, Anthony Joseph, Dima, Cristian Sergiu, Herman, Herman
• Application Number:
  US11/096,687
• Publication Number:
  US 20100013615A1
• Time in Patent Office: Days
• US Class Current: 340/425.500
• CPC Class Codes: B60Q 9/006 using a distance sensor
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A method and system for sensing an obstacle comprises transmitting an electromagnetic signal from a mobile machine to an object. A reflected electromagnetic signal is received from the object to determine a distance between the object and the mobile machine. An image patch is extracted from a region associated with the object. Each image patch comprises coordinates (e.g., three dimensional coordinates) associated with corresponding image data (e.g., pixels). If an object is present, image data may include at least one of object density data and object color data. Object density data is determined based on a statistical measure of variation associated with the image patch. Object color data based on the color of the object detected with brightness normalization. An object is classified or identified based on the determined object density and determined object color data.

Claims

1. A method for detecting an obstacle, the method comprising:

transmitting an electromagnetic signal from a vehicle to an object;

receiving a reflected signal from an observed point associated with the object to determine multidimensional coordinates of the observed point with respect to the vehicle or a reference point;

extracting an image patch from image data associated with the object and defined with reference to determined multidimensional coordinates;

determining an object density of the object based on a statistical measure of variation of observed points associated with the object;

determining observed color data based on an observed color of the object detected within the image patch; and

classifying the object based on the determined object density and determined object color data.

2. The method according to claim 1 wherein the determining of the observed color data comprises disregarding the brightness component, V, of the observed color data in a hue-saturation-value color space.

3. The method according to claim 1 wherein the determining of the observed color data comprises disregarding an intensity component, I, of the observed color data in a hue-saturation-intensity color space.

4. The method according to claim 1 wherein the determining of the observed color data comprises disregarding a lightness component, L, of the observed color data in a CIE LUV color space.

5. The method according to claim 1 wherein the determining of the observed color data comprises normalizing a red component, a green component, and blue component of the observed color data in red-green-blue color space.

6. The method according to claim 1 further comprising:

classifying an object as vegetation if the object density is less than a particular threshold and if the observed color data is indicative of a reference vegetation color.

7. The method according to claim 1 further comprising:

classifying an object as an animal if the object emits an infrared radiation pattern of an intensity, size and shape indicative of the presence of an animal.

8. The method according to claim 1 further comprising:

classifying the object as an animal if the object emits an infrared radiation pattern indicative of the presence of an animal and if the color data is indicative of an animal color, wherein reference animal colors are stored for comparison to the observed color data, the observed color data being compensated by discarding at least one of a brightness, lightness, or intensity component of a color space.

9. The method according to claim 1 further comprising:

classifying the object as a human being if the object emits an infrared radiation pattern indicative of the presence of a human being and if observed color data is indicative of flesh color or clothing colors.

10. The method according to claim 1 wherein the statistical measure comprises at least one of a standard deviation of a range or eigenvalues of a covariance matrix for the multidimensional coordinates associated with an object.

11. The method according to claim 1 further comprising:

estimating spatial location data associated with the object by averaging the determined multidimensional coordinates.

12. The method according to claim 1 further comprising:

establishing a traversability map in a horizontal plane associated with the vehicle, the map divided into a plurality of cells where each cell is indicative of whether or not the respective cell is traversable.

13. The method according to claim 1 further comprising:

establishing an obstacle map in a vertical plane associated with the vehicle, the map divided into a plurality of cells where each cell is indicative of whether or not the respective cell contains a certain classification of an obstacle or does not contain the certain classification of obstacle.

14. The method according to claim 13 wherein the classification comprises an obstacle selected from the group consisting of an animal, a human being, vegetation, grass, ground-cover, crop, man-made obstacle, machine, and tree trunk.

15. A system for sensing an obstacle, the system comprising:

a transmitter for transmitting an electromagnetic signal from a vehicle to an object;

a receiver for receiving a reflected signal from an observed point associated with the object to determine multidimensional coordinates of the observed point with respect to the vehicle or a reference point;

an image extractor for extracting an image patch in a region associated with the object and defined with reference to determined multidimensional coordinates;

a range assessment module for determining an object density of the object based on a statistical measure of variation associated with the image patch;

a color assessment module for determining object color data based on the color of the object detected with brightness normalization; and

a classifier for classifying the object based on the determined object density and determined object color data.

16. The system according to claim 15 wherein the color assessment module disregards a brightness component, V, of the observed color data in a hue-saturation-value color space.

17. The system according to claim 15 wherein the color assessment module disregards an intensity component, I, of the observed color data in a hue-saturation intensity color space.

18. The system according to claim 15 wherein the color assessment module disregards a lightness component, L, of the object color data in a CIE LUV color space.

19. The system according to claim 15 wherein the color assessment module normalizes a red component, a green component, and blue component of the object color data in red-green-blue color space.

20. The system according to claim 15 wherein the classifier classifies an object as vegetation if the object density is less than a particular threshold and if the color data is indicative of a vegetation color.

21. The system according to claim 15 wherein the infrared assessment module determines whether the object emits an infrared radiation pattern of at least one of an intensity, size, and shape indicative of animal or human life.

22. The system according to claim 15 wherein the classifier classifies an object as an animal if the object emits an infrared radiation pattern indicative of the presence of an animal.

23. The system according to claim 15 wherein the classifier classifies the object as an animal if the object emits an infrared radiation pattern indicative of the presence of an animal and if the color data is indicative of an animal color, wherein reference animal colors are stored for comparison to detected color data.

24. The system according to claim 15 wherein the classifier classifies the object as a human being if the object emits an infrared radiation pattern indicative of the presence of an animal and if the color data is indicative of flesh color or clothing colors, wherein reference human flesh colors, and reference clothing colors are stored for comparison to detected color data.

25. The system according to claim 15 wherein the statistical measure comprises a standard deviation of a range of eigenvalues of the covariance matrix for the multidimensional coordinates associated with an object.

26. The system according to claim 15 wherein the range assessment module estimates spatial location data associated with the object by averaging the determined multidimensional coordinates.

27. The system according to claim 15 further comprising a mapper for establishing a traversability map in a horizontal plane associated with the vehicle, the map divided into a plurality of cells where each cell is indicative of whether or not the respective cell is traversable.

28. The system according to claim 15 further comprising a mapper for establishing an obstacle map in a vertical plane associated with the vehicle, the map divided into a plurality of cells where each cell is indicative of whether or not the respective cell contains a certain classification of an obstacle or does not contain the certain classification of obstacle.

29. The system according to claim 28 wherein the classification comprises an obstacle selected from the group consisting of an animal, a human being, vegetation, grass, ground-cover, crop, man-made obstacle, machine, and tree trunk.