Machine vision based obstacle avoidance system

• US 8,675,060 B2
• Filed: 12/21/2009
• Issued: 03/18/2014
• Est. Priority Date: 08/28/2009
• Status: Expired due to Fees

• Current Assignee: Indian Institute of Science
• Original Assignee: Indian Institute of Science
• Inventors: Ghose, Debasish, Kompella, Varun Raj
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): MCINNISH, KEVIN K
• Application Number:
  US12/643,207
• Publication Number:
  US 20110050883A1
• Time in Patent Office: 1,548 Days
• Field of Search: 348/26, 348/28, 348/61, 348/62, 348/113, 348135-142, 382/265, 382/274, 382162-169, 382/100, 382/114, 382106-107, 382/286, 382/294, 382/299, 382/300, 382/308
• US Class Current: 348/113
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Machine vision based obstacle avoidance system is provided. The system utilizes a CCD camera to capture an image. A normalized image and dynamic masking is used in object detection.

Claims

1. A mobile platform comprising:

a base structure;

a drive mechanism coupled to the base structure;

a machine vision system including;

an optical input module configured to capture an image in the form of a signal;

a pre-processing module configured to;

receive the signal from the optical input module, convert the signal into a first digital image,

set a first threshold comprising;

I_L=max{i|f(i−

δ

)>

f(i), f(i+δ

)>

f(i)}

set a second threshold comprising;

I_R=min{j|f(j−

δ

)>

f(j), f(j+δ

)>

f(j)}

wherein i and j are each an intensity, f(i) is a number of pixels having the intensity i found from an intensity histogram, and δ

is a resolution parameter;

invert the first digital image;

an image normalizing module configured to normalize the inverted digital image by;

setting a first plurality of pixels in the inverted digital image having an intensity less than the first threshold or greater than the second threshold to a first predetermined intensity, and

setting a second plurality of pixels in the inverted digital image having an intensity greater than or equal to the first threshold or less than or equal to the second threshold to a second predetermined intensity;

an edge detection module configured to;

receive the normalized digital image, and perform edge detection on the normalized digital image, thereby outputting a bounding graph indicating edges of detected objects in the image; and

a detection module configured to;

receive the bounding graph, determine local maximums and zeros in the bounding graph, and determine a dynamic mask based on the local maximums and zeros, wherein the dynamic mask is applied to a second digital image to determine the presence of an object.

2. The mobile platform of claim 1, further comprising an image filtering module configured to:

receive the bounding graph from the edge detection module, and filter noise peaks in the bounding graph.

3. The mobile platform of claim 2, further comprising a clipping module configured to:

receive the bounding graph from the image filtering module, remove a portion of the bounding graph corresponding to additional noise in the bounding graph, and replace the removed portion with a smoothed portion.

4. The mobile platform of claim 1, wherein the dynamic mask indicates an object detected in a field of view of the optical input module, and wherein a width and a height of the dynamic mask indicate a traversability of an area corresponding to the dynamic mask.

5. The mobile platform of claim 1, further comprising an object detection module configured to:

partition the first digital image into a plurality of sections, and determine a location of the object by determining a section of the plurality of sections having the dynamic mask located therein.

6. The mobile platform of claim 5, wherein the object detection module is configured to send an object avoidance signal to the drive mechanism thereby causing the base structure to avoid the detected object, wherein the object avoidance signal is based on the determined location of the object.

7. The mobile platform of claim 1, further comprising a histogram generation module configured to:

receive an input image, and generate the intensity histogram indicating frequencies of pixel intensities of the plurality of pixels ranging from 0 to 255, wherein the intensity histogram is used to generate a binary image by assigning pixels with predetermined intensities, and the binary image is used to determine an existence of an object.

8. The mobile platform of claim 1, wherein a second dynamic mask is applied to the second digital image to determine a presence of objects that are in close proximity to the optical input module, wherein the second dynamic mask corresponds to a bottom portion of the image.

9. The mobile platform of claim 1, wherein the pre-processing module is configured to convert the first digital image into grayscale.

10. The mobile platform of claim 1 wherein the base structure comprises one of a motorized wheel chair, a wheel chair, a robotic cleaner, a robotic vacuum, and a cane.

11. A method of determining a presence of an object, comprising:

inverting a digital image comprising a plurality of pixels;

setting a first threshold comprising;

I_L=max{i|f(i−

δ

)>

f(i), f(i+δ

)>

f(i)}

setting a second threshold comprising;

I_R=min{j|f(j−

δ

)>

f(j), f(j+δ

)>

f(j)}

wherein i and j are each an intensity, f(i) is a number of pixels having the intensity i found from an intensity histogram, and δ

is a resolution parameter;

normalizing the inverted digital image by;

setting first pixels in the plurality of pixels having an intensity value less than the first threshold or greater than the second threshold to a first predetermined intensity, and setting second pixels in the plurality of pixels having an intensity value between the first threshold and the second threshold to a second predetermined intensity;

performing edge detection on the normalized digital image, thereby outputting a bounding graph indicating an edge of a detected object in the digital image;

determining local maximums and zeros in the bounding graph;

determining a dynamic mask based on the local maximums and zeros; and

applying the dynamic mask to the normalized digital image to determine a presence of an object.

12. The method of claim 11, further comprising filtering noise peaks in the bounding graph.

13. The method of claim 12, further comprising removing a portion of the bounding graph corresponding to additional noise in the bounding graph and replacing the removed portion with a smoothed portion.

14. The method of claim 11, wherein the dynamic mask indicates the object detected in a field of view, and wherein a width and a height of the dynamic mask indicate a traversability of an area corresponding to the dynamic mask.

15. The method of claim 11, further comprising applying a first fixed mask to the normalized digital image and partitioning the fixed mask digital image into a plurality of sections to determine a location of the object.

16. The method of claim 15, further comprising sending an object avoidance signal to a drive mechanism thereby causing a base structure to avoid the detected object, wherein the object avoidance signal is based on the determined location of the object.

17. The method of claim 11, further comprising determining an intensity value corresponding to a maximum peak of an intensity histogram for a digital image, wherein the intensity histogram is used to generate a binary image by assigning pixels with the first and second predetermined intensity.

18. The method of claim 17, further comprising applying a first fixed mask to the input image to generate the intensity histogram;

and applying a second fixed mask to the binary image to determine a presence of objects that are in close proximity to a base platform by looking for an intensity switch.

19. The method of claim 11, further comprising converting the digital image into grayscale.

20. A mobile platform comprising:

a base structure comprising one of a motorized wheel chair, a wheel chair, a robotic cleaner, a robotic vacuum, and a cane;

a drive mechanism coupled to the base structure;

a machine vision system including;

an optical input module that captures an image in the form of a signal;

a pre-processing module configured to;

receive the signal from the optical input module, convert the signal into a grayscale digital image, invert the digital image,

set a first threshold comprising;

I_L=max{i|f(i−

δ

)>

f(i), f(i+δ

)>

f(i)}

set a second threshold comprising;

I_R=min{j|f(j−

δ

)>

f(j), f(j+δ

)>

f(j)}

wherein i and j are each an intensity, f(i) is a number of pixels having the intensity i found from an intensity histogram, and δ

is a resolution parameter;

an image normalizing module configured to normalize the inverted digital image by;

setting a first plurality of pixels having an intensity less than the first threshold or greater than the second threshold to a first predetermined intensity, and setting a second plurality of pixels having an intensity greater than or equal to the first threshold or less than or equal to the second threshold to a second predetermined intensity;

an edge detection module configured to;

receive the normalized digital image, and perform edge detection the normalized digital image, thereby outputting a bounding graph indicating edges of detected objects in the image;

an image filtering module configured to;

receive the bounding graph from the edge detection module, and filter noise peaks in the bounding graph;

a clipping module configured to;

receive the filtered bounding graph from the image filtering module, remove a portion of the filtered bounding graph corresponding to additional noise in the filtered bounding graph, and replace the removed portion with a smoothed portion;

a detection module configured to;

receive the clipped and filtered bounding graph, determine local maximums and zeros in the graph, and determine a dynamic mask based on the local maximums and zeros, wherein the dynamic mask is applied to a second digital image to determine the presence of an object;

and an object detection module configured to;

partition the second digital image into a plurality of sections, and determine a location of the object by determining a section of the plurality of sections having the dynamic mask located therein.