Systems and methods for multi-dimensional object detection

• US 9,633,436 B2
• Filed: 07/18/2013
• Issued: 04/25/2017
• Est. Priority Date: 07/26/2012
• Status: Active Grant

• Current Assignee: Infosys Limited
• Original Assignee: Infosys Limited
• Inventors: Hota, Rudra Narayan, Pisipati, Radha Krishna, Syed, Shahanaz, Jonna, Kishore, Bandyopadhyay, Subhadip
• Primary Examiner(s): HU, FRED H.
• Application Number:
  US13/945,830
• Publication Number:
  US 20140029852A1
• Time in Patent Office: 1,377 Days
• Field of Search: None
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

Systems and methods for multi-dimensional object detection are described. Embodiments disclose receiving image frames, extracting image components in the image frame, identifying line segments in the extracted components, grouping the line segments into groups, based at least in part on one or more similarities between the slope associated with a line segment and the spatial proximity between the line segments, and merging each of the one or more identified line segments in a selected group into a single line segment. Embodiments additionally disclose detecting the position of one or more objects in the image frame by identifying objects in the image frame, producing a second version of the image frame, applying at least one image classifier to the image frame and the second version of the image frame, and identifying coordinates associated with at least one target object. Some embodiments additionally couple lane and object detection with alert generation.

Claims

1. A computer implemented method for multi-dimensional object detection, the method comprising:

receiving, by a computing apparatus, a video sequence consisting of a plurality of image frames; and

processing, by the computing apparatus, a received image frame of the plurality of image frames, wherein the processing comprises;

analyzing the received image frame for the presence of one or more line segments, wherein the analyzing comprises;

extracting one or more image components in the image frame, wherein an image component includes at least one edge;

identifying one or more line segments in the extracted image components;

grouping the one or more identified line segments into one or more groups, wherein the grouping is based at least in part on one or more similarities between a slope associated with a line segment and a spatial proximity between the line segment and one or more other identified line segments; and

merging each of the one or more identified line segments in a selected group into a single straight line segment and identifying the coordinates of the single straight line segment therefrom; and

detecting the position of one or more objects in the image frame, wherein detecting comprises;

identifying one or more suspected target objects in the image frame, wherein a suspected target object is characterized by a variance in at least one image feature;

producing a second version of the image frame by removing the one or more suspected target objects from the image frame, wherein the one or more suspected target objects are not present in the second version of the image frame;

applying at least one image classifier to the image frame and the second version of the image frame to identify at least one target object; and

identifying coordinates associated with the at least one target object.

2. The computer implemented method as claimed in claim 1, further comprising categorizing the identified at least one target object into an object class, wherein object classes are selected from a group consisting of lanes, a zebra pattern, obstacles, signs, vehicles and pedestrians.

3. The computer implemented method as claimed in claim 2, further comprising categorizing the identified at least one target object as belonging to an identified lane by comparing coordinates associated with the position of the identified object with coordinates associated with the position of an identified merged line segment.

4. The computer implemented method as claimed in claim 3, further comprising identifying the position of the computing apparatus and associating the position thereof with an identified lane.

5. The computer implemented method as claimed in claim 4, further comprising triggering an alarm if the identified at least one target object is categorized as belonging to the same lane as the computing apparatus.

6. The computer implemented method as claimed in claim 1, further comprising detecting a zebra pattern structure in the image frame, wherein the detecting comprises:

identifying at least three parallel and equidistant line segments from the group of one or more line segments previously identified, wherein the identified at least three line segments are additionally parallel, with a tolerance of five degrees, to a vertical axis associated with the image frame and the distance between centroids of the adjacent line segments is less than a predetermined constant.

7. The computer implemented method as claimed in claim 6, further comprising presenting an alarm if the identified at least three line segments are not parallel, with a tolerance of five degrees, to the vertical axis associated with the image frame.

8. The computer implemented method as claimed in claim 1, further comprising identifying one or more characters in the received image frame by:

performing an optical character recognition (OCR) pass on the image frame; and

reconstructing output generated by the OCR pass.

9. The computer implemented method as claimed in claim 8, further comprising:

matching the reconstructed OCR output with one or more characters selected from a natural language dictionary.

10. The computer implemented method as claimed in claim 9, further comprising displaying the one or more characters on a display device communicably connected to the computing apparatus.

11. The computer implemented method as claimed in claim 1, further comprising cascading two or more image feature classifiers involved in the detection of an object in the image frame.

12. A system for multi-dimensional object detection, the system comprising:

at least one processor in operable communication with a processor readable memory, the memory containing one or more programming instructions whereby the processor is configured to;

extract at least one image frame from a received video sequence; and

process the extracted image frame, wherein the processing is performed by;

a lane detection module, wherein the lane detection module detects one or more edges and one or more line segments in the image frame, clusters the one or more line segments into one or more groups, and merges one or more line segments present in each group into one or more corresponding merged straight line segments;

an obstacle detection module, wherein the obstacle detection module identifies at least one object in the image frame and applies at least one image classifier to the image frame and a second image frame generated by the obstacle detection module by removing the at least one identified object from the image frame, wherein the second image frame is a version of the image frame without the at least one identified object;

a character recognition module, wherein the character recognition module runs an optical character recognition operation on the image frame and constructs a character output thereby; and

a coordination module, wherein the coordination module receives information about the one or more corresponding merged line segments from the lane detection module and information about the one or more objects from the obstacle detection module, and generates one or more recommendations, including an alarm and a message display.

13. The system as claimed in claim 12, wherein the processor is further configured to categorize the identified at least one object into an object class selected from a group consisting of lanes, obstacles, signs, vehicles and pedestrians.

14. The system as claimed in claim 13, wherein the processor is further configured to categorize the identified at least one object as belonging to an identified lane by comparing coordinates associated with the position of the identified at least one object with coordinates associated with the position of an identified merged line segment.

15. The system as claimed in claim 14, wherein the processor is further configured to identify the position of the computing apparatus, and associate the position thereof with an identified lane.

16. The system as claimed in claim 15, wherein the processor is further configured to present an alarm if the identified at least one object is categorized as belonging to the same lane as the computing apparatus.

17. The system as claimed in claim 12, wherein the processor is further configured to detect a zebra pattern structure in the image frame, wherein the detecting comprises:

identifying at least three parallel and equidistant line segments from the group of one or more line segments previously identified, wherein the identified at least three line segments are additionally parallel, with a tolerance of five degrees, to a vertical axis associated with the image frame, and the distance between centroids of the line segments is less than a predetermined constant.

18. The system as claimed in claim 17, wherein the processor is further configured to present an alarm if the identified at least three line segments are not parallel, with a tolerance of five degrees, to the vertical axis associated with the image frame.

19. The system as claimed in claim 12, wherein the processor is further configured to identify one or more characters in the received image frame by:

performing an optical character recognition (OCR) pass on the image frame; and

reconstructing output generated by the OCR pass.

20. The system as claimed in claim 19, wherein the processor is further configured to:

match the reconstructed OCR output with one or more characters selected from a natural language dictionary.

21. The system as claimed in claim 20, wherein the processor is further configured to display one or more characters recognized on a display device communicably connected to the computing apparatus.

22. The system as claimed in claim 12, wherein the processor is further configured to cascade two or more image feature classifiers involved in the detection of an object in the image frame.

23. A non-transitory computer readable storage medium having stored thereon computer executable instructions for performing a method for multi-dimensional object detection, the method comprising:

receiving a video sequence consisting of a plurality of image frames; and

processing a received image frame of the plurality of image frames, wherein the processing comprises;

analyzing the received image frame for the presence of one or more line segments, wherein the analyzing comprises;

extracting one or more image components in the image frame, wherein an image component includes at least one edge;

identifying one or more line segments in the extracted image components;

grouping the one or more identified line segments into one or more groups, wherein the grouping is based at least in part on one or more similarities between a slope associated with a line segment and a spatial proximity between the line segment and one or more other identified line segments; and

merging each of the one or more identified line segments in a selected group into a single straight line segment and identifying the coordinates of the single straight line segment therefrom; and

detecting the position of one or more objects in the image frame, wherein detecting comprises;

identifying one or more suspected target objects in the image frame, wherein a suspected target object is characterized by a variance in at least one image feature;

producing a second version of the image frame by removing the one or more suspected target objects from the image frame, wherein the one or more suspected target objects are not present in the second version of the image frame;

applying at least one image classifier to the image frame and the second version of the image frame to identify at least one target object; and

identifying coordinates associated with the at least one target object.