Object detection using candidate object alignment

• US 9,183,447 B1
• Filed: 06/11/2012
• Issued: 11/10/2015
• Est. Priority Date: 06/09/2011
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Gdalyahu, Yoram, Livyatan, Harel, Taieb, Yoav, Stein, Gideon P., Schwarz, Oded, Viente, Kfir
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US13/493,136
• Time in Patent Office: 1,247 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 2210/21 Collision detection, intersection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Classification of an object in the field of view of a camera. A processor is configured to capture multiple image frames from the camera. A candidate image is detected in the image frames. Alignment of the candidate image is determined relative to at least one previously known training image by inputting the candidate image into a trained alignment classifier and outputting one or more alignment variables therefrom associated with the candidate image. The candidate image and the alignment variable(s) are input into a trained object classifier. The object is classified responsive to the alignment variable(s).

Claims

1. A method for classifying an object in the field of view of a camera, wherein a processor is configured to capture a plurality of image frames from the camera, the method comprising:

detecting a candidate image in the image frames;

determining alignment of said candidate image relative to at least one previously known training image by inputting the candidate image into a trained alignment classifier and outputting a plurality of alignment variables therefrom associated with the candidate image by performing a separate regression for each of said alignment variables;

inputting the candidate image and said alignment variables into a trained object classifier; and

classifying the object by the trained object classifier responsive to said alignment variables.

2. The method of claim 1, wherein the processor and the camera are included in a driver assistance system.

3. The method of claim 2, wherein said classifying said candidate image includes classifying the object as belonging to a single class of objects selected from the group consisting of:

pedestrians, traffic signs, vehicles and light sources.

4. The method of claim 2, wherein the candidate image is of a pedestrian and at least one of said alignment variables is selected from the group consisting of:

a vertical shift, a horizontal shift and a scale.

5. The method of claim 2, wherein the candidate image is of a rectangular traffic sign and at least one of said alignment variables is selected from the group consisting of:

a vertical shift and a horizontal shift in the vertical and horizontal edges respectively of said rectangular sign.

6. The method of claim 1, further comprising

previously training said trained alignment classifier on a plurality of training images for said alignment classifier.

7. The method of claim 6 further comprising:

generating said training images by marking a plurality of images as perfect images of a training set and by perturbing alignment of said perfect images.

8. The method of claim 6 further comprising:

generating said training images by;

marking a plurality of images as perfect images of a training set,

detecting a candidate training image,

comparing said candidate training image to at least one of said perfect images, and

outputting a difference of at least one of said alignment variables between said candidate training image and at least one of said perfect images.

9. The method of claim 1, further comprising:

prior to said determining alignment, performing a preliminary object classification using a trained preliminary object classifier.

10. The method of claim 9, further comprising:

prior to said performing said preliminary object classification, preliminary aligning by outputting at least one preliminary alignment variable and outputting said preliminary alignment variable to said trained preliminary classifier.

11. A system including a camera and a processor, the system configured to classify an object in the field of view of the camera, wherein the processor is configured to capture a plurality of image frames from the camera, the system configured to perform the method steps of claim 1.