Object detection using candidate object alignment

• US 10,025,998 B1
• Filed: 09/18/2015
• Issued: 07/17/2018
• Est. Priority Date: 06/09/2011
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Gdalyahu, Yoram, Livyatan, Harel, Schwarz, Oded, Taieb, Yoav, Stein, Gideon P., Viente, Kfir
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US14/858,798
• Time in Patent Office: 1,033 Days
• Field of Search: None
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 2210/21 Collision detection, intersection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Classification of an object in the field of view of a camera. A processor is configured to capture multiple image frames from the camera. A candidate image is detected in the image frames. Alignment of the candidate image is determined relative to at least one previously known training image by inputting the candidate image into a trained alignment classifier and outputting one or more alignment variables therefrom associated with the candidate image. The candidate image and the alignment variable(s) are input into a trained object classifier. The object is classified responsive to the alignment variable(s).

Claims

1. A method for classifying an object in the field of view of a camera, wherein a processor is configured to capture a plurality of image frames from the camera, the method comprising:

detecting a candidate image in the image frames;

determining alignment of said candidate image relative to at least one previously known training image by inputting the candidate image into a trained alignment classifier and outputting an alignment variable therefrom associated with the candidate image;

inputting the candidate image and said alignment variable into a trained object classifier; and

classifying the candidate image, wherein classifying the candidate image comprises classifying the object by the trained object classifier responsive to said alignment variable.

2. The method of claim 1, wherein the processor and the camera are included in a driver assistance system.

3. The method of claim 2, wherein said classifying the object comprises classifying the object as belonging to a single class of objects selected from the group consisting of:

pedestrians, traffic signs, vehicles and light sources.

4. The method of claim 2, wherein the candidate image is of a pedestrian and said alignment variable is selected from the group consisting of:

a vertical shift, a horizontal shift and a scale.

5. The method of claim 2, wherein the candidate image is of a rectangular traffic sign and said alignment variable is selected from the group consisting of:

a vertical shift and a horizontal shift in the vertical and horizontal edges respectively of said rectangular sign.

6. The method of claim 1, wherein said variable includes a plurality of alignment variables, the method further comprising:

performing a separate regression for each of said alignment variables.

7. The method of claim 1, further comprising

previously training said trained alignment classifier on a plurality of training images for said alignment classifier.

8. The method of claim 7 further comprising:

generating said training images by marking a plurality of images as perfect images of a training set and by perturbing alignment of said perfect images.

9. The method of claim 7 further comprising:

generating said training images by;

marking a plurality of images as perfect images of a training set,

detecting a candidate training image,

comparing said candidate training image to at least one of said perfect images, and

outputting a difference of said alignment variable between said candidate training image and at least one of said perfect images.

10. The method of claim 1, further comprising:

prior to inputting the candidate image into the trained alignment classifier, performing a preliminary object classification using a trained preliminary object classifier.

11. The method of claim 10,

prior to said performing said preliminary object classification, preliminary alignment classifying by outputting at least one preliminary alignment variable and outputting said preliminary alignment variable to said trained preliminary classifier.