Low- and high-fidelity classifiers applied to road-scene images

• US 10,373,019 B2
• Filed: 01/13/2016
• Issued: 08/06/2019
• Est. Priority Date: 01/13/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Nariyambut Murali, Vidya, Schrier, Madeline Jane
• Primary Examiner(s): Azima, Shaghayegh
• Application Number:
  US14/995,134
• Publication Number:
  US 20170200063A1
• Time in Patent Office: 1,301 Days
• CPC Class Codes: G06F 18/285 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 10/87 using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 30/19113 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system

Abstract

Disclosures herein teach applying a set of sections spanning a down-sampled version of an image of a road-scene to a low-fidelity classifier to determine a set of candidate sections for depicting one or more objects in a set of classes. The set of candidate sections of the down-sampled version may be mapped to a set of potential sectors in a high-fidelity version of the image. A high-fidelity classifier may be used to vet the set of potential sectors, determining the presence of one or more objects from the set of classes. The low-fidelity classifier may include a first Convolution Neural Network (CNN) trained on a first training set of down-sampled versions of cropped images of objects in the set of classes. Similarly, the high-fidelity classifier may include a second CNN trained on a second training set of high-fidelity versions of cropped images of objects in the set of classes.

Claims

1. A method for object classification and location information detection, comprising:

down-sampling an image to a down-sampled version of the image;

wherein down-sampling the image to the down-sampled version of the image comprises calculating a maximum factor by which the image can be down-sampled to generate the down-sampled version while maintaining a ratio of entropy in the down-sampled version to entropy in the image above a predetermined threshold level;

extracting a set of overlapping zones covering the down-sampled version, as definable by a sliding window with dimensions equal to dimensions of the set of overlapping zones;

selecting a probable zone from the set of overlapping zones for which a low-fidelity classifier, comprising a first Convolutional Neural Network (CNN), indicates a probability of a presence of an object pertaining to a class of objects classifiable by the low-fidelity classifier;

mapping the probable zone selected from the down-sampled version to a sector of a higher-resolution version of the image;

confirming the presence of the object by applying the sector to a high-fidelity classifier, comprising a second CNN, where applying the sector indicates the presence; and

providing a driving assistance to an automated driving system of a vehicle to be executed by the automated driving system based on the presence of the object.

2. The method of claim 1, further comprising:

cropping a set of images of objects at a set of image sizes, images in the set of images classified according to a set of detection classes by labels assigned to the images;

down-sampling the set of images to create a down-sampled set of labeled images;

training the low-fidelity classifier with the down-sampled set of labeled images; and

training the high-fidelity classifier with at least one of the set of images and comparable images selected for purposes of training.

3. The method of claim 2, further comprising:

collecting a training set of images depicting pedestrians in various positions and contexts for inclusion within the set of images; and

labeling the training set of images according to a common class in the set of detection classes.

4. The method of claim 1, further comprising searching zones in the set of overlapping zones to which the low-fidelity classifier has yet to be applied for at least one additional probable zone while simultaneously confirming the presence of the object by applying the sector to the high-fidelity classifier.

5. The method of claim 1, further comprising:

capturing, by a camera affixed to the vehicle, a series of images of oncoming road-scenes at a frame-rate satisfying a predefined threshold; and

processing the series of images at a processing-rate also satisfying the predefined threshold, the predefined threshold providing sufficient time for a pre-determined autonomous response by the automated driving system of the vehicle to classify information in the series of images.

6. The method of claim 1, further comprising:

abstracting a set of scaled zones from the down-sampled version, scaled zones in the set of scaled zones having differing dimensions from the dimensions of the sliding window and commensurate with scaled dimensions of a scaled sliding window;

selecting a scaled zone from the set of scaled zones for which the low-fidelity classifier indicates a probability of an existence of a scaled object classifiable by the low-fidelity classifier;

mapping the scaled zone to a scaled sector of the higher-resolution version; and

confirming the existence of the scaled object by applying the scaled sector to the high-fidelity classifier, where applying the scaled sector results in a probability of the existence.