Low- And High-Fidelity Classifiers Applied To Road-Scene Images

• US 20190311221A1
• Filed: 06/18/2019
• Published: 10/10/2019
• Est. Priority Date: 01/13/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Nariyambut Murali, Vidya, Schrier, Madeline Jane
• Granted Patent: US 11,200,447 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/285 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 10/87 using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 30/19113 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system

Abstract

Disclosures herein teach applying a set of sections spanning a down-sampled version of an image of a road-scene to a low-fidelity classifier to determine a set of candidate sections for depicting one or more objects in a set of classes. The set of candidate sections of the down-sampled version may be mapped to a set of potential sectors in a high-fidelity version of the image. A high-fidelity classifier may be used to vet the set of potential sectors, determining the presence of one or more objects from the set of classes. The low-fidelity classifier may include a first Convolution Neural Network (CNN) trained on a first training set of down-sampled versions of cropped images of objects in the set of classes. Similarly, the high-fidelity classifier may include a second CNN trained on a second training set of high-fidelity versions of cropped images of objects in the set of classes.

Claims

1. A method comprising:

processing an image with a low-fidelity classifier, the low-fidelity classifier trained to calculate a probability, for each of a plurality of zones of the image, that a zone of the image comprises an object;

identifying a probable zone having a high probability for comprising the object;

forward feeding the probable zone to a high-fidelity classifier, the high-fidelity classifier trained to confirm a presence of the object in the probable zone.

2. The method of claim 1, further comprising:

receiving the image from a camera of a vehicle; and

down-sampling the image to generate a down-sampled version of the image;

wherein the down-sampled version of the image is processed by the low-fidelity classifier.

3. The method of claim 2, wherein the probable zone is identified based on the down-sampled version of the image and wherein the method further comprises mapping the probable zone to a higher-resolution version of the image.

4. The method of claim 3, wherein forward feeding the probable zone to the high-fidelity classifier comprises forward feeding the probable zone in the higher-resolution version of the image.

5. The method of claim 2, further comprising calculating a maximum factor by which the image can be down-sampled to generate the down-sampled version while maintaining a ratio of entropy in the down-sampled version such that the entropy of the down-sampled version is maintained above a threshold level.

6. The method of claim 2, further comprising extracting the plurality of zones from the image by sliding a window over the image pixel-by-pixel such that each zone of the plurality of zones is the size of the window and overlaps surrounding zones.

7. The method of claim 6, further comprising:

abstracting a set of scaled zones from the down-sampled version, wherein scaled zones in the set of scaled zones have different dimensions from of the sliding window;

selecting a scaled zone from the set of scaled zones for which the low-fidelity classifier indicates a probability of an existence of a scaled object; and

mapping the scaled zone to a scaled sector of the high-resolution version.

8. The method of claim 1, wherein the low-fidelity classifier and the high-fidelity classifier are each portions of a same convolutional neural network.

9. The method of claim 1, wherein the image is of a scene surrounding a vehicle and wherein the method further comprises:

determining whether the object exists in the scene surrounding the vehicle based on whether the presence of the object is confirmed by the high-fidelity classifier;

determining a driving maneuver for the vehicle based on whether the object exists in the scene surrounding the vehicle; and

providing the driving maneuver to an automated driving system of the vehicle to be executed by the automated driving system.

10. The method of claim 1, further comprising:

cropping each of a plurality of images that are labeled as comprising the object;

down-sampling each of the plurality of images to generate a down-sampled set of images;

training the low-fidelity classifier with the down-sampled set of images; and

training the high-fidelity classifier with at least one of the plurality of images at full resolution.

11. A system comprising:

a vehicle;

a camera affixed to the vehicle configured to capture an image of a scene surrounding the vehicle; and

a processor in communication with the camera and programmable to execute instructions stored in non-transitory computer readable storage media, the instructions comprising;

processing an image with a low-fidelity classifier, the low-fidelity classifier trained to calculate a probability, for each of a plurality of zones of the image, that a zone of the image comprises an object;

identifying a probable zone having a high probability for comprising the object;

forward feeding the probable zone to a high-fidelity classifier, the high-fidelity classifier trained to confirm a presence of the object in the probable zone.

12. The system of claim 11, wherein the instructions further comprise:

receiving the image from a camera of a vehicle; and

down-sampling the image to generate a down-sampled version of the image;

wherein the down-sampled version of the image is processed by the low-fidelity classifier.

13. The system of claim 12, wherein the probable zone is identified based on the down-sampled version of the image and wherein the method further comprises mapping the probable zone to a higher-resolution version of the image.

14. The system of claim 13, wherein forward feeding the probable zone to the high-fidelity classifier comprises forward feeding the probable zone in the higher-resolution version of the image.

15. The system of claim 12, wherein the instructions further comprise calculating a maximum factor by which the image can be down-sampled to generate the down-sampled version while maintaining a ratio of entropy in the down-sampled version such that the entropy of the down-sampled version is maintained above a threshold level.

16. A processor that is programmable to execute instructions stored in non-transitory computer readable storage media, the instructions comprising:

processing an image with a low-fidelity classifier, the low-fidelity classifier trained to calculate a probability, for each of a plurality of zones of the image, that a zone of the image comprises an object;

identifying a probable zone having a high probability for comprising the object;

forward feeding the probable zone to a high-fidelity classifier, the high-fidelity classifier trained to confirm a presence of the object in the probable zone.

17. The processor of claim 16, wherein the instructions further comprise:

receiving the image from a camera of a vehicle; and

down-sampling the image to generate a down-sampled version of the image;

wherein the down-sampled version of the image is processed by the low-fidelity classifier.

18. The processor of claim 17, wherein the probable zone is identified based on the down-sampled version of the image and wherein the method further comprises mapping the probable zone to a higher-resolution version of the image.

19. The processor of claim 17, further comprising:

calculating a maximum factor by which the image can be down-sampled to generate the down-sampled version while maintaining a ratio of entropy in the down-sampled version such that the entropy of the down-sampled version is maintained above a threshold level; and

extracting the plurality of zones from the image by sliding a window over the image pixel-by-pixel such that each zone of the plurality of zones is the size of the window and overlaps surrounding zones.

20. The processor of claim 16, wherein the low-fidelity classifier and the high-fidelity classifier are each portions of a same convolutional neural network.