Self-learning object detection and classification systems and methods

• US 8,254,670 B2
• Filed: 02/25/2009
• Issued: 08/28/2012
• Est. Priority Date: 02/25/2009
• Status: Active Grant

• Current Assignee: Toyota Motor Corporation
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Prokhorov, Danil V.
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Patel, Nirav G
• Application Number:
  US12/392,379
• Publication Number:
  US 20100215254A1
• Time in Patent Office: 1,280 Days
• Field of Search: 382/159
• US Class Current: 382/159
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors

Abstract

A method of object classification based upon fusion of a remote sensing system and a natural imaging system is provided. The method includes detecting an object using the remote sensing system. An angle of view of a video camera of the natural imaging system is varied. An image including the object is generated using the natural imaging system. The natural imaging system may zoom in on the object. The image represented in either pixel or transformed space is compared to a plurality of templates via a competition based neural network learning algorithm. Each template has an associated label determined statistically. The template with a closest match to the image is determined. The image may be assigned the label associated with the relative location of the object, the relative speed of the object, and the label of the template determined statistically to be the closest match to the image.

Claims

1. A method of object classification, comprising:

generating an image using a natural imaging system;

detecting an object displayed in the image using a remote sensing system;

mapping a return of the remote sensing system with the image;

comparing the image to a plurality of templates, each template having an associated label;

determining which of the plurality of templates is a closest match to the image represented in pixel or transformed space based on performing a competition based neural network learning algorithm (LA) and reinitializing one or more neurons; and

assigning the label to the image associated with a template of the plurality of templates determined to be the closest match to the image represented in pixel or transformed space.

2. The method of claim 1 further comprising varying an angle of view of a video camera of the natural imaging system to zoom in on the object detected by the remote sensing system.

3. The method of claim 2, wherein the step of varying the angle of view of the video camera includes narrowing the angle of view by optically zooming the video camera.

4. The method of claim 3, wherein optically zooming the video camera comprises optically zooming the video camera by a predetermined zoom factor.

5. The method of claim 1 further comprising updating one or more counter associated with a winning neuron.

6. The method of claim 1 further comprising determining if the object is on a road using map information and/or a global positioning unit for automatic labeling of the image.

7. The method of claim 1 further comprising determining a relative speed of the object for automatic labeling of the image.

8. The method of claim 1 further comprising updating one or more templates using the image.

9. An object classification system, comprising:

a natural imaging system that generates an image;

a remote sensing system that detects an object displayed in the image using a remote sensing system;

a computer-implemented algorithm that maps a return of the remote sensing system with the image; and

a self-learning and categorization system including logic that (i) compares the image represented in pixel or transformed space to a plurality of templates, each template having an associated label, (ii) determines which of the plurality of templates is a closest match to the image represented in pixel or transformed space, (iii) assigns the label to the image associated with a template of the plurality of templates determined to be the closest match to the image represented in pixel or transformed space and (iv) performs a competition based neural network learning algorithm (LA) and reinitializes one or more neurons.

10. The object classification system of claim 9, wherein the learning and classification system comprises a plurality of components, where each component has an associated template and an associated label.

11. The object classification system of claim 10, wherein each component has one or more counters associated therewith.

12. The object classification system of claim 9 further comprising map information and/or a global positioning unit used to determine if the target object is on a road for automatic labeling of the image.

13. The object classification system of claim 9, wherein the learning and categorization system comprises logic for performing a competition based neural network learning algorithm (LA) during which one or more neurons are reinitialized.

14. The object classification system of claim 9, wherein the natural imaging system comprises a video camera having an optical zoom that zooms in on the object to generate the image.

15. The object classification system of claim 14, wherein the video camera is a first video camera and the image is a first image, the natural imaging system comprising a plurality of video cameras such that, during operation, a second video camera provides a second image having a relatively wide angle of view and the first camera provides the first image having a relatively narrow angle of view.

16. A vehicle comprising an image classification system, the vehicle comprising:

a global positioning unit;

a remote detection system configured to provide a return for a target object within a range of the remote detection system;

a natural imaging system including a video camera for providing an image of the target object;

a computer-implemented algorithm that associates the return of the remote detection system with the image of the natural imaging system using a perspective mapping transformation;

a self-learning and categorization system that classifies the image using information from the global positioning unit; and

map information used by the learning and categorization system for classifying the image.

17. The vehicle of claim 16 further comprising an inertial measurement unit that provides information to the learning and categorization system for classifying the image.

18. The vehicle of claim 16, wherein the video camera has an optical zoom for zooming in on the target object to provide the image.

19. The vehicle of claim 16 further comprising a fuzzy logic decision system for automatic labeling of the image.

20. The vehicle of claim 16 further comprising counters for automatic labeling of the image.