Apparatus, method, and computer product for discriminating object

• US 7,817,848 B2
• Filed: 11/16/2006
• Issued: 10/19/2010
• Est. Priority Date: 11/28/2005
• Status: Expired due to Fees

• Current Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Original Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Inventors: Sawada, Junichi, Sakata, Katsumi
• Primary Examiner(s): Mariam; Daniel G
• Assistant Examiner(s): Woldemariam; Aklilu K
• Application Number:
  US11/600,191
• Publication Number:
  US 20070206849A1
• Time in Patent Office: 1,433 Days
• Field of Search: 382/159, 382/104, 382/157, 382/195, 701/1, 706/15, 706/20
• US Class Current: 382/159
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An apparatus discriminates a potential obstacle in the path of a vehicle from among various objects in an image shot by a monocular camera. First, an object detecting unit detects an object in the image by applying a saliency calculation to the image. Second, an object discriminating unit discriminates an object from among the objects detected by the object detecting unit as a potential obstacle by applying a neural network method to the objects.

Claims

1. An apparatus for discriminating an imaged object within an image based on the image shot by a monocular camera, the apparatus comprising:

an object detecting unit that detects an object in the image by applying a saliency calculation to the image; and

an object discriminating unit that applies a neural network method to the object detected by the object detecting unit to discriminate whether the object is a potential obstacle, wherein

the object detecting unit detects the object as a salient feature that is not found in a learning base commonly extracted from a plurality of background images without the object.

2. The apparatus according to claim 1, wherein the monocular camera and the apparatus are installed on a vehicle, and the apparatus further comprises a risk determining unit that determines a risk of collision of the vehicle with the potential obstacle, and outputs a risk signal indicative of the risk.

3. The apparatus according to claim 2, further comprising a distance acquiring unit that acquires a first distance between the vehicle and the potential obstacle, wherein

the risk determining unit determines the risk based on the first distance.

4. The apparatus according to claim 3, wherein the distance acquiring unit includes an irradiating unit that irradiates waves toward the potential obstacle, detects reflected waves from the potential obstacle, and calculates the first distance based on traveling speed of the waves and time difference between irradiation of the waves and detection of the reflected waves.

5. The apparatus according to claim 3, wherein the risk determining unit further comprises a distance calculating unit that calculates a second distance between the vehicle and the potential obstacle from the image, and the risk determining unit determines the risk based on the first distance and the second distance.

6. The apparatus according to claim 2, further comprising an irradiating unit that is installed on a vehicle, and discriminates whether an object in a running path of the vehicle is a potential obstacle by irradiating waves toward the object and detecting reflected waves from the object, wherein

the risk determining unit determines the risk based on discrimination results by the irradiating unit and the object discriminating unit.

7. The apparatus according to claim 1, further comprising an irradiating unit that is installed on a vehicle, and acquires a direction of an object in a running path of the vehicle by irradiating waves toward the object and detecting reflected waves from the object, wherein

the object detecting unit detects the object by applying the saliency calculation to an area on the image corresponding to the direction.

8. The apparatus according to claim 1, further comprising a distance acquiring unit that is installed on a vehicle, and acquires a distance between the vehicle and an object in a running path of the vehicle, wherein

the object discriminating unit applies the neural network method in accordance with the distance.

9. An apparatus for discriminating an imaged object within an image based on the image shot by a monocular camera, the apparatus comprising:

a learning base storing unit that creates and stores a learning base commonly extracted from a plurality of background images without the image object;

a discriminating-pattern storing unit that learns and stores specific patterns of an object in advance;

an object detecting unit that compares the learning base and an actual input background image, and detects a residual based on the learning base and the actual input background image as a potential object; and

an object discriminating unit that compares the potential object with the specific patterns present in the discriminating-pattern storing unit, and discriminates whether the potential object is a specific object, wherein the object detecting unit detects the object as a salient feature that is not found in a learning base commonly extracted from a plurality of background images without the object.

10. A warning apparatus operating with an object discriminating device for discriminating an imaged object within an image based on shot image, the warning apparatus comprising:

a warning unit that outputs a warning in accordance with a discrimination result obtained by the object discriminating device,

wherein the object discriminating device includes a learning base storing unit that creates and stores therein a learning base commonly extracted from a plurality of background images without the image object, a discriminating-pattern storing unit that learns and stores therein specific patterns of an object in advance, an object detecting unit that compares the learning base and an actual input background image, and detects a residual based on the learning base and the actual input background image as a potential object, and an object discriminating unit that compares the potential object with the specific patterns present in the discriminating-pattern storing unit, and discriminates whether the potential object is a specific object.

11. A vehicle-controlling apparatus installed on a vehicle and operating with an object discriminating device for discriminating an imaged object within an image based on shot image, the vehicle-controlling apparatus comprising:

a controlling unit that controls driving of the vehicle in accordance with a discrimination result obtained by the object discriminating device,

wherein the object discriminating device includes a learning base storing unit that creates and stores a learning base commonly extracted from a plurality of background images without the image object, a discriminating-pattern storing unit that learns and stores specific patterns of an object in advance, an object detecting unit that compares the learning base and an actual input background image, and detects a residual based on the learning base and the actual input background image as a potential object, and an object discriminating unit that compares the potential object with the specific patterns present in the discriminating-pattern storing unit, and discriminates whether the potential object is a specific object and the object detecting unit detects the object as a salient feature that is not found in a learning base commonly extracted from a plurality of background images without the object.

12. A method of discriminating an imaged object within an image based on the image shot by a monocular camera, the method comprising:

detecting an object in the image by applying a saliency calculation to the image, with a processor; and

applying a neural network method in the processor to the to detected object to discriminate whether the object is a potential obstacle, wherein

the detecting includes detecting the object as a salient feature that is not found in a learning base commonly extracted from a plurality of background images without the object.

13. A method of discriminating an imaged object within an image shot by a monocular camera, the method comprising:

creating and storing in a memory device a learning base commonly extracted from a plurality of background images without the image object;

learning and storing specific patterns of an object in advance;

comparing the learning base and an actual input background image with a processor;

detecting in the processor a residual based on the learning base and the actual input background image as a potential object; and

discriminating whether the potential object is a specific object by comparing the potential object with the specific patterns stored in advance at the storing, wherein the detecting includes detecting the object as a salient feature that is not found in a learning base commonly extracted from a plurality of background images without the object.

14. A non-transitory computer-readable recording medium that stores therein a computer program that causes a computer to execute a method of discriminating an object within an image based on the image shot by a monocular camera, the computer program causing the computer to execute:

detecting an object in the image by applying a saliency calculation to the image; and

applying a neural network method to detected object to discriminate whether the object is a potential obstacle, wherein

the detecting includes detecting the object as a salient feature that is not found in a learning base commonly extracted from a plurality of background images without the object.

15. A computer program stored on a non-transitory computer-readable recording medium that stores therein a computer program that causes a computer to execute a method of discriminating an object within an image shot by a monocular camera, the computer program causing the computer to execute:

creating and storing a learning base commonly extracted from a plurality of background images without the image object;

learning and storing specific patterns of an object in advance;

comparing the learning base and an actual input background image;

detecting a residual based on the learning base and the actual input background image as a potential object; and

discriminating whether the potential object is a specific object by comparing the potential object with the specific patterns stored in advance at the storing, wherein the detecting includes detecting the object as a salient feature that is not found in a learning base commonly extracted from a plurality of background images without the object.