DETECTION DEVICE, DETECTION PROGRAM, DETECTION METHOD, VEHICLE EQUIPPED WITH DETECTION DEVICE, PARAMETER CALCULATION DEVICE, PARAMETER CALCULATING PARAMETERS, PARAMETER CALCULATION PROGRAM, AND METHOD OF CALCULATING PARAMETERS

• US 20150347831A1
• Filed: 05/27/2015
• Published: 12/03/2015
• Est. Priority Date: 05/28/2014
• Status: Abandoned Application

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: YOKOI, KENSUKE, TAMATSU, YUKIMASA, SATO, IKURO
• Application Number:
  US14/722,397
• Publication Number:
  US 20150347831A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

A detection device has a neural network process section performing a neural network process using parameters to calculate and output a classification result and a regression result of each of frames in an input image. The classification result shows a presence of a person in the input image. The regression result shows a position of the person in the input image. The parameters are determined based on a learning process using a plurality of positive samples and negative samples. The positive samples have segments of a sample image containing at least a part of the person and a true value of the position of the person in the sample image. The negative samples have segments of the sample image containing no person.

Claims

1. A detection device comprising a neural network processing section capable of performing a neural network process using predetermined parameters in order to calculate and output a classification result and a regression result of each of a plurality of frames in an input image, the classification result representing a presence of a person in the input image, and the regression result representing a position of the person in the input image,

wherein the parameters are determined on the basis of a learning process using a plurality of positive samples and negative samples, each of the positive samples comprising a set of a segment of a sample image containing at least a part of a person and a true value of the position of the person in the sample image, and each of the negative samples comprising a segment of the sample image containing no person.

2. The detection device according to claim 1, further comprising an integration section capable of integrating the regression results of the position of the person in the frames which have been classified to indicate the presence of the person, and specifying the position of the person in the input image.

3. The detection device according to claim 1, wherein the number of the parameters does not depend on the number of the positive samples or the number of negative samples.

4. The detection device according to claim 1, wherein the position of the person contains a lower end position of the person.

5. The detection device according to claim 4, further comprising a calculation section capable of calculating a distance between a vehicle body of an own vehicle and the person on the basis of the lower end position of the person, and the input image is obtained by an in-vehicle camera mounted in the vehicle body of the own vehicle.

6. The detection device according to claim 5, wherein the position of the person contains a specific part of the person, and

the calculation section corrects the distance between the person and the vehicle body of the own vehicle by using the position of the person at a timing t and the position of the person at the timing t+1 while assuming that a height measured from the lower end position of the person and a position of a specific part of the person has a constant value, where the position of the person at the timing t is obtained by processing the input image captured by at the timing t and transmitted from the in-vehicle camera, and the position of the person at the timing t+1 is obtained by processing the input image captured by at the timing t+1 and transmitted from the in-vehicle camera.

7. The detection device according to claim 6, wherein

the calculation section corrects the distance between the person and the vehicle body of the own vehicle by solving a state space model by using time-series observation values, the state space model comprises an equation which describes a system model and an equation which describes an observation model, the system model shows a time expansion of the distance between the person and the vehicle body of the own vehicle, and uses an assumption in which the height measured from the lower end position of the person to the specific part of the person has a constant value, the observation model shows a relationship between the position of the person and the distance between the person and the vehicle body of the own vehicle.

8. The detection device according to claim 6, wherein

the calculation section corrects the distance between the person and the vehicle body of the own vehicle by using an upper end position of the person as the specific part and the assumption in which the height of the person has a constant value.

9. The detection device according to claim 1, wherein the position of the person contains a central position of the person in a horizontal direction.

10. The detection device according to claim 1, wherein the integration section performs a grouping of the frames in which the person is present, and integrates regression results of the person in each of the grouped frames.

11. The detection device according to claim 1, wherein the integration section integrates the regression results of the position of the person on the basis of the regression results having a high regression accuracy in the regression results of the position of the person.

12. The detection device according to claim 1, wherein the parameters are determined so that a cost function having a first term and a second term converges, wherein the first term is used by a classification regarding whether or not the person is present in the input image, and the second term is used by a regression of the position of the person.

13. The detection device according to claim 12, wherein the position of the person includes positions of a plurality of parts of the person, and the second term has coefficients corresponding to the positions of the parts of the person, respectively.

14. A detection program capable of performing a neural network process using predetermined parameters executed by a computer, wherein the neural network process is capable of obtaining and outputting a classification result and a regression result of each of a plurality of frames in an input image, the classification result representing a presence of a person in the input image, and the regression result representing a position of the person in the input image, and

the parameters are determined on the basis of a learning process on the basis of a plurality of positive samples, each of the positive samples comprising a set of a segment in a sample image containing at least a part of the person and a true value of the position of the person in the sample image, and a plurality of negative samples, each of the negative samples comprising a segment of the sample image containing no person.

15. A detection method comprising steps of:

calculating parameters for use in a neural network process by performing a learning process on the basis of a plurality of positive samples and negative samples, each of the positive samples comprising a set of a segment of a sample image containing at least a part of the person and a true value of the position of the person in the sample images, and each of the negative samples comprising a segment of the sample image containing no person;

performing the neural network process using the parameters; and

outputting classification results of a plurality of frames in an input image, a classification result representing a presence of a person in the input image, and a regression result of a position of the person in the input image.

16. A vehicle comprising:

a vehicle body;

an in-vehicle camera mounted in the vehicle body and is capable of generating an image of a scene in front of the vehicle body;

a neural network processing section capable of inputting the image as an input image transmitted from the in-vehicle camera, performing a neural network process using predetermined parameters, outputting classification results and regression results of each of a plurality of frames in the input image, the classification results representing a presence of a person in the input image, and the regression results representing a lower end position of the person in the input image;

an integration section capable of integrating the regression results of the position of the person in the frames in which the person is presence, and specifying a lower end position of the person in the input image;

a calculation section capable of calculating a distance between the person and the vehicle body on the basis of the specified lower end position of the person; and

a display device capable of displaying an image containing the distance between the person and the vehicle body,

wherein the predetermined parameters are determined by learning on the basis of a plurality of positive samples and negative samples, each of the positive samples comprise a set of a segment of a sample image containing at least a part of the person and a true value of the position of the person in the sample images, and each of the negative samples comprise a segment of the sample image containing no person.

17. A parameter calculation device capable of performing learning of a plurality of positive samples and negative samples, in order to calculate parameters for use in a neural network process of an input image, wherein each of the positive samples comprises a set of a segment of a sample image containing at least a part of the person and a true value of the position of the person in the sample images, and each of the negative samples comprises a segment of the sample image containing no person.

18. A parameter calculation program, to be executed by a computer, of performing a function of a parameter calculation device capable of performing learning of a plurality of positive samples and negative samples, in order to calculate parameters for use in a neural network process of an input image,

wherein each of the positive samples comprises a set of a segment of a sample image containing at least a part of the person and a true value of the position of the person in the sample images, and each of the negative samples comprises a segment of the sample image containing no person.

19. A method of calculating parameters for use in a neural network process of an input image, by performing learning of a plurality of positive samples and negative samples, where each of the positive samples comprises a set of a segment of a sample image containing at least a part of the person and a true value of the position of the person in the sample images, and each of the negative samples comprises a segment of the sample image containing no person.