Object appearance frequency estimating apparatus

• US 9,245,189 B2
• Filed: 11/07/2011
• Issued: 01/26/2016
• Est. Priority Date: 11/29/2010
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Sasaki, Mikio
• Primary Examiner(s): Girma, Fekadeselassie
• Assistant Examiner(s): BAMERT, JOHN J
• Application Number:
  US13/290,340
• Publication Number:
  US 20120133497A1
• Time in Patent Office: 1,541 Days
• Field of Search: 340/995.24, 340/905, 340/995.13, 340/7.48, 701/438, 701/414, 701/423
• US Class Current: 1/1
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/0969 having a display in the form of a map
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

An estimation apparatus of an object appearance frequency is provided. The apparatus estimates an appearance frequency of objects, such as pedestrians, in a predetermined estimation area. The apparatus calculates a matrix F_PFP by searching an appearance frequency data in the past. In detail, an estimated result that is an output of an estimating module is expressed as a vector. Objects to be estimated are classified into a total of 12 kinds, such as a “man, woman, child, bike, unknown, dog” and a “right, left”. Feature vector of appearance frequency of pedestrians, i.e., objective variables of estimation, is expressed by 12th dimension vector space. Moreover, status information is used as explaining variables, which explain the feature vectors when a feature vector occurs. The regression relationship of the feature vector with respect to the status information vector expressed in 28th dimension is solved by the linear least squares method.

Claims

1. An object appearance frequency estimating apparatus mountable on a vehicle, and being capable of estimating an appearance frequency of an object on an outside of the vehicle in a predetermined estimation area, comprising:

a storage module;

a frequency data acquisition module operatively connected to the storage module, and configured to acquire frequency data relating to the appearance frequency of the object based on previously collected data, the frequency data including previously collected space-time information, and frequency information for defining the appearance frequency with respect to the previously collected space-time information, and further configured to store the acquired frequency data in the storage module, and to output an appearance frequency signal;

an estimating module operatively connected to the storage module and the frequency data acquisition module, and configured to process the appearance frequency signal received from the frequency data acquisition module, to estimate an appearance frequency of the object in a predetermined estimation area based on the appearance frequency signal to generate an estimated appearance frequency signal to enable the object to be identified, and to store the estimated appearance frequency signal of the object in the predetermined estimation area in the storage module; and

a notifying module operatively connected to the estimating module, configured to generate a driver warning when the estimated appearance frequency signal comprises at least one of a pedestrian, a bike and an animal in the predetermined estimation area, and configured to generate the driver warning when the estimated appearance frequency signal comprises at least another of pedestrian, bike, and animal that is different from the at least one of the pedestrian, the bike, and the animal, wherein

the previously collected space-time information is defined as a status information vector which uses status information as explanatory variables, the status information including place, time, and weather information,

the frequency information is defined as a feature vector which is expressed by using feature information as an objective variable, and

the feature information includes information about types of objects and information about a partial area provided by dividing a view from the vehicle into a plurality of areas.

2. The object appearance frequency estimating apparatus in claim 1,

wherein

the estimating module estimates appearance frequencies of objects in the predetermined estimation area by solving the regression relationship of the feature vector with respect to the status information vector by using the linear least squares method.

3. The object appearance frequency estimating apparatus in claim 1, further

comprising:

a camera module which captures view from the vehicle; and

a recognition module which generates the frequency information based on a result of image recognition on an image captured by the camera module.

4. The object appearance frequency estimating apparatus in claim 3, further

comprising:

a calculating module which calculates a reliability degree of the result of image recognition by the recognition module.

5. The object appearance frequency estimating apparatus in claim 4, wherein

the calculating module calculates the reliability degree based on at least one of the following factors, a kind of object, a sensing place, a sensor and recognition technique for image recognition, and a driving speed of the vehicle.

6. The object appearance frequency estimating apparatus in claim 3, further

comprising:

a transmitting module which uploads upload information including information about a point where the vehicle runs and the result of image recognition by the recognition module to a center which handles the upload information.

7. The object appearance frequency estimating apparatus in claim 6, wherein

the upload information includes the reliability degree calculated in the calculating module.

8. The object appearance frequency estimating apparatus in claim 6, wherein

the upload information includes a result of the estimating module.

9. The object appearance frequency estimating apparatus in claim 3, wherein

the center accumulates the upload information, and the apparatus further comprising a module which acquires, in a case that a differential between a result of estimation by the estimating module and a result of image recognition by the recognition module is equal to or more than a threshold value, the upload information accumulated in the center.

10. The object appearance frequency estimating apparatus in claim 3, further

comprising:

a parameter setting module which sets a recognition parameter for image recognition into the recognition module.

11. The object appearance frequency estimating apparatus in claim 10, wherein

the parameter setting module sets at least one of the recognition parameter based on data of appearance frequency in the past.

12. The object appearance frequency estimating apparatus in claim 10, wherein

the parameter setting module sets the recognition parameter in accordance with a target object.

13. The object appearance frequency estimating apparatus in claim 10, wherein

the parameter setting module corrects the recognition parameter in a case that a differential between a result of estimation by the estimating module and a result of image recognition by the recognition module is equal to or more than a threshold value.

14. The object appearance frequency estimating apparatus in claim 1, further

comprising:

an attractor information acquisition module which acquires information about an attractor which is an element being capable of attracting people and of driving those people to come to there and leave from there; and

a time gate setting module which sets up a time gate which shows an existence probability of the object in the attractor, the existence probability being affected by a time factor, and wherein

the estimating module estimates the appearance frequency of the object in the predetermined estimation area by considering a distance between the attractor and the predetermined area and the time gate.

15. The object appearance frequency estimating apparatus in claim 14, wherein

the time gate is defined by a rectangular wave which has height corresponding to the appearance frequency of the object, and

the estimating module estimates the appearance frequency of the object in the predetermined estimation area by considering propagation of the rectangular wave which is divided at a branch from the attractor to the predetermined area.

16. The object appearance frequency estimating apparatus in claim 15, wherein

the rectangular wave defining the time gate is initialized to have height corresponding to an average accommodation number in the attractor.

17. The object appearance frequency estimating apparatus in claim 15, wherein

the estimating module estimates the appearance frequency of the object in the predetermined estimation area based on a reverse propagation in which the rectangular wave propagates in a reverse direction of time.

18. The object appearance frequency estimating apparatus in claim 14, wherein

the estimating module estimates the appearance frequency of the object in the predetermined estimation area by considering a passing time of the vehicle in the predetermined area.

19. In a vehicle-mountable-object appearance frequency apparatus, a method for estimating an appearance frequency of an object outside of a vehicle in a predetermined estimation area for use in a vehicle-mountable object appearance frequency apparatus, comprising:

acquiring frequency data relating to the appearance frequency of the object based on previously collected data, the frequency data including previously collected space-time information for defining space-time, and frequency information for defining the appearance frequency with respect to the previously collected space-time information;

estimating an appearance frequency of an object in a predetermined estimation area based on the acquired frequency data to generate an estimated appearance frequency signal;

generating a driver warning when the estimated appearance frequency signal comprises at least one of a pedestrian, a bike and an animal in the predetermined estimation area; and

generating the driver warning when the estimated frequency signal comprises at least another of pedestrian, bike, and animal that is different from the at least one of the pedestrian, the bike, and the animal,

wherein

the object is at least one of a pedestrian, a bike, and an animal,

the previously collected space-time information is defined as a status information vector which uses status information as explaining variables, the status information including a place, a time, and weather information,

the frequency information is defined as a feature vector which is expressed by using feature information as an objective variable, and

the feature information includes information about kind of objects and information about a partial area provided by dividing a view from the vehicle into a plurality of areas.

20. An object appearance frequency estimating apparatus, comprising:

a storage module;

a frequency data acquisition module operatively connected to the storage module, and configured to acquire frequency data with both previously collected space-time information, including location, time, attractor and weather information, and frequency information for defining an object appearance frequency within a partial area based on the previously collected space-time information, to store the acquired frequency data in the storage module, and to output an appearance frequency signal;

an estimating module operatively connected to the storage module and the frequency data acquisition module, and configured to process the appearance frequency signal, to estimate the object appearance frequency of an approaching object in a predetermined estimation area, to store an estimated object appearance frequency signal of the approaching object in the storage module, and to output the estimated object appearance frequency signal,

a calculation module operatively connected to frequency data acquisition module, and configured to receive the estimated object appearance frequency signal, to generate a corresponding reliability degree indicating to what extent the object appearance frequency accurately identifies the approaching object, to store the corresponding reliability degree in the storage module, and to output a corrected object appearance frequency signal; and

a notifying module operatively connected to the estimating module, configured to receive the corrected object appearance frequency signal and to generate a driver warning when the corrected object appearance frequency signal of the approaching object comprises at least one of a pedestrian, a bike and an animal in the predetermined estimation area, and configured to generate the driver warning when the corrected object appearance frequency signal comprises at least another of pedestrian, bike, and animal that is different from the at least one of the pedestrian, the bike, and the animal.

21. The object appearance frequency estimating apparatus of claim 20, wherein the calculation module is located in a control center.

22. An object appearance frequency estimating apparatus, comprising:

an image capturing module configured to capture data related to an image of an approaching object in a predetermined estimation area and to output an image signal;

a recognition module operatively connected to the image capturing module and configured to process the image signal based on stored past image recognition parameters to generate an image recognition result;

an estimating module configured to estimate an appearance frequency of the approaching object in the predetermined estimation area and to output an object appearance frequency signal;

a storage module operatively configured to store the image recognition result along with the stored past image recognition parameters; and

a notifying module configured to generate a driver warning when the predetermined estimation area is approached,

wherein each module is configured to perform each corresponding function for at least two of a pedestrian, a bike, and an animal.