OPERATION ESTIMATION APPARATUS, ROBOT, AND OPERATION ESTIMATION METHOD

• US 20160342830A1
• Filed: 05/17/2016
• Published: 11/24/2016
• Est. Priority Date: 05/18/2015
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Ariizumi, Takahiro, Nakamura, Takahiro
• Granted Patent: US 9,990,540 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/40202 Human robot coexistence
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An operation estimation apparatus includes a distance measurement unit configured to measure a distance for a predetermined range, a human-body processing unit configured to detect a person on the basis of a distribution of measured distances, a tracking unit configured to assign an identifier to each detected person, and a situation estimation unit configured to estimate reaction of the person on the basis of a human data height which is a value of a height direction included in the measured distance in a first period set by prompting the person to start and end hand raising, wherein the situation estimation unit measures the human data height for each person to which the identifier is assigned a plurality of times in the first period, extracts an identifier for which human data heights are in the predetermined range, and estimates the reaction of the person corresponding to the extracted identifier.

Claims

1. An operation estimation apparatus comprising:

a distance measurement unit configured to measure a distance for a predetermined range;

a human-body processing unit configured to detect a person on the basis of a distribution of distances measured by the distance measurement unit;

a tracking unit configured to assign an identifier to each person detected by the human-body processing unit; and

a situation estimation unit configured to estimate a reaction of the person on the basis of a human data height which is a value of a height direction included in the measured distance in a first period set by prompting the person to start and end hand raising,

wherein the situation estimation unit measures the human data height for each person to which the identifier is assigned by the tracking unit a plurality of times in the first period, extracts an identifier for which a plurality of human data heights are in the predetermined range, and estimates the reaction of the person corresponding to the extracted identifier.

2. The operation estimation apparatus according to claim 1, wherein the situation estimation unit calculates a hand-raising threshold value set on the basis of an average value or a maximum value of the human data heights in a second period in which no person performs the hand raising for each person and human physique data, extracts a person for which a number of times that the human data height exceeding the hand-raising threshold value is measured is greater than a predetermined number of times in the first period, and estimates the reaction of the extracted person.

3. The operation estimation apparatus according to claim 2, wherein the situation estimation unit determines the predetermined number of times on the basis of the number of times that the human data height exceeding the hand-raising threshold value is measured and information indicating a distribution associated with whether the hand raising is performed in the first period.

4. The operation estimation apparatus according to claim 1, wherein the human-body processing unit divides each of a plurality of predetermined ranges into two or more regions, selects a threshold value of a distribution of distances at which the person is detected on the basis of human physique data for each division region, and detects the person on the basis of the distribution of the distances which are measured.

5. The operation estimation apparatus according to claim 1,

wherein the distance measurement unit includes:

a plurality of distance measurement units configured to measure distances for different predetermined ranges, and

wherein the human-body processing unit includes;

a plurality of clustering units configured to generate human cluster data by clustering the distances measured by the distance measurement units; and

a cluster data integration unit configured to integrate the human cluster data generated by the plurality of clustering units.

6. The operation estimation apparatus according to claim 1,

wherein the distance measurement unit includes:

a plurality of distance measurement units configured to measure distances for different predetermined ranges,

wherein the human-body processing unit includes;

a ranging data integration unit configured to integrate the distances measured by the plurality of distance measurement units for two or more predetermined ranges; and

a clustering unit configured to classify the measured distances as a cluster on the basis of a result of integration by the ranging data integration unit and detect the person on the basis of the classified cluster, and

wherein the tracking unit assigns the identifier to each person detected by the clustering unit.

7. The operation estimation apparatus according to claim 1,

wherein the distance measurement unit includes:

a plurality of distance measurement units configured to measure distances for different predetermined ranges,

wherein the human-body processing unit includes;

a ranging data integration unit configured to integrate the distances measured by the plurality of distance measurement units for two or more predetermined ranges;

a dividing unit configured to re-divide each of two or more predetermined ranges which are integrated into m (m is an integer greater than or equal to

     2) regions which are equal to or different from predetermined regions in shape and number;

m clustering units configured to classify the measured distances included in each of the m regions as a cluster for each of the m regions and detect the person on the basis of the classified cluster; and

a cluster data integration unit configured to integrate results of detections by the m clustering units, and

wherein the tracking unit assigns the identifier to each person detected by the clustering unit on the basis of the integrated results.

8. The operation estimation apparatus according to claim 1, wherein the tracking unit detects the person at a time subsequent to a predetermined time on the basis of a position of human data corresponding to the person detected at the predetermined time.

9. The operation estimation apparatus according to claim 2,

wherein the second period is a period set by prompting the person to end the hand raising, and

wherein the situation estimation unit calculates an upper-limit threshold value for determining the hand raising on the basis of the human data height for each piece of human data corresponding to each person in the second period, extracts human data for which the number of times that the human data height greater than the hand-raising threshold value and less than the upper-limit threshold value is measured is greater than the predetermined number of times in a third period in which the human performs the hand raising, and estimates the reaction of the person corresponding to the extracted human data.

10. The operation estimation apparatus according to claim 1,

wherein the tracking unit generates virtual distance information indicating distance information virtually measured around a position of human data corresponding to the person detected at a predetermined time in information of the distance measured at a time subsequent to the predetermined time,

wherein a clustering unit which generates human cluster data by clustering the distances generates mixed distance information in which the virtual distance information is mixed with the information of the distance measured at the subsequent time,

wherein the human-body processing unit detects the person at the subsequent time on the basis of the mixed distance information, and

wherein the tracking unit assigns the identifier to each person detected by the clustering unit.

11. The operation estimation apparatus according to claim 10, wherein the tracking unit changes a density of the virtual distance information around the position of the human data corresponding to the person detected at the predetermined time according to a time period in which the person is in a predetermined range from the position.

12. The operation estimation apparatus according to claim 10, wherein the tracking unit scatters the virtual distance information around the human data detected at the predetermined time according to a time period in which the person is in a predetermined range from the position so that a center is dense.

13. The operation estimation apparatus according to claim 6,

wherein the human-body processing unit includes:

a density calculation unit configured to calculate a density with a peripheral cluster for each cluster on the basis of a distance between clusters classified by the clustering unit which generates human cluster data obtained by clustering the distances; and

a re-clustering unit configured to calculate a threshold value set on the basis of human physique data from a measurement value of a height included in a region of a cluster for each cluster with respect to the cluster in which the density calculated by the density calculation unit is low, extract a measurement point having a measurement value of the height greater than the calculated threshold value, and generate one or more small clusters smaller than the cluster among clusters by performing a re-clustering process on the extracted measurement point, and

wherein the situation estimation unit estimates the reaction of the person on the basis of the number of small clusters among the clusters.

14. The operation estimation apparatus according to claim 4, wherein the human-body processing unit selects physique data of a child as the human physique data for a region close to a robot which outputs a signal indicating a prompt for the person to start or end the hand raising for each division region.

15. The operation estimation apparatus according to claim 1, comprising:

a control unit configured to output a signal indicating a prompt for the person to start or end the hand raising and control behavior of the robot on the basis of the estimated reaction of the person.

16. A robot comprising:

the operation estimation apparatus according to claim 1;

a storage unit in which a signal output by the robot and an order of output are associated and stored; and

a control unit configured to output a signal indicating a prompt for the person to start or end the hand raising and control behavior of the robot on the basis of the reaction of the person estimated by the operation estimation apparatus.

17. An operation estimation method comprising:

a distance measurement procedure in which a distance measurement unit measures a distance for a predetermined range;

a human processing procedure in which a human-body processing unit detects a person on the basis of a distribution of distances measured in the distance measurement procedure;

a tracking procedure in which a tracking unit assigns an identifier to each person detected in the human processing procedure; and

a situation estimation procedure in which a situation estimation unit measures human data height for each person to which the identifier is assigned in the tracking procedure a plurality of times on the basis of the human data height which is a value of a height direction included in the measured distance in a first period set by prompting the person to start and end hand raising, extracts an identifier for which a plurality of human data heights are in the predetermined range, and estimates the reaction of the person corresponding to the extracted identifier.