CONTROL APPARATUS, OBJECT DETECTION SYSTEM, OBJECT DETECTION METHOD AND PROGRAM

• US 20190318498A1
• Filed: 03/20/2019
• Published: 10/17/2019
• Est. Priority Date: 04/17/2018
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: YOSHIKAWA, Kei, YOSHIZAWA, Shintaro, MORI, Yuto
• Granted Patent: US 10,957,067 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

A control apparatus capable of efficiently detecting a target object even when the target object is shielded by other objects is provided. An object recognition unit 114 recognizes a target object 80 present in a 3D environment 4 by using measurement data acquired from a sensor 12. An information generation unit 116 generates 3D environmental information by integrating a plurality of measurement data. A position determination unit 120 determines an optimal position of the sensor 12 for performing the next measurement. A sensor control unit 140 moves the sensor 12 to the determined optimal position. The position determination unit 120 determines, by using the 3D environmental information, a position of the sensor 12 where the sensor 12 can take an image in which a size of an area shielded by at least one first object is larger as the optimal position.

Claims

1. A control apparatus configured to detect a target object to be detected by controlling a sensor configured to measure surroundings in a three-dimensional (3D) environment, comprising:

hardware, including at least one memory configured to store a computer program and at least one processor configured to execute the computer program;

an object recognition unit, implemented by the hardware, configured to recognize the target object present in the 3D environment by using measurement data acquired from the sensor;

an information generation unit, implemented by the hardware, configured to generate 3D environmental information indicating each point on an object present in the 3D environment by integrating a plurality of measurement data;

a position determination unit, implemented by the hardware, configured to determine an optimal position of the sensor for performing next measurement; and

a sensor control unit, implemented by the hardware, configured to move the sensor to the determined optimal position, wherein

the position determination unit determines, by using the 3D environmental information, a position of the sensor where the sensor can take an image in which a size of an area shielded by at least one first object is larger as the optimal position, the at least one first object being the target object recognized by the object recognition unit.

2. The control apparatus according to claim 1, wherein the position determination unit calculates, as a candidate for the optimal position, a position of the sensor where the sensor can measure a placement available area according to a position and a shape of a storage object in which the target object can be placed, and selects the optimal position from the candidate, the placement available area being an area where the target object can be placed.

3. The control apparatus according to claim 2, wherein the position determination unit determines whether or not, among candidate positions, there is an unmeasured position from which the sensor can measure the area shielded by the first object but has not performed measurement yet, and when it is determined that there is the unmeasured position, performs a process for determining the optimal position.

4. The control apparatus according to claim 3, further comprising a removal determination unit configured to, when it is determined that there is no unmeasured position, determine the first object to be removed so that the area shielded by the first object can be measured.

5. The control apparatus according to claim 4, further comprising an arm control unit configured to control an arm so that the determined first object is removed.

6. The control apparatus according to claim 1, wherein the position determination unit determines whether or not a search of the placement available area, which is the area where the target object can be placed, has been completed by using the 3D environmental information, and when it is determined that the search has not been completed, performs a process for determining the optimal position.

7. An object detection system comprising:

a sensor configured to measure surroundings in a 3D environment; and

a control apparatus configured to detect a target object to be detected by controlling the sensor,

wherein the control apparatus comprises;

hardware, including at least one memory configured to store a computer program and at least one processor configured to execute the computer program;

an object recognition unit, implemented by the hardware, configured to recognize the target object present in the 3D environment by using measurement data acquired from the sensor;

an information generation unit, implemented by the hardware, configured to generate 3D environmental information indicating each point on an object present in the 3D environment by integrating a plurality of measurement data;

a position determination unit, implemented by the hardware, configured to determine an optimal position of the sensor for performing next measurement; and

a sensor control unit, implemented by the hardware, configured to move the sensor to the determined optimal position, and

wherein the position determination unit determines, by using the 3D environmental information, a position of the sensor where the sensor can take an image in which a size of an area shielded by at least one first object is larger as the optimal position, the at least one first object being the target object recognized by the object recognition unit.

8. An object detection method for detecting a target object to be detected by controlling a sensor configured to measure surroundings in a 3D environment, comprising:

recognizing the target object present in the 3D environment by using measurement data acquired from the sensor;

generating 3D environmental information indicating each point on an object present in the 3D environment by integrating a plurality of measurement data;

determining, by using the 3D environmental information, a position of the sensor where the sensor can take an image in which a size of an area shielded by at least one first object is larger as an optimal position of the sensor for performing next measurement, the at least one first object being the recognized target object; and

moving the sensor to the determined optimal position.

9. A non-transitory computer readable medium storing a program for performing an object detection method in which a target object to be detected is detect by controlling a sensor configured to measure surroundings in a 3D environment, the program being adapted to cause a computer to perform:

recognizing the target object present in the 3D environment by using measurement data acquired from the sensor;

generating 3D environmental information indicating each point on an object present in the 3D environment by integrating a plurality of measurement data;

determining, by using the 3D environmental information, a position of the sensor where the sensor can take an image in which a size of an area shielded by at least one first object is larger as an optimal position of the sensor for performing next measurement, the at least one first object being the recognized target object; and

moving the sensor to the determined optimal position.