Moving object controller, moving object control method, and integrated circuit

• US 9,958,868 B2
• Filed: 03/22/2016
• Issued: 05/01/2018
• Est. Priority Date: 03/23/2015
• Status: Active Grant

• Current Assignee: Kyushu Institute of Technology, Megachips Corporation
• Original Assignee: Kyushu Institute of Technology, Megachips Corporation
• Inventors: Shingu, Fumiya, Ikoma, Norikazu, Nagamine, Kenta, Hasegawa, Hiromu
• Primary Examiner(s): BUTLER, RODNEY ALLEN
• Application Number:
  US15/076,813
• Publication Number:
  US 20160282876A1
• Time in Patent Office: 770 Days
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A moving object controller efficiently generates an environmental map and performs highly accurate state estimation in a short time to appropriately control a moving object. An observation obtaining unit obtains observation data from an observable event. A landmark prediction unit generates a landmark prediction signal including predictive information about a landmark at a current time. A landmark detection unit detects information about the landmark at the current time, and generates a landmark detection signal indicating the detection result. A state estimation unit estimates an internal state of the moving object to obtain data indicating an estimated internal state of the moving object at the current time, and estimates the environmental map based on the landmark detection signal to obtain data indicating an estimated environmental map at the current time.

Claims

1. A moving object controller for controlling a moving object while performing processing for generating an environmental map expressed using information about a landmark and performing processing for estimating an internal state of the moving object, the controller comprising:

observation obtaining circuitry configured to obtain observation data from an observable event, the observation data being image data obtained by the moving object capturing an environment surrounding the moving object;

landmark prediction circuitry configured to generate a landmark prediction signal including predictive information about a landmark at a current time based on internal state data indicating an internal state of the moving object at a time preceding the current time, the internal state of the moving object being obtained by using at least one of (1) a state vector indicating an internal state for a position and an angle of the moving object and (2) variance-covariance matrix data obtained based on the position of the moving object, and environmental map data at the time preceding the current time;

landmark detection circuitry configured to detect information about the landmark at the current time based on the landmark prediction signal generated by the landmark prediction circuitry, and generate a landmark detection signal indicating a result of the detection; and

state estimation circuitry configured to estimate the internal state of the moving object based on the observation data obtained by the observation obtaining circuitry and the landmark detection signal to obtain data indicating an estimation internal state internal state of the moving object at the current time, and estimate the environmental map based on the landmark detection signal to obtain data indicating an estimation environmental map at the current time.

2. The moving object controller according to claim 1, wherein

the landmark prediction circuitry calculates a predictive image area that is predicted to contain the landmark in an image formed using the image data obtained at the current time based on the internal state data indicating the internal state of the moving object at the time preceding the current time and the environmental map data at the time preceding the current time, and generates the landmark prediction signal containing information about the calculated predictive image area, and

the landmark detection circuitry performs local search in which the predictive image area of the image formed using the image data at the current time is searched to detect an image area corresponding to the landmark, and the local search is started at an initial position that is a pixel position included in a central area of the predictive image area calculated by the landmark prediction circuitry and is sequentially performed at positions away from the initial position.

3. The moving object controller according to claim 2, wherein

the landmark prediction circuitry generates the landmark prediction signal including information about estimation accuracy of information about a position of the landmark obtained at the time preceding the current time, and

the landmark detection circuitry performs global search in which the image formed using the image data at the current time is searched to detect the image area corresponding to the landmark when the estimation accuracy of the positional information about the landmark obtained at the time preceding the current time is lower than a predetermined value based on the landmark prediction signal.

4. The moving object controller according to claim 2, wherein

the landmark prediction circuitry generates the landmark prediction signal including information about estimation accuracy of the internal state data of the moving object obtained at the time preceding the current time, and

the landmark detection circuitry performs global search in which the image formed using the image data at the current time is searched to detect the image area corresponding to the landmark when the estimation accuracy of the internal state data of the moving object obtained at the time preceding the current time is lower than a predetermined value based on the landmark prediction signal.

5. The moving object controller according to claim 2, wherein

the landmark detection circuitry includes detection information obtaining circuitry configured to store a value indicating the number of times the landmark has been detected in searching for the landmark within a predetermined period of time, and

the landmark detection circuitry performs the local search when the value indicating the number of times the landmark has been detected, the number being stored in the detection information obtaining circuitry, is greater than a predetermined value.

6. The moving object controller according to claim 2, wherein

the landmark detection circuitry includes detection information obtaining circuitry configured to store landmark detection information indicating whether the landmark has been detected in searching for the landmark within a predetermined period of time, and

the landmark detection circuitry performs global search in which the image formed using the image data at the current time is searched to detect the image area corresponding to the landmark when the landmark has not been detected within the predetermined period of time based on the landmark detection information stored by the detection information obtaining circuitry.

7. A moving object control method for controlling a moving object while performing processing for generating an environmental map expressed using information about a landmark and performing processing for estimating an internal state of the moving object, the method comprising:

obtaining observation data from an observable event, the observation data being image data obtained by the moving object capturing an environment surrounding the moving object;

generating a landmark prediction signal including predictive information about a landmark at a current time based on internal state data indicating an internal state of the moving object at a time preceding the current time, the internal state of the moving object being obtained by using at least one of (1) a state vector indicating an internal state for a position and an angle of the moving object and (2) variance-covariance matrix data obtained based on the position of the moving object, and environmental map data at the time preceding the current time;

detecting information about the landmark at the current time based on the landmark prediction signal generated by the step of generating the landmark prediction signal, and generating a landmark detection signal indicating a result of the detection; and

estimating the internal state of the moving object based on the observation data obtained by the step of obtaining the observation data and the landmark detection signal to obtain data indicating an estimation internal state of the moving object at the current time, and estimating the environmental map based on the landmark detection signal to obtain data indicating an estimation environmental map at the current time.

8. An integrated circuit used in a moving object controller for controlling a moving object while performing processing for generating an environmental map expressed using information about a landmark and performing processing for estimating an internal state of the moving object, the integrated circuit comprising:

observation obtaining circuitry configured to obtain observation data from an observable event, the observation data being image data obtained by the moving object capturing an environment surrounding the moving object;

landmark prediction circuitry configured to generate a landmark prediction signal including predictive information about a landmark at a current time based on internal state data indicating an internal state of the moving object at a time preceding the current time, the internal state of the moving object being obtained by using at least one of (1) a state vector indicating an internal state for a position and an angle of the moving object and (2) variance-covariance matrix data obtained based on the position of the moving object, and environmental map data at the time preceding the current time;

landmark detection circuitry configured to detect information about the landmark at the current time based on the landmark prediction signal generated by the landmark prediction circuitry, and generate a landmark detection signal indicating a result of the detection; and

state estimation circuitry configured to estimate the internal state of the moving object based on the observation data obtained by the observation obtaining circuitry and the landmark detection signal to obtain data indicating an estimation internal state of the moving object at the current time, and estimate the environmental map based on the landmark detection signal to obtain data indicating an estimation environmental map at the current time.