Autonomous movement device, autonomous movement method and non-transitory recording medium

• US 10,296,009 B2
• Filed: 09/19/2016
• Issued: 05/21/2019
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Narikawa, Tetsuro
• Primary Examiner(s): Hutchinson, Alan D
• Application Number:
  US15/269,917
• Publication Number:
  US 20170176997A1
• Time in Patent Office: 974 Days
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

To quickly recover from a status in which a local device position estimation is unavailable. A movement processor of an autonomous movement device executes a drive process for a driver so as to move autonomously. A position estimator estimates the local device position. An estimation determiner determines whether or not the estimation result of the local device position by the position estimator is uniquely defined. A movement process controller controls the movement processor so as to move to a position where the estimation result of the local device position by the position estimator is uniquely defined when the estimation determiner determines that the estimation result is not defined uniquely.

Claims

1. An autonomous movement device comprising:

an image picker;

an image memory;

a position change history memory;

a controller; and

a driver,

wherein;

the image memory stores information on a plurality of images picked up by the image picker, and

the controller;

executes a drive process for the driver so as to move autonomously based on information on an image picked up by the image picker;

executes a local device position estimating process that estimates a local device position in a real-time manner, using the information on the plurality of images stored in the image memory;

determines whether or not an estimation result of the local device position is uniquely defined;

records, every time the local device position is changed by the drive process, an amount of change in position in the position change history memory; and

upon determination that the estimation result is not uniquely defined;

(i) executes a reverse-direction movement control that controls the drive process so as to move the autonomous movement device in a reverse direction to the amount of change in position recorded in the position change history memory such that the autonomous movement device moves to a position where the estimation result of the local device position is uniquely defined; and

(ii) by executing the local device position estimating process in a real-time manner during execution of the reverse-direction movement control, estimates the local device position using the information on the plurality of images picked up by the image picker during the execution of the reverse-direction movement control and stored in the image memory.

2. The autonomous movement device according to claim 1, further comprising a communicator that communicates with an external server, wherein at least one of the drive process, the estimation, the determination, and the control by the controller is executed by the external server via the communicator.

3. The autonomous movement device according to claim 1, wherein the controller further obtains corresponding feature points from the plurality of images, and determines that the estimation result is not defined uniquely when a number of the corresponding feature points obtained from the plurality of images utilized for the local device position estimation is less than a predetermined number.

4. The autonomous movement device according to claim 1, wherein the controller further obtains corresponding feature points from the plurality of images, estimates the local device position by minimizing a cost function that is defined based on a coordinate of the obtained feature point in the image, and determines that the estimation result is not defined uniquely when a minimum value of the cost function is greater than a predetermined value.

5. The autonomous movement device according to claim 1, wherein the controller further obtains corresponding feature points from the plurality of images, estimates the local device position by minimizing a cost function that is defined based on a coordinate of the obtained feature point in the image, and determines that the estimation result is not defined uniquely when the cost function includes a plurality of minimum values, and a value obtained by dividing a smallest minimum value from the plurality of minimum values by a second smallest minimum value from the plurality of minimum values is greater than a predetermined value.

6. The autonomous movement device according to claim 1, further comprising a position history memory;

wherein the controller records, upon determination that the estimation result is uniquely defined, information on the position at a time point of the determination that the estimation result is uniquely defined, together with clock time information indicating a clock time at the time point, in the position history memory, and controls, upon determination that the estimation result is not uniquely defined, the drive process so as to move in the reverse direction to the amount of change in position recorded in the position change history memory while tracking back to the time point at which the local device was located at the position recorded in the position history memory.

9. The autonomous movement device according to claim 1, wherein the controller causes the image picker to pick up the image while performing a rotation action, and controls the drive process so as to move toward a direction in which, among the images picked up by the image picker while the rotation action is performed, the image is picked up having largest information utilizable for the local device position estimation upon determination that the estimation result is not defined uniquely.

10. The autonomous movement device according to claim 9, wherein a rotation angle per a rotation action is an angle equal to or less than a half of a view angle of the image picker.

11. The autonomous movement device according to claim 1, where the image memory stores the information on the image picked up by the image picker every time the autonomous movement device moves by a reference translation distance.

12. The autonomous movement device according to claim 11, wherein a value obtained by multiplying a diameter of a wheel of the driver by a predetermined value is set as the reference translation distance.

13. The autonomous movement device according to claim 11, wherein the controller further extracts, from the plurality of images, a feature point where a position is estimatable, and sets a value obtained by dividing an average value of distances between all of the respective extracted feature points and the local device position by a predetermined value as the reference translation distance.

14. The autonomous movement device according to claim 1, wherein upon determination that the estimation result of the local device position estimated using the information on the plurality of images picked up by image picker during execution of the reverse-direction movement control and stored in the image memory is uniquely defined, the controller ends the reverse-direction movement control.

15. The autonomous movement device according to claim 1, wherein:

in a case in which the reverse-direction movement control is executed using all the amounts of change in position that are recorded in the position change history memory, the controller ends the reverse-direction movement control upon determination that the estimation result of the local device position estimating process executed during the execution of the reverse-direction movement control is uniquely defined, and the controller causes the image picker to pick up the image while performing a rotation action and executes a rotation movement control that controls the drive process to move the autonomous movement device in a direction in which the image picker is oriented at a time point of the picking up of an image that has the greatest amount of information usable for estimation of the local device position among the images picked up by the image picker during the rotation action, and

by executing the local device position estimating process during executing of the rotation movement control, estimates the local device position using the information on the plurality of images picked up by the image picker during the execution of the rotation movement control and stored in the image memory.

16. The autonomous movement device according to claim 1, wherein the controller executes the local device position estimating process during the execution of the reverse-direction movement control in a state in which the drive process is controlled to cause the autonomous movement device to autonomously move at a speed lower than a speed of autonomous movement at which the autonomous movement devices autonomously moves in a case in which the controller determines that the estimation result is uniquely defined.

7. An autonomous movement device comprising:

an image picker;

an image memory;

a position change history memory;

a position history memory;

a controller; and

a driver,

wherein;

the image memory stores information on a plurality of images picked up by the image picker, and

the controller;

executes a drive process for the driver so as to move autonomously based on information on an image picked up by the image picker;

executes a local device position estimating process that estimates a local device position in a real-time manner, using the information on the plurality of images stored in the image memory;

determines whether or not an estimation result of the local device position is uniquely defined;

records, every time the local device position is changed by the drive process, an amount of change in position in the position change history memory;

records, upon determination that the estimation result is uniquely defined, information on the position at a time point of the determination that the estimation result is uniquely defined, together with clock time information indicating a clock time at the time point, in the position history memory;

estimates a present local device position by cumulating, to the position recorded in the position history memory, the amount of change in position recorded in the position change history memory from the clock time corresponding to the position and up to a present clock time upon determination that the estimation result is not uniquely defined; and

upon determination that the estimation result is not uniquely defined;

(i) executes a return movement control that controls the drive process so as to move the autonomous movement device toward the position recorded in the position history memory from the position estimated by the cumulating such that the autonomous movement device moves to a position where the estimation result of the local device position is uniquely defined, and

(ii) by executing the local device position estimating process in a real-time manner during execution of the return movement control, estimates the local device position using the information on the plurality of images picked up by the image picker during execution of the return movement control and stored in the image memory.

8. The autonomous movement device according to claim 7, wherein upon determination that the estimation result of the local device position estimated using the information on the plurality of images picked up by image picker during execution of the return movement control and stored in the image memory is uniquely defined, the controller ends the return movement control.

17. An autonomous movement method comprising:

a movement process step of executing a drive process for a driver so as to move autonomously based on information on an image picked up by an image picker;

an image storing step of storing information on a plurality of images picked up by the image picker;

a position estimating step of estimating a local device position in a real-time manner, using the information on the plurality of images stored in the image storing step;

an estimation determining step of determining whether or not an estimation result of the local device position in the position estimation step is uniquely defined;

a recording step of recording, every time the local device position is changed in the movement process step, an amount of change in position in a position change history memory; and

upon determination in the estimation determining step that the estimation result is not uniquely defined;

(i) a movement process control step of controlling the drive process in the movement process step so as to move the autonomous movement device in a reverse direction to the amount of change in position recorded in the position change history memory such that the autonomous movement device moves to a position where the estimation result of the local device position in the position estimation step is uniquely defined; and

(ii) a movement and estimation step of estimating, by executing the position estimating step in a real-time manner during the movement process control step, the local device position using the information on the plurality of images that are picked up by the image picker during execution of the movement process control step and are stored in the image storing step.

18. A non-transitory recording medium having stored therein a program that causes a computer to execute:

a movement process step of executing a drive process for a driver so as to move autonomously based on information on an image picked up by an image picker;

an image storing step of storing information on a plurality of images picked up by the image picker;

a position estimating step of estimating a local device position in a real-time manner, using the information on the plurality of images stored in the image storing step;

an estimation determining step of determining whether or not an estimation result of the local device position in the position estimation step is uniquely defined;

a recording step of recording, every time the local device position is changed in the movement process step, an amount of change in position in a position change history memory; and

upon determination in the estimation determining step that the estimation result is not uniquely defined;

(i) a movement process control step of controlling the drive process in the movement process step so as to move the autonomous movement device in a reverse direction to the amount of change in position recorded in the position change history memory such that the autonomous movement device moves to a position where the estimation result of the local device position in the position estimation step is uniquely defined; and

(ii) a movement and estimation step of estimating, by executing the position estimating step in a real-time manner during the movement process control step, the local device position using the information on the plurality of images that are picked up by the image picker during execution of the movement process control step and are stored in the image storing step.

19. An autonomous movement method comprising:

a movement process step of executing a drive process for a driver so as to move autonomously based on information on an image picked up by an image picker;

an image storing step of storing information on a plurality of images picked up by the image picker;

a position estimating step of estimating a local device position in a real-time manner, using the information on the plurality of images stored in the image storing step;

an estimation determining step of determining whether or not an estimation result of the local device position in the position estimation step is uniquely defined;

a position change history recording step of recording, every time the local device position is changed in the movement process step, an amount of change in position in a position change history memory;

a position history recording step of recording, upon determination that the estimation result is uniquely defined in the estimation determining step, information on the position at a time point of the determination that the estimation result is uniquely defined, together with clock time information indicating a clock time at the time point, in a position history memory;

a cumulated position estimating step of estimating a present local device position by cumulating, to the position recorded in the position history memory, the amount of change in position recorded in the position change history memory from the clock time corresponding to the position and up to a present clock time upon determination that the estimation result is not uniquely defined in the estimation determining step; and

upon determination in the estimation determining step that the estimation result is not uniquely defined;

(i) a movement process control step of controlling the drive process in the movement process step so as to move the autonomous movement device toward the position recorded in the position history memory from the position estimated in the cumulated position estimating step, such that the autonomous movement device moves to a position where the estimation result of the local device position obtained in the position estimating step is uniquely defined; and

(ii) a movement and estimation step of estimating, by executing the position estimating step in a real-time manner during the movement process control step, the local device position using the information on the plurality of images that are picked up by the image picker during execution of the movement process control step and are stored in the image storing step.

20. A non-transitory recording medium having stored therein a program that causes a computer to execute:

a movement process step of executing a drive process for a driver so as to move autonomously based on information on an image picked up by an image picker;

an image storing step of storing information on a plurality of images picked up by the image picker;

a position estimating step of estimating a local device position in a real-time manner, using the information on the plurality of images stored in the image storing step;

an estimation determining step of determining whether or not an estimation result of the local device position in the position estimation step is uniquely defined;

a position change history recording step of recording, every time the local device position is changed in the movement process step, an amount of change in position in a position change history memory;

a position history recording step of recording, upon determination that the estimation result is uniquely defined in the estimation determining step, information on the position at a time point of the determination that the estimation result is uniquely defined, together with clock time information indicating a clock time at the time point, in a position history memory;

a cumulated position estimating step of estimating a present local device position by cumulating, to the position recorded in the position history memory, the amount of change in position recorded in the position change history memory from the clock time corresponding to the position and up to a present clock time upon determination that the estimation result is not uniquely defined in the estimation determining step; and

upon determination in the estimation determining step that the estimation result is not uniquely defined;

(i) a movement process control step of controlling the drive process in the movement process step so as to move the autonomous movement device toward the position recorded in the position history memory from the position estimated in the cumulated position estimating step such that the autonomous movement device moves to a position where the estimation result of the local device position obtained in the position estimating step is uniquely defined; and

(ii) a movement and estimation step of estimating, by executing the position estimating step in a real-time manner during the movement process control step, the local device position using the information on the plurality of images that are picked up by the image picker during execution of the movement process control step and are stored in the image storing step.