Autonomous movement device, autonomous movement method, and non-transitory recording medium

• US 10,296,002 B2
• Filed: 11/23/2016
• Issued: 05/21/2019
• Est. Priority Date: 03/14/2016
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Maeno, Yasushi
• Primary Examiner(s): Worden, Thomas E
• Application Number:
  US15/359,689
• Publication Number:
  US 20170261985A1
• Time in Patent Office: 909 Days
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30

Abstract

The movement direction-and-speed determiner of an autonomous movement device determines the direction and speed for an autonomous movement based on information on the plurality of picked-up images by an image picker. A feature point obtainer obtains a feature point from the picked-up image by the image picker. A map memory stores the position of the obtained feature point by the feature point obtainer. A corresponding feature-point number obtainer obtains the number of correspondences between the obtained feature point by the feature point obtainer and the feature point having the position stored in the map memory. When the obtained number of corresponding feature points by the corresponding feature-point number obtainer is equal to or smaller than a threshold, in order to increase the number of corresponding feature points, a moving direction-and-speed changer changes the moving direction and speed to those determined by the movement direction-and-speed determiner.

Claims

1. An autonomous movement device comprising:

a map memory;

a hardware processor which is configured to;

control an autonomous movement of the autonomous movement device based on information on a plurality of images picked up by a camera;

obtain a feature point in an image among the plurality of images picked up by the camera by analyzing the image;

store, in the map memory, a position of the obtained feature point; and

obtain a corresponding feature point number that is a number of correspondences between a presently obtained feature point and a previously obtained feature point whose position is stored in the map memory; and

an image memory sequentially storing an image picked up by the camera,

wherein the hardware processor is further configured to estimate a local device position of the autonomous movement device based on the position of the feature point stored in the map memory,

wherein the hardware processor determines whether an estimation status of the local device position is a tracking successful status or a tracking unsuccessful status based on the obtained corresponding feature point number,

wherein the hardware processor obtains a corresponding feature point number by referring to a predetermined image among a plurality of images sequentially stored in the image memory when the estimation status of the local device position is determined to be in the tracking successful status, and when the estimation status of the local device position is determined to be in the tracking unsuccessful status, the hardware processor obtains a corresponding feature point number by referring to a previous image which is an image stored prior to the predetermined image among the plurality of images sequentially stored in the image memory, and

wherein the hardware processor controls the autonomous movement in accordance with the obtained corresponding feature point number.

2. The autonomous movement device according to claim 1, wherein:

the hardware processor obtains a corresponding feature point number by referring to a feature point in a neighboring image which is an image having a predetermined relationship with one of the predetermined image and the previous image, among the plurality of images sequentially stored in the image memory, when the obtained corresponding feature point number is equal to or larger than a predetermined reference corresponding feature point number; and

when the corresponding feature point number, obtained by referring to the neighboring image, is equal to or larger than a predetermined second reference corresponding feature point number, the hardware processor determines that the position of the local device is in the tracking successful status and stores the corresponding feature point number and the position of the feature point in the map memory.

3. The autonomous movement device according to claim 2, wherein the hardware processor changes a movement direction when the obtained corresponding feature point number is equal to or less than a movement change threshold which is larger than the second reference corresponding feature point number.

4. The autonomous movement device according to claim 2, wherein the hardware processor reduces a movement speed when the obtained corresponding feature point number is equal to or less than a movement change threshold which is larger than the second reference corresponding feature point number.

5. The autonomous movement device according to claim 3, wherein the hardware processor sets the movement speed to zero for a predetermined time, when the obtained corresponding feature point number is equal to or less than the second reference corresponding feature point number.

6. The autonomous movement device according to claim 5, further comprising:

a movement record memory storing a past movement record,

wherein, when the obtained corresponding feature point number is equal to or less than the second reference corresponding feature point number, even after the predetermined time has elapsed after having set the movement speed to zero, the hardware processor sets a movement direction based on the movement record stored in the movement record memory so as to return to a past position.

7. The autonomous movement device according to claim 1, wherein:

the hardware processor is configured to obtain a surrounding brightness in a surrounding environment; and

the hardware processor reduces the moving speed or stops the autonomous movement device when the surrounding brightness is darker than a predetermined brightness.

8. The autonomous movement device according to claim 6, wherein:

the hardware processor is configured to obtain a destination for a movement; and

the hardware processor sets the movement direction so as to move toward the obtained destination, after the autonomous movement device has returned to the past position.

9. The autonomous movement device according to claim 8, wherein the hardware processor is configured to, after the autonomous movement device has returned to the past position from a first position, and then moved toward the obtained destination, when the obtained corresponding feature point number becomes equal to or smaller than the reference corresponding feature point number, change the obtained destination before the autonomous movement device has moved beyond the first position.

10. The autonomous movement device according to claim 1, wherein the hardware processor is configured to:

select a plurality of destination candidates;

obtain the feature point from images in directions of the selected destination candidates; and

set a destination to the destination candidate in which a number of the obtained feature points is maximum among the plurality of destination candidates.

11. An autonomous movement method, comprising:

controlling an autonomous movement based on information on a plurality of images picked up by a camera;

obtaining a feature point in an image among the plurality of images picked up by the camera by analyzing the image;

storing, in a map memory, a position of the obtained feature point;

obtaining a corresponding feature point number that is a number of correspondences between a presently obtained feature point and a previously obtained feature point whose position is stored in the map memory;

sequentially storing an image picked up by the camera in an image memory;

estimating a local device position based on the position of the feature point stored in the map memory;

determining whether an estimation status of the local device position is a tracking successful status or a tracking unsuccessful status based on the obtained corresponding feature point number;

obtaining a corresponding feature point number by referring to a predetermined image among a plurality of images sequentially stored in the image memory when the estimation status of the local device position is determined to be in the tracking successful status, and obtaining a corresponding feature point number by referring to a previous image which is an image stored prior to the predetermined image among the plurality of images sequentially stored in the image memory when the estimation status of the local device position is determined to be in the tracking unsuccessful status; and

controlling the autonomous movement in accordance with the obtained corresponding feature point number.

12. A nontransitory computer readable storage medium having stored thereon a program that, when executed, cause one or more processors to perform functions comprising:

controlling an autonomous movement based on information on a plurality of images picked up by a camera;

obtaining a feature point in an image among the plurality of images picked up by the camera by analyzing the image;

storing, in a map memory, a position of the obtained feature point;

obtaining a corresponding feature point number that is a number of correspondences between a presently obtained feature point and a previously obtained feature point whose position is stored in the map memory;

sequentially storing an image picked up by the camera in an image memory;

estimating a local device position based on the position of the feature point stored in the map memory;

determining whether an estimation status of the local device position is a tracking successful status or a tracking unsuccessful status based on the obtained corresponding feature point number;

obtaining a corresponding feature point number by referring to a predetermined image among a plurality of images sequentially stored in the image memory when the estimation status of the local device position is determined to be in the tracking successful status, and obtaining a corresponding feature point number by referring to a previous image which is an image stored prior to the predetermined image among the plurality of images sequentially stored in the image memory when the estimation status of the local device position is determined to be in the tracking unsuccessful status; and

controlling the autonomous movement in accordance with the obtained corresponding feature point number.