Control method and system, and mobile robot using the same

• US 10,513,037 B2
• Filed: 09/27/2018
• Issued: 12/24/2019
• Est. Priority Date: 12/15/2017
• Status: Active Grant

• Current Assignee: Ankobot (Shanghai) Smart Technologies Co., Ltd.
• Original Assignee: Ankobot (Shanghai) Smart Technologies Co., Ltd.
• Inventors: Hou, Xiru
• Primary Examiner(s): Oh, Harry Y
• Application Number:
  US16/144,129
• Publication Number:
  US 20190184571A1
• Time in Patent Office: 453 Days
• Field of Search: 700259
• CPC Class Codes: A47L 1/00 Cleaning windows
• CPC Class Codes: A47L 11/4011 Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
• CPC Class Codes: A47L 11/4061 Steering means; Means for avoiding obstacles; Details related to the place where the driver is accommodated
• CPC Class Codes: A47L 2201/04 Automatic control of the travelling movement; Automatic obstacle detection
• CPC Class Codes: A47L 9/2805 Parameters or conditions being sensed
• CPC Class Codes: A47L 9/2852 Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles
• CPC Class Codes: A47L 9/2889 Safety or protection devices or systems, e.g. for prevention of motor over-heating or for protection of the user
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/0003 Home robots, i.e. small robots for domestic use
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means

Abstract

The present application provides a control method and system, and mobile robot using the same. The mobile robot comprises: a storage device, configured to store a simultaneous localization and mapping application and a behavior control application; an image acquisition device, configured to acquire images of an operating environment; a processing device, configured to control the image acquisition device to capture pictures in order to acquire an image containing the ground, and invoke the simultaneous localization and mapping application and the behavior control application to control the behaviors of the mobile robot when identified that the image contains flexible obstacle; and a movement system, configured to drive the mobile robot to move based on a control instruction output by the processing device. In the present application, the flexible obstacle can be detected effectively, and the behaviors of the mobile robot can be controlled correspondingly based on detection results.

Claims

1. A method for controlling a mobile robot, wherein the mobile robot comprises a processing device and an image acquisition device, and the method is performed by the processing device and comprises the following steps:

under an operating mode of the mobile robot, controlling the image acquisition device to capture pictures in order to acquire images containing the ground; and

identifying by a flexible obstacle image classifier, at least one image containing the ground captured by the image acquisition device, and controlling behaviors of the mobile robot when identified that the at least one image contains a flexible obstacle;

wherein, a manner of controlling behaviors of the mobile robot comprises;

acquiring positions of matching features in at least two images, and determining a position and a pose of the mobile robot based on a corresponding relationship between an image coordinate system and a physical space coordinate system and based on the positions of the matching features, wherein, the at least two images are an image at a previous time and an image at the current time respectively;

acquiring at least one image, determining a position of the flexible obstacle in the at least one image based on the positions of features in the at least one image, and determining a size information of the flexible obstacle based on a standard measure in the at least one image; and

controlling behaviors of the mobile robot based on the determined position and pose of the mobile robot and based on the position and the size information of the flexible obstacle.

2. The method for controlling a mobile robot of claim 1, wherein, a manner of acquiring the positions of matching features in at least two images comprise tracking positions of corresponding features contained in at least two images, wherein, the corresponding features are a pair of matching features.

3. The method for controlling a mobile robot of claim 1, wherein, a manner of determining a size information of the flexible obstacle based on a standard measure in the at least one image comprises:

identifying a reference object with a known size in the at least one image; and

calculating the size information of the flexible obstacle in the at least one image based on the size of the reference object.

4. The method for controlling a mobile robot of claim 1, further comprising a step of acquiring movement information of the mobile robot.

5. The method for controlling a mobile robot of claim 1, wherein, the step of controlling behaviors of the mobile robot comprises any one of the following steps:

controlling the mobile robot to move according to an original navigation route and cross the flexible obstacle;

controlling the mobile robot to modify the original navigation route, move according to the modified navigation route and cross the flexible obstacle;

controlling the mobile robot to modify the original navigation route, move according to the modified navigation route to avoid the flexible obstacle.

6. The method for controlling a mobile robot of claim 1, wherein, the step of controlling behaviors of the mobile robot comprises:

controlling the mobile robot to stop moving.

7. The method for controlling a mobile robot of claim 5, further comprising a step of controlling the mobile robot to send alarm information.

8. A mobile robot, comprising:

a storage device, configured to store a simultaneous localization and mapping application and a behavior control application;

an image acquisition device, configured to acquire images of an operating environment under an operating mode of the mobile robot;

a processing device, connected with the storage device and the image acquisition device, and configured to control the image acquisition device to capture pictures in order to acquire at least one image containing the ground under the operating mode of the mobile robot, identify the at least one image containing the ground by a flexible obstacle image classifier, and invoke the simultaneous localization and mapping application and the behavior control application from the storage device to control behaviors of the mobile robot when identified that the image contains a flexible obstacle; and

a movement system, connected with the processing device, and configured to drive the mobile robot to move based on a control instruction output by the processing device;

wherein, the processing device is configured to invoke the simultaneous localization and mapping application and the behavior control application from the storage device to perform the following steps;

acquiring positions of matching features in at least two images, and determining a position and a pose of the mobile robot based on a corresponding relationship between an image coordinate system and a physical space coordinate system and based on the positions of the matching features, wherein, the at least two images are an image at a previous time and an image at the current time respectively;

acquiring at least one image, determining a position of the flexible obstacle in the at least one image based on the positions of features in the at least one image, and determining a size information of the flexible obstacle based on a standard measure in the at least one image; and

controlling behaviors of the mobile robot based on the determined position and pose of the mobile robot and based on the position and the size information of the flexible obstacle.

9. The mobile robot of claim 8, wherein, the image acquisition device is arranged at the top surface of the mobile robot, or at a side surface of the mobile robot, or at the junction of the top surface and a side surface of the mobile robot.

10. The mobile robot of claim 8, further comprising a tracking device, which is connected with the image acquisition device and configured to track positions of corresponding features contained in the at least two images to acquire positions of matching features in the at least two images, wherein, the corresponding features are a pair of matching features.

11. The mobile robot of claim 10, further comprising a movement sensing device which is connected with the processing device and configured to acquire movement information of the mobile robot.

12. The mobile robot of claim 8, wherein, a manner of invoking the simultaneous localization and mapping application and the behavior control application from the storage device to control behaviors of the mobile robot comprises any of the following ways:

sending the control instruction to the movement system to control the mobile robot to move according to an original navigation route and cross the flexible obstacle;

sending the control instruction to the movement system to control the mobile robot to modify the original navigation route, move according to the modified navigation route and cross the flexible obstacle; and

sending the control instruction to the movement system to control the mobile robot to modify the original navigation route, move according to the modified navigation route to avoid the flexible obstacle.

13. The mobile robot of claim 8, wherein, the step of invoking the simultaneous localization and mapping application and the behavior control application from the storage device to control behaviors of the mobile robot comprises:

sending the control instruction to the movement system to control the mobile robot to stop moving.

14. The mobile robot of claim 8, further comprising an alarm device which is connected with the processing device and configured to output an alarm information when the processing device identified that the image contains a flexible obstacle.

15. The mobile robot of claim 8, further comprising a cleaning system which is connected with the control system and configured to clean the ground during the movement of the mobile robot.

16. A system for controlling a mobile robot, wherein the mobile robot is configured with an image acquisition device, and the system comprises:

a storage device, configured to store a simultaneous localization and mapping application and a behavior control application; and

a processing device, connected with the storage device and the image acquisition device, and configured to control the image acquisition device to capture pictures in order to acquire at least one image containing the ground under an operating mode of the mobile robot, identify the at least one image containing the ground by a flexible obstacle image classifier, and invoke the simultaneous localization and mapping application and the behavior control application from the storage device to control behaviors of the mobile robot when identified that the image contains a flexible obstacle

wherein, the processing device is configured to invoke the simultaneous localization and mapping application and the behavior control application from the storage device to perform the following steps;

acquiring positions of matching features in at least two images, and determining a position and a pose of the mobile robot based on a corresponding relationship between an image coordinate system and a physical space coordinate system and based on the positions of the matching features, wherein, the at least two images are an image at a previous time and an image at the current time respectively;

acquiring at least one image, determining a position of the flexible obstacle in the at least one image based on the positions of features in the at least one image, and determining a size information of the flexible obstacle based on a standard measure in the at least one image; and

controlling behaviors of the mobile robot based on the determined position and pose of the mobile robot and based on the position and the size information of the flexible obstacle.