Moving robot and control method thereof

• US 11,330,948 B2
• Filed: 11/21/2017
• Issued: 05/17/2022
• Est. Priority Date: 11/24/2016
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Noh, Dongki, Baek, Seungmyun, Baek, Seungmin, Lee, Sunghun
• Primary Examiner(s): Nguyen, Kira
• Application Number:
  US16/463,785
• Publication Number:
  US 20190320867A1
• Time in Patent Office: 1,638 Days
• CPC Class Codes: A47L 11/4011 Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
• CPC Class Codes: A47L 11/4061 Steering means; Means for avoiding obstacles; Details related to the place where the driver is accommodated
• CPC Class Codes: A47L 2201/04 Automatic control of the travelling movement; Automatic obstacle detection
• CPC Class Codes: A47L 9/2805 Parameters or conditions being sensed
• CPC Class Codes: A47L 9/2852 Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles
• CPC Class Codes: B25J 11/00 Manipulators not otherwise provided for
• CPC Class Codes: B25J 11/0085 Cleaning
• CPC Class Codes: B25J 19/02 Sensing devices
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 19/06 Safety devices
• CPC Class Codes: B25J 19/061 with audible signals audio controls B25J13/003
• CPC Class Codes: B25J 9/00 Programme-controlled manipulators
• CPC Class Codes: B25J 9/0003 Home robots, i.e. small robots for domestic use
• CPC Class Codes: B25J 9/16 Programme controls programme controls in general G05B19/00, e.g. numerical pogramme controls G05B19/18; recording or playback systems G05B19/42
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1674 characterised by safety, monitoring, diagnostic
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

Disclosed is a moving robot including: a travel unit configured to move a body; an image acquisition unit configured to acquire a surrounding image of the body; a sensor unit having one or more sensors configured to detect an obstacle while the body moves; a controller configured to: upon detection of an obstacle by the sensor unit, recognize an attribute of the obstacle based on an image acquired by the image acquisition unit, and control driving of the travel unit based on the attribute of the obstacle; and a sound output unit configured to: output preset sound when the recognized attribute of the obstacle indicates a movable obstacle. Accordingly, the moving robot improves stability, user convenience, driving efficiency, and cleaning efficiency.

Claims

1. A moving robot comprising:

a travel unit configured to move a body;

an image acquisition unit configured to acquire a surrounding image of the body;

a sensor unit having one or more sensors configured to detect an obstacle while the body moves;

a controller configured to;

upon detection of an obstacle by the sensor unit, recognize an attribute of the obstacle based on an image acquired by the image acquisition unit, and control driving of the travel unit based on the recognized attribute of the obstacle;

a sound output unit configured to;

output preset sound when the recognized attribute of the obstacle indicates a movable obstacle; and

a storage unit configured to store a plurality of continuous images acquired by the image acquisition unit;

wherein the controller is further configured to;

upon detection of an obstacle by the sensor unit,

based on a moving direction and a speed of the body, select an image captured at a particular point in time before the obstacle is detected by the sensor from among the plurality of continuous images;

based on data that is pre-learned using machine learning, recognize an attribute of an obstacle included in the image captured at the particular point in time.

2. The moving robot of claim 1, wherein the movable obstacle is part of a human body or a pet.

3. The moving robot of claim 1, wherein the controller is further configured to stop moving the body when the recognized attribute of the obstacle indicates a movable obstacle.

4. The moving robot of claim 3, wherein the controller is further configured to resume previous movement when movement of the movable object is detected within a particular standby time.

5. The moving robot of claim 3, wherein the controller is further configured to perform avoidance driving with respect to the movable object when movement of the movable object is not detected within the particular standby time.

6. The moving robot of claim 3, wherein the controller is further configured to control the body to move forward and backward repeatedly within a predetermined distance range.

7. The moving robot of claim 1, wherein the controller is further configured to perform a control operation so that avoidance driving is performed in a different pattern based on the recognized attribute of the obstacle.

8. The moving robot of claim 1, wherein the preset sound is alert sound, or a message for guiding the movable obstacle to move.

9. The moving robot of claim 1, wherein the controller comprises:

an obstacle recognition module configured to recognize an obstacle from an image, acquired by the image acquisition unit, based on data that is pre-learned using machine learning; and

a driving control module configured to control driving of the travel unit based on an attribute of the obstacle.

10. The moving robot of claim 9, further comprising:

a communication unit configured to receive machine learning-related data from a specific server,

wherein the object recognition module is updated based on the machine learning-related data received from the specific server.

11. The moving robot of claim 1, wherein the controller is further configured to, when the moving speed is slower, select an image captured at an earlier point in time before an obstacle is detected by the sensor unit.

12. The moving robot of claim 1, wherein the controller is configured to extract a partial region from an image acquired by the image acquisition unit based on a direction from which an object is detected by the sensor.

13. A control method of a moving robot, the method comprising:

detecting, by a sensor unit, an obstacle while a body of the moving robot moves;

upon detection of an obstacle by the sensor unit, recognize an attribute of the obstacle based on an image acquired by an image acquisition unit;

when the recognized attribute of the obstacle indicates a movable obstacle, outputting preset sound; and

controlling driving of a travel unit based on the recognized attribute of the obstacle,

wherein the recognizing of an attribute of the obstacle comprises selecting an image captured at a particular point in time before the obstacle is detected by the sensor unit from among a plurality of continuous images acquired by the image acquisition unit based on a moving direction and a speed of the body, and

recognizing the attribute of the obstacle included in the image captured at the particular point in time.

14. The control method of claim 13, wherein the movable obstacle is part of a human body or a pet.

15. The control method of claim 13, further comprising:

stopping moving the body when the recognized attribute of the obstacle indicates a movable obstacle.

16. The control method of claim 15, further comprising:

detecting whether the movable obstacle moves within a particular standby time;

wherein previous movement of the body before the stopping is resumed when movement of the movable obstacle is detected within the particular standby time.

17. The control method of claim 15, further comprising:

detecting whether the movable obstacle moves within a particular standby time,

wherein, when movement of the movable obstacle is not detected within the particular standby time, the controlling of driving of the travel unit based on the recognized attribute of the obstacle comprises controlling the travel unit to perform avoidance driving with respect to the movable obstacle.

18. The moving robot of claim 17, wherein the controller is further configured to:

perform control operations comprising;

when the obstacle is detected from a forward right side of the body, extracting a lower right part of the image acquired by the image acquisition unit;

when the obstacle is detected from a forward left side of the body, extracting a lower left part of the image acquired by the image acquisition unit; and

when the obstacle is detected from a front side of the body, extracting a lower center part of the image acquired.