Mobile robot and path planning method thereof for manipulating target objects

• US 8,340,901 B2
• Filed: 09/21/2009
• Issued: 12/25/2012
• Est. Priority Date: 05/20/2009
• Status: Expired due to Fees

• Current Assignee: National Taiwan University of Science and Technology
• Original Assignee: National Taiwan University of Science and Technology
• Inventors: Fahn, Chin-Shyurng, Wu, Chien-Hsin
• Primary Examiner(s): MANCHO, RONNIE M
• Application Number:
  US12/563,165
• Publication Number:
  US 20100298977A1
• Time in Patent Office: 1,191 Days
• Field of Search: 701/201, 701/301, 701/431, 701/410, 382/103, 382/107, 382/153, 382/291, 382/276, 318/568.12, 700/246, 700/213, 700/245, 700/215, 340/557, 342/118, 348/135, 348/142
• US Class Current: 701/431
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means

Abstract

A mobile robot and a path planning method are provided for the mobile robot to manipulate the target objects in a space, wherein the space consists of a periphery area and a central area. With the present method, an initial position is defined and the mobile robot is controlled to move within the periphery area from the initial position. Next, the latest image is captured when the mobile robot moves, and a manipulating order is arranged according to the distances estimated between the mobile robot and each of target objects in the image. The mobile robot is controlled to move and perform a manipulating action on each of the target object in the image according to the manipulating order. The steps of obtaining the image, planning the manipulating order, and controlling the mobile robot to perform the manipulating action are repeated until the mobile robot returns to the initial position.

Claims

1. A path planning method for manipulating target objects, adapted to plan moving paths of a mobile robot to manage the plurality of target objects in a space, wherein the space comprises a periphery area and a central area, the path planning method comprising:

defining an initial position;

controlling the mobile robot to move from the initial position in the periphery area, wherein the mobile robot successively captures images while moving;

obtaining a latest one of the images captured by the mobile robot;

planning a manipulating order according to distances respectively estimated between the target objects and the mobile robot in the latest one of the images;

controlling the mobile robot to move according to the manipulating order and perform a manipulating action on the target objects in the one of the images, wherein the manipulating action is approaching and collecting one of the target objects;

repeating the steps of obtaining the latest one of the images, planning the manipulating order, and controlling the mobile robot to perform the manipulating action until the mobile robot returns to the initial position; and

controlling the mobile robot to move to the central area and perform the manipulating action on the target objects in the central area.

2. The path planning method as claimed in claim 1, wherein the step of controlling the mobile robot to move to the central area, and perform the manipulating action on the target objects in the central area further includes:

dividing the central area into a plurality of sub-areas, and defining a traversal order of visiting the sub-areas;

controlling the mobile robot to move into one of the sub-areas according to the traversal order, wherein the mobile robot successively captures images while moving;

obtaining a latest one of the captured images in the one of the sub-areas;

planning the manipulating order according to the distances respectively estimated between the target objects and the mobile robot in the one of the images;

controlling the mobile robot to move according to the manipulating order and perform the manipulating action on the target objects in the one of the images; and

repeating the steps of controlling the mobile robot to move into the one of the sub-areas, obtaining the one of the images, planning the manipulating order, and controlling the mobile robot to perform the manipulating action until the mobile robot has passed through each of the sub-areas.

3. The path planning method as claimed in claim 1, wherein the target objects are in form of a specific shape, and the step of planning the manipulating order according to the distances respectively estimated between the target objects and the mobile robot in the one of the images includes:

searching the specific shape to obtain at least one candidate object in the one of the images;

estimating the distances respectively between each of the candidate objects and the mobile robot;

selecting the candidate objects situated at the distances less than or equal to a first threshold as the target objects in the one of the images; and

planning the target objects situated at the shorter distances from the mobile robot to have prior orders when planning the manipulating order of managing the target objects.

4. The path planning method as claimed in claim 3, wherein the step of searching the specific shape to obtain the candidate objects in the one of the images comprises:

performing an edge detection operation on the one of the images to extract a plurality of edge features; and

determining whether the edge features constitute the specific shape or not to obtain the candidate objects.

5. The path planning method as claimed in claim 3, wherein after the step of searching the specific shape to obtain the candidate objects in the one of the images, the path planning method further includes:

dividing the one of the images into a plurality of sub-image areas, wherein each of the sub-image areas is respectively associated with an orientation directed from the mobile robot;

calculating the amounts of the candidate objects located in each of the sub-image areas;

obtaining one of the sub-image areas corresponding to a most one of the amounts; and

setting the orientation associated with the obtained sub-image area as a moving direction.

6. The path planning method as claimed in claim 5, further including:

controlling the mobile robot to move in the moving direction.

7. The path planning method as claimed in claim 3, wherein the step of estimating the distances respectively between each of the candidate objects and the mobile robot includes:

estimating the distances respectively between each of the candidate objects and the mobile robot according to sizes of each of the candidate objects in the one of the images.

8. The path planning method as claimed in claim 3, wherein the specific shape is in form of a circle, and the step of estimating the distances respectively between each of the candidate objects and the mobile robot includes:

estimating the distances respectively between each of the candidate objects and the mobile robot according to radii of each of the candidate objects in the one of the images.

9. The path planning method as claimed in claim 3, wherein after the step of selecting the candidate objects situated at the distances less than or equal to the first threshold as the target objects in the one of the images, the path planning method further includes:

obtaining current positions of each of the target objects in a latest one of the captured images and corresponding previous positions in one of the captured images previous to the latest one of the captured images; and

deleting the target objects having displacements, which are the distances between the current positions and the corresponding previous positions, greater than a second threshold.

10. The path planning method as claimed in claim 3, wherein the target objects coincide with a specific color, and after the step of selecting the candidate objects situated at the distances less than or equal to the first threshold as the target objects in the one of the images, the path planning method further includes:

obtaining colors of each of the target objects; and

deleting the target objects having the colors, which do not coincide with the specific color.

11. The path planning method as claimed in claim 1, wherein the step of planning the manipulating order according to the distances respectively estimated between the target objects and the mobile robot in the one of the images includes:

determining whether a new target object appears in a latest one of the captured images or not at every predetermined period; and

rearranging the manipulating order according to the distance between the new target object and the mobile robot if the new target object appears.

12. The path planning method as claimed in claim 1, wherein when the mobile robot moves in the periphery area, the path planning method further includes:

detecting environment information by use of an ultrasonic sensing apparatus; and

controlling the mobile robot to move along a border in the space, and keeping the mobile robot a predetermined distance away from the border according to the environment information.

13. The path planning method as claimed in claim 1 further including:

obtaining a plurality of edge features in the one of the images;

determining that static obstacles exist in positions of each of the edge features when each of the edge features constitutes a straight line having a slope of 90 degrees or substantially 90 degrees; and

controlling the mobile robot to avoid the static obstacles when the mobile robot moves.

14. The path planning method as claimed in claim 1 further including:

estimating a moving velocity of an object according to a position of the object in a latest one of the captured images and a position of the object in one of the captured images previous to the latest one of the captured images;

determining that the object is a dynamic obstacle when the moving velocity is greater than a third threshold; and

controlling the mobile robot to avoid the dynamic obstacle when the mobile robot approaches to the dynamic obstacle.

15. The path planning method as claimed in claim 1, wherein when the mobile robot moves in the periphery area, the path planning method further includes:

controlling the mobile robot to move toward a border then along the border, and keeping the mobile robot a predetermined distance away from the border when the mobile robot moves in the periphery area and deviates a distance greater than a fourth threshold from the border in the space, wherein the fourth threshold is greater than the predetermined distance.

16. A mobile robot comprising:

a driving apparatus steering the mobile robot to move;

an image capturing apparatus successively acquiring images when the mobile robot moves;

a target object manipulating apparatus managing a plurality of target objects in a space, wherein the space consists of a periphery area and a central area; and

a control module coupled to the image capturing apparatus, the driving apparatus, and the target object manipulating apparatus,

wherein the control module defines an initial position, enables the driving apparatus to steer the mobile robot to move from the initial position in the periphery area, obtains a latest one of the images acquired by the image capturing apparatus, plans a manipulating order according to distances respectively estimated between the target objects and the mobile robot in the one of the images, enables the driving apparatus to steer the mobile robot to move according to the manipulating order, and activates the target object manipulating apparatus to perform a manipulating action on the target objects in the one of the images, wherein the manipulating action is approaching and collecting one of the target objects,

wherein the control module repeats the procedures of obtaining the one of the images, planning the manipulating order, and activating the target object manipulating apparatus to perform the manipulating action until the mobile robot returns to the initial position,

wherein after the mobile robot returns to the initial position, the control module enables the driving apparatus to steer the mobile robot to move to the central area and activates the target object manipulating apparatus to perform the manipulating action on the target objects in the central area.

17. The mobile robot as claimed in claim 16, wherein the control module divides the central area into a plurality of sub-areas after the mobile robot returns to the initial position, defines a traversal order of visiting the sub-areas, enables the driving apparatus to steer the mobile robot to move into one of the sub-areas according to the traversal order, obtains a latest one of the images acquired by the image capturing apparatus in the one of the sub-areas, plans the manipulating order according to the distances respectively estimated between the target objects and the mobile robot in the one of the images, enables the driving apparatus to steer the mobile robot to move according to the traversal order, and activates the target object manipulating apparatus to perform the manipulating action on the target objects in the one of the images, and the control module repeats the procedures of enabling the driving apparatus to steer the mobile robot to move into the one of the sub-areas, obtaining the one of the images, planning the manipulating order, and activating the target object manipulating apparatus to perform the manipulating action until the mobile robot has passed through each of the sub-areas.

18. The mobile robot as claimed in claim 16, wherein the target objects are in form of a specific shape, and the control module searches the specific shape to obtain at least one candidate object in the one of the images, estimates the distances respectively between each of the candidate objects and the mobile robot, selects the candidate objects situated at the distances less than or equal to a first threshold as the target objects in the one of the images, and plans the target objects situated at the shorter distances from the mobile robot to have prior orders when planning the manipulating order of managing the target objects.

19. The mobile robot as claimed in claim 18, wherein the control module performs an edge detection operation on the one of the images to extract a plurality of edge features and determines whether the edge features constitute the specific shape or not to obtain the candidate objects.

20. The mobile robot as claimed in claim 18, wherein the control module divides the one of the images into a plurality of sub-image areas, wherein each of the sub-image areas is respectively associated with an orientation directed from the mobile robot, and the control module calculates the amounts of the candidate objects located in each of the sub-image areas, obtains one of the sub-image areas covering a most one of the amounts, and sets the orientation associated with the obtained sub-image area as a moving direction.

21. The mobile robot as claimed in claim 18, wherein the control module enables the driving apparatus to steer the mobile robot to move in the moving direction.

22. The mobile robot as claimed in claim 18, wherein the control module estimates the distances respectively between each of the candidate objects and the mobile robot according to sizes of each of the candidate objects in the one of the images.

23. The mobile robot as claimed in claim 18, wherein the specific shape is in form of a circle, and the control module estimates the distances respectively between each of the candidate objects and the mobile robot according to radii of each of the candidate objects in the one of the images.

24. The mobile robot as claimed in claim 18, wherein the control module obtains current positions of each of the target objects in a latest one of the captured images and corresponding previous positions in one of the captured images previous to the latest one of the captured images, and deletes the target objects having displacements, which are the distances between the current positions and the corresponding previous positions, greater than a second threshold.

25. The mobile robot as claimed in claim 18, wherein the target objects coincide with a specific color, and the control module obtains the colors of each of the target objects and deletes the target objects having the colors, which do not coincide with the specific color.

26. The mobile robot as claimed in claim 16, wherein the control module determines whether a new target object appears in a latest one of the captured images or not at every predetermined period, and if the new target object appears, the control module rearranges the manipulating order according to the distance between the new target object and the mobile robot.

27. The mobile robot as claimed in claim 16 further including:

an ultrasonic sensing apparatus coupled to the control module to detect environment information,

wherein the control module enables the driving apparatus to steer the mobile robot to move along a border in the space and keep the mobile robot a predetermined distance away from the border according to the environment information.

28. The mobile robot as claimed in claim 16, wherein the control module obtains a plurality of edge features in the one of the images, determines that static obstacles exist in positions of each of the edge features that constitute a straight line having a slope of 90 degrees or substantially 90 degrees, and enables the driving apparatus to steer the mobile robot to avoid the static obstacles when the driving apparatus steers the mobile robot.

29. The mobile robot as claimed in claim 16, wherein the control module estimates a moving velocity of an object according to a position of the object in a latest one of the captured images and a position of the object in one of the captured images previous to the latest one of the captured images, determines that the object is a dynamic obstacle when the moving velocity is greater than a third threshold, and enables the driving apparatus to steer the mobile robot to avoid the dynamic obstacle when the mobile robot approaches to the dynamic obstacle.

30. The mobile robot as claimed in claim 16, wherein the control module enables the driving apparatus to steer the mobile robot to move toward a border then along the border and keep the mobile robot a predetermined distance away from the border when the mobile robot moves in the periphery area, and deviates a distance greater than a fourth threshold from the border in the space, wherein the fourth threshold is greater than the predetermined distance.

31. The mobile robot as claimed in claim 16 further including:

a body comprising a chamber and an outlet, wherein the outlet is linked to the chamber.

32. The mobile robot as claimed in claim 31, wherein the driving apparatus comprises a first wheel set and a second wheel set, an axis of the first wheel set is fixed in the body and rotates back and forth about a first axis as an axle center, and an axis of the second wheel set is fixed in the body and rotates left and right about a second axis as an axle center, wherein the first axis and the second axis are substantially perpendicular to each other.

33. The mobile robot as claimed in claim 32, wherein the first wheel set comprises a first wheel and a second wheel and drives the mobile robot to move in a straight line, and the first wheel and the second wheel are parallel to each other and rotate about the first axis as the axle center.

34. The mobile robot as claimed in claim 32, wherein the second wheel set is a directional wheel, an axis of the directional wheel is fixed in the body through a connector, and the directional wheel rotates left and right about the second axis as the axle center to drive the mobile robot to turn.

35. The mobile robot as claimed in claim 32, wherein the target object manipulating apparatus comprises a third wheel, an axis of the third wheel is fixed in the body, and the third wheel has a plurality of fans stretching in a radial direction of a third axis.

36. The mobile robot as claimed in claim 35, wherein the body further comprises a guiding component, and the guiding component covers a part of the third wheel.

37. The mobile robot as claimed in claim 35, wherein the target object manipulating apparatus conducts the target objects through the guiding component to the chamber by virtue of the fans turned with the third wheel when manipulating the target objects.

38. The mobile robot as claimed in claim 35, wherein the first wheel set is located between the third wheel and the second wheel set, and the second wheel set is located between the first wheel set and the outlet.

39. The mobile robot as claimed in claim 38, wherein the radii of the first wheel and the second wheel in the first wheel set are greater than the radius of the directional wheel in the second wheel set.