Robot device, method of controlling the same, computer program, and robot system

• US 9,908,241 B2
• Filed: 09/07/2016
• Issued: 03/06/2018
• Est. Priority Date: 07/25/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Shimizu, Satoru, Kawamoto, Kenta, Iwai, Yoshiaki
• Primary Examiner(s): Jen, Ian
• Application Number:
  US15/258,436
• Publication Number:
  US 20160375585A1
• Time in Patent Office: 545 Days
• CPC Class Codes: B25J 9/1679 characterised by the tasks executed
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/37618 Observe, monitor position, posture of tool
• CPC Class Codes: G05B 2219/39478 Control force and posture of hand
• CPC Class Codes: G05B 2219/39517 Control orientation and position of object in hand, roll between plates
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Provided is a robot device including an image input unit for inputting an image of surroundings, a target object detection unit for detecting an object from the input image, an object position detection unit for detecting a position of the object, an environment information acquisition unit for acquiring surrounding environment information of the position of the object, an optimum posture acquisition unit for acquiring an optimum posture corresponding to the surrounding environment information for the object, an object posture detection unit for detecting a current posture of the object from the input image, an object posture comparison unit for comparing the current posture of the object to the optimum posture of the object, and an object posture correction unit for correcting the posture of the object when the object posture comparison unit determines that there is a predetermined difference or more between the current posture and the optimum posture.

Claims

1. A system included in a mobile apparatus, comprising:

circuitry configured to

detect an object in image data,

determine current posture data of the object from the image data,

acquire target posture data of the object based on surrounding environment information of the object, the target posture data also being with respect to a position of a part of the mobile apparatus, and

determine whether at least a predetermined difference exists between the current posture data of the object and the target posture data of the object by comparing the current posture data of the object with the target posture data of the object.

2. The system according to claim 1, wherein the circuitry is further configured to obtain posture change data when the predetermined difference exists.

3. The system according to claim 2, wherein the circuitry is further configured to cause an end effector to move according to the posture change data.

4. The system according to claim 3, wherein the circuitry is further configured to send a query to a user before causing the end effector to move according to the posture change data.

5. The system according to claim 3, further comprising the end effector and the end effector is coupled to the circuitry.

6. The system according to claim 1, further comprising an image sensor configured to generate the image data.

7. The system according to claim 1, further comprising a distance sensor configured to detect a distance to a surrounding object.

8. The system according to claim 1, further comprising a microphone configured to receive an audio input from a user.

9. The system according to claim 1, wherein the circuitry is further configured to execute a task based on an instruction of a user, and when the circuitry is not executing the task, the circuitry performs the determination of whether at least the predetermined difference exists by comparing the current posture data of the object with the target posture data of the object.

10. The system according to claim 1, further comprising moving device configured to move a position of a part of the mobile apparatus.

11. The system according to claim 1, wherein the circuitry is further configured to acquire each target posture data for each of objects and retain the each target posture data for each of the objects in an object posture database.

12. The system according to claim 1, wherein the target posture data is based on a posture taught from a user.

13. The system according to claim 1, wherein the circuitry is further configured to acquire the surrounding environment information of the object from an environment map.

14. The system according to claim 1, wherein the circuitry is further configured to update the environment map based on at least the image data.

15. The system according to claim 1, wherein the mobile apparatus is a robot and the system is a robot system.

16. The system according to claim 1, further comprising a plurality of actuators corresponding to the joints, each actuator operating a respective joint.

17. The system according to claim 15, wherein the plurality of links and joints form a tectonic belt.

18. The system according to claim 1, wherein the target posture data is determined by learning.

19. A method for a device comprising:

detecting, with circuitry, an object in image data;

determining, with the circuitry, current posture data of the object from the image data;

acquiring, with the circuitry, target posture data of the object based on surrounding environment information of the object, the target posture data also being with respect to a position of a part of the device; and

determining, with the circuitry, whether at least a predetermined difference exists between the current posture data of the object and the target posture data of the object by comparing the current posture data of the object with the target posture data of the object.

20. A device comprising:

circuitry configured to

detect an object in image data,

determine current posture data of the object from the image data,

acquire target posture data of the object based on surrounding environment information of the object, the target posture data also being with respect to a position of a part of the device, and

determine whether at least a predetermined difference exists between the current posture data of the object and the target posture data of the object by comparing the current posture data of the object with the target posture data of the object.