Control device, control method, computer program product, and robot control system

• US 9,766,604 B2
• Filed: 07/25/2012
• Issued: 09/19/2017
• Est. Priority Date: 08/02/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kawamoto, Kenta, Iwai, Yoshiaki, Shimizu, Satoru
• Primary Examiner(s): Bhatia, Ajay
• Assistant Examiner(s): NGUYEN, PHUONG H
• Application Number:
  US14/234,658
• Publication Number:
  US 20140157156A1
• Time in Patent Office: 1,882 Days
• Field of Search: 715764
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 2219/40099 Graphical user interface for robotics, visual robot user interface
• CPC Class Codes: G05B 2219/40123 Indicate, select features on display, remote manipulator will execute
• CPC Class Codes: G05B 2219/40409 Robot brings object near operator, operator places object in correct position
• CPC Class Codes: G05B 2219/40411 Robot assists human in non-industrial environment like home or office
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

A control system, method and computer program product cooperate to assist control for an autonomous robot. The system includes a communications interface that exchanges information with the autonomous robot (22). A user interface displays a scene of location in which the autonomous robot (22) is positioned, and also receives an indication of a user selection of a user selected area within the scene. The communications interface transmits an indication of said user selected area to the autonomous robot (22) for further processing of the area by said autonomous robot (22).

Claims

1. A control system for an autonomous robot, comprising:

circuitry configured to

exchange information with the autonomous robot;

control display of a scene of a location in which the autonomous robot is positioned, receive an indication of a user selection of a searching area to be searched by the autonomous robot within said scene, and receive as input a category of a target object within the searching area, wherein

said circuitry transmits an indication of said searching area and the category of the target object to said autonomous robot to control the autonomous robot to identify within said searching area one or more candidate target objects in the category and assign one of at least three likelihood degrees to each candidate target object, the likelihood degrees indicating a likelihood that a respective candidate target object is a target object in the category.

2. The control system of claim 1, further comprising:

said autonomous robot having a receiver, wherein said autonomous robot moves to another location in response to said receiver receiving said indication of said searching area.

3. The control system of claim 1, further comprising:

a display, wherein

said scene is displayed on said display and includes a space diagram of a room and the searching area is a partial area of said room, said space diagram including objects in said room.

4. The control system of claim 1, further comprising:

a receiver that receives images from the autonomous robot; and

the autonomous robot, wherein the autonomous robot transmits a second image of the searching area after transmitting a higher quality image used for selection of the searching area.

5. The control system of claim 1, further comprising:

at least one of a tablet computer and a smartphone that includes said circuitry and a display controller.

6. The control system of claim 1, further comprising:

a controller that generates the indication of said searching area in response to receiving the indication of the user selection.

7. The control system of claim 2, wherein

said autonomous robot is configured to detect a smaller area within said searching area based on further user input transmitted from the circuitry.

8. The control system of claim 7, wherein

said autonomous robot is configured to transmit specified range information that indicates the smaller area that is detected.

9. The control system of claim 1, wherein

said scene is a captured image.

10. The control system of claim 1, wherein

said scene is a computer generated graphic.

11. The control system of claim 1, wherein

the circuitry is configured to receive voice or gesture input commands.

12. The control system of claim 1, further comprising:

a display that displays candidate target objects identified by the autonomous robot and user feedback sent via said circuitry to assist in controlling said autonomous robot.

19. The control system of claim 1, further comprising a controller configured to control a position of the autonomous robot and to control an orientation of a camera of the autonomous robot.

20. The control system of claim 1, further comprising a controller configured to control a position of the autonomous robot and to control an orientation of a camera of the autonomous robot such that the searching area is fully captured by the camera.

21. The control system of claim 1, wherein

the circuitry further receives a user selection of an object, and

transmits the user selection of the object to the autonomous robot to control the autonomous robot to search for the object within the searching area.

13. A display control method for an autonomous robot, comprising:

exchanging information with the autonomous robot via a communications interface;

controlling display of a scene of a location in which the autonomous robot is positioned;

receiving via a user interface an indication of a user selection of a searching area to be searched by the autonomous robot within said scene and a category of a target object within the searching area; and

forming with a processing circuit and transmitting an indication of said searching area and the category of the target object to said autonomous robot to control the autonomous robot to identify within said searching area one or more candidate target objects in the category and assign one of at least three likelihood degrees to each candidate target object, the likelihood degrees indicating a likelihood that a respective candidate target object is a target object in the category.

14. The method of claim 13, further comprising:

receiving said indication of said searching area at said autonomous robot; and

moving said autonomous robot to another location in response to said receiving.

15. The method of claim 14, further comprising

detecting with said autonomous robot a smaller area within said searching area in response to receiving further user input.

16. The control method of claim 13, wherein

said scene is a captured image.

17. The control method of claim 13, wherein

said scene is a computer generated graphic.

18. A non-transitory computer readable storage medium having instructions stored therein that when executed by a processing circuit perform a display control method for an autonomous robot, comprising:

exchanging information with the autonomous robot via a communications interface;

controlling display of a scene of a location in which the autonomous robot is positioned;

receiving via a user interface an indication of a user selection of a searching area to be searched by the autonomous robot within said scene and a category of a target object within the searching area; and

forming with the processing circuit and transmitting an indication of said searching area and the category of the target object to said autonomous robot to control the autonomous robot to identify within said searching area one or more candidate target objects in the category and assign one of at least three likelihood degrees to each candidate target object, the likelihood degrees indicating a likelihood that a respective candidate target object is a target object in the category.