DISPLAY CONTROL DEVICE, DISPLAY CONTROL METHOD, COMPUTER PROGRAM PRODUCT, AND COMMUNICATION SYSTEM

• US 20190126467A1
• Filed: 11/20/2018
• Published: 05/02/2019
• Est. Priority Date: 08/02/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kawamoto, Kenta, Iwai, Yoshiaki, Shimizu, Satoru
• Granted Patent: US 10,843,337 B2
• Time in Patent Office: Days
• CPC Class Codes: B25J 13/06 Control stands, e.g. consoles, switchboards
• CPC Class Codes: B25J 13/08 by means of sensing devices, e.g. viewing or touching devices
• CPC Class Codes: B25J 13/084 Tactile sensors in general G01L5/16, G01L5/22
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/0003 Home robots, i.e. small robots for domestic use
• CPC Class Codes: B25J 9/16 Programme controls programme controls in general G05B19/00, e.g. numerical pogramme controls G05B19/18; recording or playback systems G05B19/42
• CPC Class Codes: B25J 9/1661 characterised by task planning, object-oriented languages
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/40099 Graphical user interface for robotics, visual robot user interface
• CPC Class Codes: G05B 2219/40123 Indicate, select features on display, remote manipulator will execute
• CPC Class Codes: G05B 2219/40409 Robot brings object near operator, operator places object in correct position
• CPC Class Codes: G05B 2219/40411 Robot assists human in non-industrial environment like home or office
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

A control system, method and computer program product cooperate to assist control for an autonomous robot. An interface receives recognition information from an autonomous robot, said recognition information including candidate target objects to interact with the autonomous robot. A display control unit causes a display image to be displayed on a display of candidate target objects, wherein at least two of the candidate target objects are displayed with an associated indication of a target object score.

Claims

1. :

A method for operating an instructing device comprising;

controlling reception of location information of a movable apparatus and an image data of a captured image that is captured with a camera at the movable apparatus, the location information being generated based on GPS;

generating feedback information to recognize a target object in the image;

controlling a display to display (i) a map indicating a location of the movable apparatus based on the location information and (ii) an indicator to indicate the target object in the display; and

controlling, with circuitry, transmission of the feedback information to the movable apparatus to control an operation of the movable apparatus and an operation of the camera.

2. :

The method according to claim 1, wherein the feedback information is generated based on a command input indicating a selection of the target object.

3. :

The method according to claim 1, wherein the movable apparatus is controlled to move towards the target object.

4. :

The method according to claim 1, wherein the movable apparatus is controlled to move within a moving range based on at least the location information.

5. :

The method according to claim 1, wherein the transmitting includes

controlling transmission of the feedback information to the movable apparatus.

6. :

The method according to claim 1, wherein the indicator is displayed adjacent to the target object within a display area of the display.

7. :

The method according to claim 1, wherein the indicator has a rectangular shape.

8. :

The method according to claim 1, wherein the indicator is superimposed on the target object of the captured image.

9. :

The method according to claim 1, wherein the controlling includes controlling the display to display the map with an indication of a location of the target object based on the location information and the feedback information.

10. :

The method according to claim 1, wherein the controlling further includes

controlling the display to display the image and the map simultaneously.

19. :

The non-transitory computer readable medium according to claim 1, wherein the controlling includes controlling the display to display the map with an indication of a location of the target object based on the location information and the feedback information.

20. :

The non-transitory computer readable medium according to claim 1, wherein the controlling further includes;

controlling the display to display the image and the map simultaneously.

11. :

A non-transitory computer readable medium having a computer program recorded thereon, the computer program configured to perform a method when executed on a computer, the method comprising;

controlling reception of location information of a movable apparatus and an image data of a captured image that is captured with a camera at the movable apparatus, the location information being generated based on GPS;

generating feedback information to recognize a target object in the image;

controlling a display to display (i) a map indicating a location of the movable apparatus based on the location information and (ii) an indicator to indicate the target object in the display; and

controlling, with circuitry, transmission of the feedback information to the movable apparatus to control an operation of the movable apparatus and an operation of the camera.

12. :

The non-transitory computer readable medium according to claim 11, wherein the feedback information is generated based on a command input indicating a selection of the target object.

13. :

The non-transitory computer readable medium according to claim 11, wherein the movable apparatus is controlled to move towards the target object.

14. :

The non-transitory computer readable medium according to claim 11, wherein the movable apparatus is controlled to move within a moving range based on at least the location information.

15. :

The non-transitory computer readable medium according to claim 11, wherein the transmitting includes;

controlling transmission of the feedback information to the movable apparatus.

16. :

The non-transitory computer readable medium according to claim 11, wherein the indicator is displayed adjacent to the target object within a display area of the display.

17. :

The non-transitory computer readable medium according to claim 11, wherein the indicator has a rectangular shape.

18. :

The non-transitory computer readable medium according to claim 11, wherein the indicator is superimposed on the target object of the captured image.

21. :

An instructing device comprising;

circuitry configured to;

receive location information of a movable apparatus and an image data of a captured image that is captured with a camera at the movable apparatus, the location information being generated based on GPS;

generate feedback information to recognize a target object in the image;

control a display to display (i) a map indicating a location of the movable apparatus based on the location information and (ii) an indicator to indicate the target object in the display; and

control transmission of the feedback information to the movable apparatus to control an operation of the movable apparatus and an operation of the camera.

22. :

The instructing device according to claim 21, wherein the feedback information is generated by the circuitry based on a command input indicating a selection of the target object.

23. :

The instructing device according to claim 21, wherein the movable apparatus is controlled to move towards the target object.

24. :

The instructing device according to claim 21, wherein the movable apparatus is controlled to move within a moving range based on at least the location information.

25. :

The instructing device according to claim 21, wherein the circuitry is further configured to;

transmit the feedback information to the movable apparatus.

26. :

The instructing device according to claim 21, wherein the indicator is displayed adjacent to the target object within a display area of the display.

27. :

The instructing device according to claim 21, wherein the indicator has a rectangular shape.

28. :

The instructing device according to claim 21, wherein the indicator is superimposed on the target object of the captured image.

29. :

The instructing device according to claim 21, wherein the circuitry is configured to control the display to display the map with an indication of a location of the target object based on the location information and the feedback information.

30. :

The instructing device according to claim 21, wherein the circuitry is further configured to;

control the display to display the image and the map simultaneously.