Display control device, display control method, computer program product, and communication system

• US 9,815,199 B2
• Filed: 11/30/2015
• Issued: 11/14/2017
• Est. Priority Date: 08/02/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kawamoto, Kenta, Iwai, Yoshiaki, Shimizu, Satoru
• Primary Examiner(s): MANCHO, RONNIE M
• Application Number:
  US14/954,066
• Publication Number:
  US 20160075022A1
• Time in Patent Office: 715 Days
• Field of Search: 700245, 700214, 700228, 700257, 700264
• CPC Class Codes: B25J 13/06 Control stands, e.g. consoles, switchboards
• CPC Class Codes: B25J 13/08 by means of sensing devices, e.g. viewing or touching devices
• CPC Class Codes: B25J 13/084 Tactile sensors in general G01L5/16, G01L5/22
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/0003 Home robots, i.e. small robots for domestic use
• CPC Class Codes: B25J 9/16 Programme controls programme controls in general G05B19/00, e.g. numerical pogramme controls G05B19/18; recording or playback systems G05B19/42
• CPC Class Codes: B25J 9/1661 characterised by task planning, object-oriented languages
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/40099 Graphical user interface for robotics, visual robot user interface
• CPC Class Codes: G05B 2219/40123 Indicate, select features on display, remote manipulator will execute
• CPC Class Codes: G05B 2219/40409 Robot brings object near operator, operator places object in correct position
• CPC Class Codes: G05B 2219/40411 Robot assists human in non-industrial environment like home or office
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

A control system, method and computer program product cooperate to assist control for an autonomous robot. An interface receives recognition information from an autonomous robot, said recognition information including candidate target objects to interact with the autonomous robot. A display control unit causes a display image to be displayed on a display of candidate target objects, wherein at least two of the candidate target objects are displayed with an associated indication of a target object score.

Claims

1. A control device for a movable apparatus that includes at least one driving motor to move the movable apparatus, the control device comprising:

circuitry configured to

generate a control signal to control the movable apparatus, and

control a display to

display (i) an image on a first sub-region of the display, the image corresponding to a captured image captured by a camera attached on the movable apparatus, and

display (ii) an overhead map, including a position of the movable apparatus, on a second sub-region of the display,

wherein a marker for identifying position of the movable apparatus is displayed on the second sub-region in relation to the overhead map,

wherein the image and the overhead map are concurrently displayed in a non-overlapping manner.

2. The control device according to claim 1, wherein the circuitry is configured to control the display to simultaneously display the captured image and the overhead map.

3. The control device according to claim 1, wherein the circuitry is configured to control the display to display the captured image in a first sub-region of the display and to display the overhead map in a second sub-region of the display, the first sub-region being distinct and separate from the second sub-region.

4. The control device according to claim 1, wherein the circuitry is configured to control to the display to superimpose a marker to identify the position of the movable apparatus.

5. The control device according to claim 1, wherein the circuitry is configured to control the display to display path information on the overhead map, the path information indicating a movement of the movable apparatus.

6. The control device according to claim 1, wherein the captured image includes a target object, and position information of the target object is displayed on the overhead map.

7. The control device according to claim 1, wherein the circuitry is configured to control to the display to superimpose a marker to identify a position of the control device.

8. The control device according to claim 5, wherein the path information indicates the movement of the movable apparatus towards a target object that is displayed on the overhead map.

9. The con device according to claim 6, wherein the circuitry is further configured to

obtain a recognition information of the target object, and

control the display to display an indication of the recognition information associated with the target object.

10. The control device according to claim 9, wherein the indication of the recognition information is a marker that is superimposed on the captured image and that is disposed adjacently to the target object.

11. The control device according to claim 1, wherein the generated control signal restricts a movement of the movable apparatus to be within a predetermined area.

12. The control device according to claim 9, wherein the circuitry is further configured to

perform a recognition process on the target object to generate the recognition information of the target object.

13. The control device according to claim 11, wherein the predetermined area is determined based on position information of the movable apparatus and position information of the control device.

14. The control device according to claim 11, wherein the predetermined area is determined based on an input from an interface of the control device.

15. The control device according to claim 14, wherein the interface is a touch panel integrally arranged with the display, and the predetermined area is determined based on a designation of an area on the captured image or the overhead map displayed on the display.

16. The control device according to claim 1, wherein the captured image is compressed it size prior to being displayed on the display.

17. The control device according to claim 1, wherein the captured image is compressed in size prior to being transmitted from the camera.

18. The control device according to claim 17, wherein the captured image is compressed by performing image compression process on only a part of the captured image.

19. The control device according to claim 1, further comprising:

an interface, the interface being implemented by a touch panel that is integrally arranged with the display.

22. The control device according to claim 1, wherein the first sub-region and the second sub region are independent non-overlapping portions of the display.

20. A non-transitory computer readable medium having a computer program recorded thereon, the computer program configured to perform a method when executed on a computer, the method comprising:

generating a control signal to control the movable apparatus; and

controlling a display to

(i) display an image on a first sub-region of the display, the image corresponding to a captured image captured by a camera attached on the movable apparatus, and

(ii) display an overhead map, including a position of the movable apparatus, on a second sub-region of the display,

wherein a marker for identifying position of the movable apparatus is displayed on the second sub-region in relation to the overhead map,

wherein the image and the overhead map are concurrently displayed in a non-overlapping manner.

21. A method for controlling a movable apparatus that includes at least one actuator to move the movable apparatus, the method comprising:

generating a control signal to control the movable apparatus; and

controlling a display to

(i) display an image on a first sub-region of the display, the image corresponding to a captured image captured by a camera attached on the movable apparatus, and

(ii) display an overhead map, including a position of the movable apparatus, on a second sub-region of the display,

wherein a marker for identifying position of the movable apparatus is displayed on the second sub-region in relation to the overhead map,

wherein the image and the overhead map are concurrently displayed in a non-overlapping manner.