Display control apparatus, display control method, and program

• US 9,880,713 B2
• Filed: 10/04/2012
• Issued: 01/30/2018
• Est. Priority Date: 11/16/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kasahara, Shunichi
• Primary Examiner(s): To, Jennifer
• Assistant Examiner(s): Li, Liang
• Application Number:
  US14/343,987
• Publication Number:
  US 20140223323A1
• Time in Patent Office: 1,944 Days
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 2203/0383 Remote input, i.e. interface arrangements in which the signals generated by a pointing device are transmitted to a PC at a remote location, e.g. to a PC in a LAN
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2370/06 Consumer Electronics Control, i.e. control of another device by a display or vice versa
• CPC Class Codes: G09G 2380/14 Electronic books and readers

Abstract

An information processing system that acquires captured image data corresponding to a user interface displayed on a first display, the user interface including a plurality of graphic representations each corresponding to a respective function; and controls a second display to display a representation of the user interface based on a recognition process performed on the user interface, the representation of the user interface indicating whether a function corresponding the plurality of the graphic representations is capable of being accessed.

Claims

1. An information processing system, comprising:

a first information processing apparatus including a first display and first circuitry, the first circuitry configured to;

acquire captured image data corresponding to a user interface displayed on a second display of a second information processing apparatus, the user interface including a plurality of graphic representations each corresponding to respective functions;

receive configuration information and authority information corresponding to the user interface from the second information processing apparatus, the authority information indicating whether the second information processing apparatus permits or restricts an execution of a command transmitted from the first circuitry to the second information processing apparatus;

recognize a screen and items included in the captured image data;

control the first display to display, when the authority information indicates that execution of the command is permitted, a first representation of the user interface based on the captured image data, the first representation of the user interface indicating that functions corresponding to the plurality of the graphic representations are capable of being accessed;

control the first display to display, when the authority information indicates that execution of the command is not permitted, a second representation of the user interface based on the captured image data, the second representation of the user interface including a restriction display image superimposed on at least one of the plurality of graphic representations corresponding to a function that is not capable of being accessed;

convert coordinates of the items according to the configuration information, the authority information, and the screen and the items included in the captured image;

transmit the converted coordinates and the command to the second information processing apparatus; and

the second information processing apparatus including second circuitry configured to;

transmit the configuration information and the authority information to the first information processing apparatus;

receive the converted coordinates and the command from the first information processing apparatus;

determine whether the converted coordinates correspond to items within the user interface;

determine, when the converted coordinates correspond to the items within the user interface, whether the command is authorized according to the authority information; and

execute the command when the command is authorized according to the authority information.

2. The information processing system of claim 1, wherein the first circuitry is further configured to capture the image data corresponding to the user interface displayed on the second display.

3. The information processing system of claim 1, wherein the configuration information indicates a position at which each of the plurality of graphic representations are displayed on the second display.

4. The information processing system of claim 1, wherein the first circuitry is further configured to generate the first representation of the user interface or the second representation of the user interface based on the acquired configuration information.

5. The information processing system of claim 1, wherein the first circuitry is further configured to control the first display to display the second representation of the user interface such that a predetermined visual effect is applied to at least one of the plurality of graphic representations corresponding to a function that is not capable of being accessed.

6. The information processing system of claim 1, wherein the first circuitry is further configured to control the first display to display the second representation of the user interface such that at least one of the plurality of graphic representations corresponding to a function that is not capable of being accessed is omitted.

7. The information processing system of claim 1, wherein the first information processing apparatus further comprises a user interface that receives a selection of one of the plurality of graphic representations.

8. The information processing system of claim 7, wherein

the first information processing apparatus further comprises a communication interface, and

the first circuitry is further configured to control the communication interface to transmit information corresponding to the selection of the one of the plurality of graphic representations to the second information processing apparatus.

9. The information processing system of claim 7, wherein

the first information processing apparatus further comprises a communication interface, and

the first circuitry is further configured to control the communication interface to transmit command identification information corresponding to the selection of the one of the plurality of graphic representations to the second information processing apparatus.

10. The information processing system of claim 7, wherein one of the plurality of graphic representations corresponds to an photo browsing application.

11. The information processing system of claim 10, wherein

the first information processing apparatus further comprises:

a user interface that receives a selection of the graphic representation corresponding to the photo browsing application displayed on the display; and

a communication interface,

the first circuitry is further configured to control the communication interface to transmit information corresponding to the selection of the graphic representation corresponding to the photo browsing application to the second information processing apparatus, and

the second information processing apparatus executes the photo browsing application in response to receiving the information.

12. The information processing system of claim 1, wherein the captured image data is still image data.

13. The information processing system of claim 1, wherein the first information processing apparatus further comprises a user interface that receives authentication information corresponding to a function that is not capable of being accessed.

14. The information processing system of claim 13, wherein the first circuitry is further configured to control the first display to change the representation of the user interface to indicate that a function is capable of being accessed based on the received authentication information.

15. The information processing apparatus of claim 13, wherein the user interface displays a screen to input the authentication information when a graphic representation corresponding to a function that is not capable of being accessed is selected at the first display.

16. An information processing method performed by an information processing system including a first information processing apparatus and a second information processing apparatus, the method comprising:

acquiring, by first circuitry of the first information processing apparatus, captured image data corresponding to a user interface displayed on a second display of the second information processing apparatus, the user interface including a plurality of graphic representations each corresponding to respective functions;

receiving, by the first circuitry, configuration information and authority information corresponding to the user interface displayed on the display from the second information processing apparatus, the authority information indicating whether the second information processing apparatus permits or restricts an execution of a command transmitted from the first circuitry to the second information processing apparatus;

recognizing, by the first circuitry, a screen and items included in the captured image data;

displaying, by a first display of the fist information processing apparatus when the authority information indicates that execution of the command is permitted, a first representation of the user interface based on the captured image data, the first representation of the user interface indicating that functions corresponding to the plurality of the graphic representations are capable of being accessed;

displaying, by the first display when the authority information indicates that execution of the command is not permitted, a second representation of the user interface based on the captured image data, the second representation of the user interface including a restriction display image superimposed on at least one of the plurality of graphic representations corresponding to a function that is not capable of being accessed;

converting, by the first circuitry, coordinates of the items according to the configuration information, the authority information, and the screen and the items included in the captured image;

transmitting, by the first circuitry, the converted coordinates and the command to the second information processing apparatus;

transmitting, by second circuitry of the second information processing apparatus, the configuration information and the authority information to the first information processing apparatus;

receiving, by the second circuitry, the converted coordinates and the command from the first information processing apparatus;

determining, by the second circuitry, whether the converted coordinates correspond to items within the user interface;

determining, by the second circuitry when the converted coordinates correspond to the items within the user interface, whether the command is authorized according to the authority information; and

executing, by the second circuitry when the command is authorized, the command according to the authority information.

17. An information processing system, comprising:

a first information processing apparatus including a first display and first circuitry, the first circuitry configured to;

acquire captured image data corresponding to a user interface displayed on a second display of a second information processing apparatus, the user interface including a plurality of graphic representations each corresponding to respective functions;

receive configuration information and authority information corresponding to the user interface from the second information processing apparatus, the authority information indicating whether the second information processing apparatus permits or restricts an execution of a command transmitted from the first circuitry to the second information processing apparatus;

recognize a screen and items included in the captured image data;

control the first display to display, when the authority information indicates that execution of the command is permitted, a first representation of the user interface based on the captured image data, the first representation of the user interface indicating that functions corresponding to the plurality of the graphic representations are capable of being accessed;

control the first display to display, when the authority information indicates that execution of the command is not permitted, a second representation of the user interface based on the captured image data, the second representation of the user interface including a restriction display image superimposed on at least one of the plurality of graphic representations corresponding to a function that is not capable of being accessed;

convert coordinates of the items according to the configuration information, the authority information, and the screen and the items included in the captured image; and

transmit the converted coordinates and the command to the second information processing apparatus, wherein

the second information processing apparatus is configured to

receive the converted coordinates and the command from the first information processing apparatus;

determine whether the converted coordinates correspond to items within the user interface;

determine, when the converted coordinates correspond to the items within the user interface, whether the command is authorized according to the authority information; and

execute the command when the command is authorized according to the authority information.