Terminal apparatus, control system, and control method for controlling collaboration among a plurality of devices

• US 9,473,617 B2
• Filed: 03/30/2011
• Issued: 10/18/2016
• Est. Priority Date: 03/30/2010
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Shinohara, Sumito, Aibara, Takehiro, Amagai, Hitoshi, Uehara, Naotaka, Kogane, Takayuki, Handa, Tetsuya, Nunokawa, Masato, Mizuno, Kimiyasu
• Primary Examiner(s): Avellino, Joseph E
• Assistant Examiner(s): Book, Phyllis A
• Application Number:
  US13/075,342
• Publication Number:
  US 20110246903A1
• Time in Patent Office: 2,029 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G08C 17/00 Arrangements for transmitting signals characterised by the use of a wireless electrical link
• CPC Class Codes: G08C 2201/20 Binding and programming of remote control devices
• CPC Class Codes: G08C 2201/41 Remote control of gateways
• CPC Class Codes: H04L 12/6418 Hybrid transport
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 69/24 Negotiation of communication capabilities
• CPC Class Codes: H04M 1/72415 for remote control of appliances
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]

Abstract

An apparatus, system, or method for causing a controlled device and a controlling terminal to operate in collaboration with each other, wherein a server apparatus stores in advance a plurality of sets of collaboration information which causes the controlled device and the controlling terminal to operate in collaboration with each other. The plurality of sets of collaboration information is respectively stored in association with a plurality of controlled apparatuses different from one another in type. When the controlling terminal requests the server apparatus to send the collaboration information designating the type of the controlled device, the server apparatus sends to the controlling terminal the collaboration information corresponding to the type of the controlled device. The controlled device and the controlling terminal operate in collaboration with each other using the collaboration information.

Claims

1. A control system comprising an information terminal as a controlled device, a mobile terminal as a controlling device, and a server apparatus, wherein the controlled device, the controlling device, and the server apparatus are communicatively connected to one another, and wherein:

the server apparatus comprises;

a memory that stores (i) a plurality of sets of collaboration information each of which is usable to cause the controlled device and the controlling device to operate in collaboration with each other, each of the plurality of sets of collaboration information including conversion information for converting a key operation input at the controlling device into a command comprehensible by the controlled device, and (ii) a plurality of command sets each of which comprises a set of control commands, each of the plurality of command sets being common to all of the plurality of sets of collaboration information, wherein each of the plurality of sets of collaboration information is stored in the memory for a respective one of a plurality of combinations of controlled devices of different types and controlling devices of different types; and

a first processor that, when a transmission request for collaboration information designating a type of controlled device and a type of controlling device is sent from the controlling device, performs control to select collaboration information and a command set corresponding to a combination of the type of the controlled device having been designated and the type of the controlling device having been designated, from among the plurality of sets of the collaboration information and the plurality of command sets stored in the memory, and to send the collaboration information and the command set having been selected to the controlling device,

the controlling device comprises;

an operation interface that is configured by various keys and that accepts input of a key operation by a user; and

a second processor that performs control to (i) send the transmission request for collaboration information designating the type of controlled device and the type of controlling device to the server apparatus and receive the collaboration information and the command set having been selected and sent from the server apparatus in response to the transmission request, (ii) send the collaboration information and the command set having been received from the server apparatus to the controlled device designated by the type, (iii) convert, using the conversion information included in the collaboration information having been received from the server apparatus, data of the input key operation accepted by the operation interface into data indicating a control command from among the control commands of the command set having been received from the server apparatus, and (iv) send the converted data indicating the control command to the controlled device designated by the type, and

the controlled device comprises;

a third processor that performs control to receive the collaboration information and the command set sent from the controlling device, and to control the controlled device to operate in accordance with a new function by executing the control command indicated by the converted data sent from the controlling device based on the collaboration information and the command set having been received from the controlling device.

2. The control system as set forth in claim 1, wherein:

the second processor performs control to send, to the controlled device, a program included in a portion of the collaboration information, and

the third processor performs control to receive the program included in the portion of the collaboration information sent from the controlling device and to process the control command indicated by the converted data sent from the controlling device by executing the program having been received.

3. The control system as set forth in claim 2, wherein the collaboration information stored in the memory includes:

a first control program that causes the controlled device to operate in the collaboration; and

a second control program that causes the controlling device to operate in the collaboration,

wherein;

the second processor performs control to send the first control program included in the collaboration information received from the server apparatus to the controlled device designated by the type, and controls the controlled device to execute the first control program,

the second processor itself performs control to execute the second control program included in the collaboration information received from the server apparatus, and thereby converts and sends the data indicating the control command to the controlled device designated by the type, and

the third processor performs control to execute control command indicated by the converted data from the controlling device by executing the first control program received from the controlling device.

4. The control system as set forth in claim 1, wherein:

the second processor performs control to receive model information indicative of a type of a controlled device from a controlled device disposed within a wirelessly communicable distance,

the second processor performs control to send the transmission request for the collaboration information designated by the model information received from the controlled device to the sever apparatus, and

the third processor performs control to send model information indicative of a type of the controlled device to the controlling device, in response to a request from the controlling device.

5. The control system as set forth in claim 1, wherein:

the first processor and the second processor establish a connection for communication by way of a public network, and

the second processor and the third processor establish a connection for communication by way of a personal network.

6. The control system as set forth in claim 2, wherein:

the controlled device includes a display that displays an image, and

the third processor performs control to perform, as an extension function related to an image display, image processing and image display, selectively in response to the control command indicated by the converted data sent from the controlling device, based on the received collaboration information and the received command set.

7. The control system as set forth in claim 1, wherein:

each of the plurality of sets of collaboration information is stored in the memory for a respective one of a plurality of combinations of the controlled devices of different types, the controlling devices of different types, and application programs of different types,

the first processor performs control to (i) search for, in response to the transmission request for collaboration information designating the type of controlled device and the type of controlling device from the controlling device, a plurality of sets of collaboration information corresponding to the combination of the type of the controlled device having been designated and the type of the controlling device having been designated, from among the plurality of sets of collaboration information stored in the memory, (ii) send a list of a plurality of application programs respectively corresponding to the plurality of sets of collaboration information having been searched for, to the controlling device, and (iii) send collaboration information and a command set corresponding to an application program selected by the controlling device in response to sending of the list to the controlling device, and

the second processor performs control to receive the list of the plurality of application programs sent from the server apparatus, in response to the transmission request for the collaboration information which is to be displayed, sends selected information of an application program selected by a user from among the list to the server apparatus, and receives collaboration information and a command set sent from the server apparatus, in response to sending of the selected information.

8. The control system as set forth in claim 1, wherein the memory further stores image data, the collaboration information comprises authentication information that allows transmission of image data from the server apparatus, and the third processor performs control to acquire the image data from the server apparatus based on the authentication information received as the collaboration information.

9. The control system as set forth in claim 1, wherein the plurality of command sets include a command set comprising a set of control commands relating to functions of processing a still image.

18. The control system as set forth in claim 9, wherein the plurality of command sets further include a command set comprising a set of control commands relating to functions of processing a moving image.

10. A controlling device which operates in collaboration with an information terminal as a controlled device, the controlling device being a mobile terminal and comprising:

an operation interface that is configured by various keys and that accepts input of a key operation by a user; and

a processor which functions as;

a first communication control unit that controls communication with a server apparatus that stores, in a memory, (i) a plurality of sets of collaboration information each of which is usable to cause the controlled device and the controlling device to operate in collaboration with each other, each of the plurality of sets of collaboration information including conversion information for converting a key operation input at the operation interface into a command comprehensible by the controlled device, and (ii) a plurality of command sets each of which comprises a set of control commands, each of the plurality of command sets being common to all of the plurality of sets of collaboration information, wherein each of the plurality of sets of collaboration information is stored in the memory for a respective one of a plurality of combinations of controlled devices of different types and controlling devices of different types;

a second communication control unit that controls communication with the controlled devices of different types;

an acquisition control unit that sends, to the server apparatus, a transmission request for collaboration information designating a type of controlling device and a type of controlled device which is an object of control to be controlled for operating in collaboration with the controlling device itself, and acquires collaboration information and a command set sent from the server apparatus in response to the transmission request; and

a collaboration control unit that controls the controlled device by converting, using the conversion information included in the collaboration information acquired from the server apparatus, data of the input key operation accepted by the operation interface into data indicating a control command from among the control commands of the command set acquired from the server apparatus, and by sending the converted data indicating the control command to the controlled device which is the object of control for operating in collaboration with the controlling device itself based on the collaboration information and the command set acquired by the acquisition control unit.

11. The controlling device as set forth in claim 10, wherein the collaboration control unit sends the collaboration information and the command set acquired by the acquisition control unit to the controlled device, and sends the converted data indicating the control command to the controlled device which processes the control command indicated by the converted data based on the collaboration information and the command set sent thereto.

12. The controlling device as set forth in claim 11, wherein the collaboration information includes:

a first control program that causes the controlled device to operate in the collaboration; and

a second control program that causes the controlling device to operate in the collaboration,

wherein the collaboration control unit sends the first control program included in the collaboration information to the controlled device to have the controlled device execute the first control program, and executes the second control program by itself.

13. The controlling device as set forth in claim 10, wherein the second communication control unit receives model information indicative of a type of controlled device disposed within a wirelessly communicable distance, and the acquisition control unit requests that the server apparatus send the collaboration information by designating the model information received by the second communication control unit.

14. The controlling device as set forth in claim 10, wherein the acquisition control unit requests that the server apparatus send the collaboration information designated by the type of the controlling device itself and the type of the controlled device to be operated in collaboration with the controlling device itself, and receives the collaboration information sent from the server apparatus in response to the request.

15. The controlling device as set forth in claim 10, wherein the acquisition control unit (i) sends the transmission request to the server apparatus requesting transmission of the collaboration information designating the type of the controlling device and the type of the controlled device to be operated in collaboration with the controlling device itself, (ii) receives a list of a plurality of application programs sent from the server apparatus in response to the transmission request so as to display the list, (iii) sends selection information of an application program selected by a user from the displayed list to the server apparatus, and (iv) receives collaboration information and a command set sent from the server apparatus in response to the transmission of the selection information.

16. A control method of a control system comprising an information terminal as a controlled device, a mobile terminal as a controlling device, and a server apparatus, wherein the controlled device, the controlling device, and the server apparatus are communicatively connected to one another, and wherein the method comprises:

a step of sending a transmission request from the controlling device to the server apparatus requesting transmission of collaboration information designating a type of the controlled device and a type of the controlling device;

a step, performed by the server apparatus in response to receiving the transmission request, of selecting collaboration information and a command set corresponding to a combination of the type of the controlled device having been designated and the type of the controlling device having been designated from among a plurality of sets of collaboration information and a plurality of command sets stored in the server apparatus, each of the plurality of sets of collaboration information being usable to cause a controlled device and a controlling device to operate in collaboration with each other, each of the plurality of sets of collaboration information including conversion information for converting a key operation input at a controlling device into a command comprehensible by a controlled device, each of the plurality of command sets comprising a set of control commands and being common to all of the plurality of sets of collaboration information, and each of the plurality of sets of collaboration information being stored for a respective one of a plurality of combinations of controlled devices of different types and controlling devices of different types;

a step, performed by the server apparatus, of sending the collaboration information and the command set having been selected to the controlling device;

a step, performed by the controlling device, of receiving the collaboration information and the command set sent from the server apparatus in response to the transmission request;

a step, performed by the controlling device, of accepting input of a key operation by a user at an operation interface of the controlling device, the operation interface being configured by various keys;

a step, performed by the controlling device, of converting, using the conversion information included in the collaboration information having been received from the server apparatus, data of the accepted input key operation input at the operation interface into data indicating a control command from among the control commands of the command set having been received from the server apparatus;

a step, performed by the controlling device, of sending the converted data indicating the control command to the controlled device designated by the type; and

a step, performed by the controlled device, of operating in accordance with a new function by processing to receive and execute the control command indicated by the converted data sent from the controlling device based on the received collaboration information and the received command set.

17. The control method as set forth in claim 16, further comprising:

a step of sending a program included in a portion of the collaboration information received by the controlling device to the controlled device,

wherein the controlled device receives the program included in the portion of the collaboration information from the controlling device and processes and executes the control command indicated by the converted data sent from the controlling device by executing the program having been received.