MOBILE TERMINAL-BASED VIRTUAL GAME CONTROLLER AND REMOTE CONTROL SYSTEM USING THE SAME

• US 20140011584A1
• Filed: 06/21/2013
• Published: 01/09/2014
• Est. Priority Date: 06/22/2012
• Status: Active Grant

• Current Assignee: Intellectual Discovery Co., Ltd.
• Original Assignee: SungKyunKwan University's Research & Business Foundation (SungKyunKwan University (SKKU))
• Inventors: SHIN, Dong Ryeol, NAM, Choon Sung
• Granted Patent: US 9,612,709 B2
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/426 involving on-screen location information, e.g. screen coordinates of an area at which the player is aiming with a light gun
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]

Abstract

Disclosed herein are a virtual controller client mobile terminal-based virtual game controller and a remote control system using the same. The remote control system includes a virtual controller server and a virtual controller client. The virtual controller server generates button setting information including mapping relationship between key inputs and virtual input messages, transfers the button setting M information, extracts a key input from a virtual input message, and provides the key input to the application. The virtual controller client specifies an arrangement and attributes of virtual buttons based on the button setting information, generates a virtual button screen on the touch screen of the mobile terminal, generates a touch input message based on touch event objects generated based on touch signals for regions corresponding to the virtual buttons, and converts the touch input message into a virtual input message and outputs the virtual input message.

Claims

1. A virtual controller client, the virtual controller client operating based on a mobile terminal so that the virtual controller client is allowed to remotely communicate with a virtual controller server running on a computer for remote key input on an application running on the computer, the virtual controller client comprising:

a button setting adjusting unit configured to receive button setting information including mapping relationship between key inputs to the application and virtual input messages from the virtual controller server, and to specify an arrangement and attributes of virtual buttons based on the received button setting information;

a user virtual button interface configured to generate a virtual button screen in which touch regions corresponding to the virtual buttons are visually displayed, and to display the virtual button screen on a touch screen of the mobile terminal;

a touch event filter configured to generate touch input messages that can be recognized as key inputs by the application, based on touch event objects that are generated from touch signals, of the touch regions corresponding to the virtual buttons, among touch signals input by the touch screen; and

a client message interfacing unit configured to convert the touch input message into a virtual input message in a form that can be recognized by the virtual controller server, and to output the virtual input message.

2. The virtual controller client of claim 1, wherein:

the user virtual button interface activates an acceleration sensor of the mobile terminal so that movements of the mobile terminal can be detected; and

wherein the virtual controller client further comprises;

an acceleration data filter configured to generate a movement input message that can be recognized as a key input by the application, based on acceleration data that is generated based on an acceleration signal generated by the acceleration sensor; and

the client message interfacing unit operable to convert the touch input message or the movement input message into a virtual input message in a form that can be recognized by the virtual controller server and to output the virtual input message.

3. A computer-readable storage medium storing a program that is run by the virtual controller client set forth in claim 1.

4. A virtual controller server, the virtual controller server operating on a computer so that the virtual controller server is allowed to remotely communicate with a virtual controller client running on a remote mobile terminal including a touch screen for remote key input on an application running on the computer, the virtual controller server comprising:

a button setting generating unit configured to generate button setting information including mapping relationship between key inputs to the application and virtual input messages;

a server message interfacing unit configured to transmit a setting message including the button setting information to the virtual controller client, and to receive a virtual input message from the virtual controller client, the virtual input message being generated based on a touch on the touch screen of the mobile terminal; and

a key mapping unit configured to identify a key input value mapped to the received virtual input message based on the button setting information.

5. The virtual controller server of claim 4, wherein the mobile terminal further comprises an acceleration sensor configured to detect movements, and wherein the server message interfacing unit operable to receive a virtual input message generated based on a movement of the mobile terminal.

6. The virtual controller server of claim 4, wherein the key mapping unit transfers a key input value to the application via a message transfer architecture of an operating system that runs the application on the computer.

7. The virtual controller server of claim 4, wherein the key mapping unit transfers a key input value to the application via an input and output application programming interface (API) of an operating system that runs the application on the computer.

8. A computer-readable storage medium storing a program that is run by the virtual controller server set forth in claim 5.

9. A remote control system, comprising:

a virtual controller server, being operated on a computer, for generating button setting information including mapping relationship between key inputs to an application running on the computer and virtual input messages, transfers the button setting information to a virtual controller client, for extracting a key input from a virtual input message received from the virtual controller client, and for providing the key input to the application; and

a virtual controller client, being operated on a mobile terminal including a touch screen and being configured to remotely communicate with the computer, for specifying an arrangement and attributes of virtual buttons based on the button setting information received from the virtual controller server, for generating a virtual button screen in which touch regions corresponding to the virtual buttons are visually displayed on the touch screen of the mobile terminal, for generating a touch input message that can be recognized as a key input by the application, based on touch event objects generated based on touch signals for the touch regions corresponding to the virtual buttons, and for converting the touch input message into a virtual input message in a form that can be recognized by the virtual controller server and output the virtual input message.

10. The remote control system of claim 9, wherein:

the mobile terminal further comprises an acceleration sensor configured to detect movements; and

the virtual controller client operates such that it activates an acceleration sensor of the mobile terminal so that movements can be detected, generates a movement input message that can be recognized as a key input by the application, based on acceleration data that is generated based on an acceleration signal generated by the acceleration sensor, and converts the touch input message or movement input message into a virtual input message in a form that can be received by the virtual controller server and then outputs the virtual input message.

11. A remote controller interfacing method, the remote controller interfacing method using a virtual controller server running on a computer and a virtual controller client running based on a remote mobile terminal including a touch screen for remote key input on an application running on the computer, the remote controller interfacing method comprising:

generating, by the virtual controller server, button setting information including mapping relationship between key inputs required by the application and virtual input messages to be transmitted by the virtual controller client, to be transferred to the virtual controller client;

specifying, by the virtual controller client, an arrangement and attributes of virtual buttons based on the button setting information, and displaying, by the virtual controller client, a virtual button screen in which the virtual button regions are visually arranged on the touch screen;

generating, by the virtual controller client, touch event objects based on a touch signal generated by the touch screen, and further a touch input message based on the valid touch event objects;

transferring, by the virtual controller client, a virtual input message generated based on the touch input message to the virtual controller server;

identifying, by the virtual controller server, a key input value mapped to the received virtual input message based on the button setting information; and

transferring, by the virtual controller server, the identified key input value to the application.

12. The remote controller interfacing method of claim 11, wherein:

the mobile terminal further comprises an acceleration sensor configured to detect movements; and

the remote controller interfacing method further comprises;

generating, by the virtual controller client, a movement input message that can be recognized as a key input by the application, based on acceleration data that is generated based on an acceleration signal generated by the acceleration sensor; and

converting, by the virtual controller client, the movement input message into a virtual input message in a form that can be received by the virtual controller server, and outputting, by the client, the virtual input message.

13. The remote controller interfacing method of claim 11, wherein the key input value identified by the virtual controller server is transferred to the application via a message transfer architecture of an operating system that runs the application on the computer.

14. The remote controller interfacing method of claim 11, wherein the key input value identified by the virtual controller server is transferred to the application via an input and output API of an operating system that runs the application on the computer.

15. A computer-readable storage medium storing a program that can implement the remote controller interfacing method set forth in claim 11.