Virtual interface devices

• US 9,342,239 B2
• Filed: 04/20/2011
• Issued: 05/17/2016
• Est. Priority Date: 04/21/2010
• Status: Active Grant

• Current Assignee: RealVNC Ltd.
• Original Assignee: RealVNC Ltd.
• Inventors: Lee, Andrew S. T., Morley, Jason Barrie, Boxall, Mark, Harter, Andy
• Primary Examiner(s): LEGGETT, ANDREA C.
• Application Number:
  US13/639,849
• Publication Number:
  US 20130174079A1
• Time in Patent Office: 1,854 Days
• Field of Search: 715/773
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A touch screen device configured to implement an auxiliary virtual keyboard on a touch screen, the device having touch/gesture sensing and image display functions, a processor, and memory storing control code and data defining the keyboard comprising a set of keys and a key layout map. The processor is configured to determine a size of a window in which the keyboard is to be displayed as a strip of keys; send instructions to display a partial view of the keyboard within the window so that a subset of keys are displayed, whereby the keyboard is scrollable in response to user input to alter the partial view to display a different subset of keys; receive touch data; map said touch data to a key using the key layout map to identify a touched key; and provide a response dependent on the identified touched key.

Claims

1. A touch screen device configured to implement an auxiliary virtual keyboard, the device comprising:

a touch sensitive display screen;

a touch screen operating system configured to provide touch and gesture sensing and image display functions on said touch sensitive display screen; and

a processor coupled to the touch sensitive display screen and to non-volatile memory storing processor control code and data defining said auxiliary virtual keyboard comprising a set of auxiliary keys and an auxiliary key layout map of said auxiliary keys;

wherein said touch screen device is connected to a remote device having a remote device operating system which is distinct from the touch screen operating system, and

further wherein said processor control code, when running on the processor is such that the processor is configured to;

receive a signal from said touch screen operating system that said auxiliary virtual keyboard is to be displayed on said touch sensitive display screen;

determine a size of an auxiliary keyboard window in which said auxiliary virtual keyboard is to be displayed as a strip of auxiliary keys;

instruct said touch screen operating system to display a partial view of said auxiliary virtual keyboard within said window so that a subset of said auxiliary keys are displayed to a user on said touch sensitive display screen, whereby said auxiliary virtual keyboard is scrollable in response to user input to alter the partial view of said auxiliary virtual keyboard to display a different subset of said auxiliary keys;

receive touch data from said touch screen operating system;

map said touch data to a key of said auxiliary keyboard using said auxiliary key layout map to identify a touched auxiliary key; and

provide a response to said remote device operating system dependent on said identified touched auxiliary key.

2. The touch screen device according to claim 1, wherein the response provided to the remote device operating system comprises generating an associated key-code and at least one key event corresponding to each identified touched auxiliary key.

3. The touch screen device according to claim 2, wherein the key event is one or more of a key-down event, a key-repeat event or a key-up event.

4. The touch screen device as claimed in claim 1 wherein said processor is further configured to modify an appearance of said virtual keyboard by co-animating a secondary auxiliary keyboard.

5. The touch screen device as claimed in claim 1 wherein said auxiliary keyboard window comprises a strip window along an edge of said display screen and wherein said auxiliary virtual keyboard is slidable longitudinally within said strip window.

6. The touch screen device as claimed in claim 5 wherein said processor control code is configured to provide markers defining groups of auxiliary keys within said auxiliary keyboard and to instruct said operating system to snap to said one or more groups of said auxiliary keys.

7. The touch screen device as claimed in claim 1 wherein said code to determine a size of an auxiliary keyboard window is configured such that at one or more edges of said auxiliary keyboard window only a portion of a said auxiliary key is displayed.

8. The touch screen device as claimed in claim 1 wherein said auxiliary keys comprise at least one key configured to perform the function of a combination of keys, wherein the combined keys comprise two or more keys selected from the group comprising tab, shift, Control, Escape, Alt, a function key, Insert, Delete, a cursor/arrow key, an Apple key, a Windows key, and an alphanumeric key.

9. The touch screen device as claimed in claim 1 wherein said linear strip of auxiliary keys forms a continuous loop which is scrollable within said auxiliary keyboard window.

10. The touch screen device as claimed in claim 1 wherein said auxiliary keyboard is implemented together with a virtual keyboard.

11. The touch screen device as claimed claim 10 wherein said code comprises code to determine a dimension of said virtual keyboard, and wherein said code to determine a size of said auxiliary keyboard view comprises code to adjust said size dependent on said dimension of said virtual keyboard.

12. The touch screen device as claimed in claim 11 wherein said code to determine a dimension of said virtual keyboard comprises code to receive from said operating system data defining an orientation of said device.

13. The touch screen device as claimed in claim 10 wherein said identified touched auxiliary key is a modifier key and said code to provide said response is configured to modify a value of a key touched on said virtual keyboard responsive to said identified touched modifier key.

14. The touch screen device as claimed in claim 13 wherein said code to modify said response comprises code to receive a value of a touched key of said virtual keyboard from said touch screen operating system and code to provide a modified value of said touched key back to said remote device operating system dependent on said identified touched modified key.

15. The touch screen device as claimed in claim 10 wherein said code to instruct said touch screen operating system to display a partial view of said auxiliary keyboard is configured to co-animate appearance of said auxiliary keyboard with said virtual keyboard on said display screen.

16. The touch screen device as claimed in claim 10 wherein said auxiliary keyboard view comprises a strip window adjacent a boundary of said virtual keyboard.

17. The touch screen device as claimed in claim 1, wherein said processor control code is further configured to

implement a remote access protocol which provides a mechanism for receiving a visual representation of the remote operating system on the touch screen device.

18. The touch screen device as claimed in claim 1, wherein said processor control code is further configured to provide a response in the form of a key-code together with a key-down event, key-up event or key-repeat event corresponding to each identified touched key on the remote device.

19. The touch screen device of claim 1, wherein the auxiliary keys are keys which are relevant only to the remote device.

20. The touch screen device of claim 1, wherein the auxiliary keys are keys which are redundant to control of the touch screen device.

21. A method of providing a virtual keyboard with a set of auxiliary keys on a display view of touch sensitive device having a touch device operating system, wherein said touch sensitive device is connected to a remote device having a remote device operating system which is distinct from the touch device operating system, the method comprising:

providing, using said touch device operating system a window for displaying the auxiliary keys in conjunction with the virtual keyboard on the display view of the touch sensitive device;

displaying a subset of the set of auxiliary keys within the window as a keyboard accessory comprising a strip, said window displaying a partial view of said strip;

scrolling said strip within said window in response to user touch on said display to alter said partial view and to alter said subset of displayed said keys;

receiving user input from a touched displayed one of said auxiliary keys at said touch device operating system; and

providing a response from said touch device operating system to said remote device operating system dependent on said identified touched auxiliary key, wherein said response corresponds to control of said remote device.

22. The method according to claim 21, wherein said displaying further comprising displaying a visual cue that said strip is scrollable.

23. The method according to claim 21, wherein said auxiliary key comprises a key to be pressed in conjunction with a key on said virtual keyboard, and wherein said method comprises modifying a key press of a key on said virtual keyboard responsive to said detected auxiliary key press.

24. The method as claimed in claim 21 wherein said displaying of said subset of said auxiliary keys comprises arranging said auxiliary keys such that an edge of said strip window passes through a said auxiliary key to display a partial image of a said auxiliary key.

25. The method as claimed in claim 21 wherein said displaying of said auxiliary keys comprises scrolling said strip of said auxiliary keys into said window when said auxiliary keys are first displayed in said touch screen.

26. The method of claim 21, wherein the auxiliary keys are keys which are relevant to control of the remote device and are redundant to control of the touch screen device.

27. A data carrier carrying processor control code for a touch screen device to implement an auxiliary virtual keyboard on a touch sensitive display screen of said touch screen device, wherein said touch screen device has a touch screen operating system configured to provide touch and gesture sensing and image display functions on said touch sensitive display screen, and wherein said touch screen device is connected to a remote device having a remote device operating system, the touch screen device including:

a processor coupled to said touch sensitive display screen and to non-volatile memory storing processor control code and data defining an image of a set of auxiliary keys and an auxiliary key layout map of said auxiliary keys; and

wherein said processor control code comprises code configured to;

receive a signal from said touch screen operating system that an auxiliary virtual keyboard is to be displayed on said touch sensitive display screen;

determine a size of an auxiliary keyboard window on said touch sensitive display screen in which said auxiliary keyboard is to be displayed as a strip of auxiliary keys;

instruct said touch screen operating system to display a partial view of said auxiliary virtual keyboard within said window so that a subset of said auxiliary keys are displayed to a user, whereby said auxiliary virtual keyboard is scrollable in response to user input to alter the partial view of said auxiliary virtual keyboard to display a different subset of said auxiliary keys;

receive touch data from said touch screen operating system;

map said touch data to a key of said auxiliary keyboard using said auxiliary key layout map to identify a touched auxiliary key; and

provide a response to said remote device operating system dependent on said identified touched auxiliary key.