Dynamically located onscreen keyboard

• US 9,110,590 B2
• Filed: 11/30/2011
• Issued: 08/18/2015
• Est. Priority Date: 09/19/2007
• Status: Active Grant

• Current Assignee: Typesoft Technologies Incorporated (Apple Inc.)
• Original Assignee: Typesoft Technologies Incorporated (Apple Inc.)
• Inventors: Marsden, Randal J., Hole, Steve
• Primary Examiner(s): EURICE, MICHAEL
• Application Number:
  US13/308,416
• Publication Number:
  US 20120075192A1
• Time in Patent Office: 1,357 Days
• Field of Search: 345/168
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1662 Details related to the integrated keyboard
• CPC Class Codes: G06F 1/1664 Arrangements for ergonomically adjusting the disposition of keys of the integrated keyboard
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/0202 Constructional details or processes of manufacture of the input device
• CPC Class Codes: G06F 3/0216 Arrangements for ergonomically adjusting the disposition of keys of a keyboard for keyboards integrated in a laptop computer G06F1/1664
• CPC Class Codes: G06F 3/0219 Special purpose keyboards
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A touch-sensitive display surface having touch-capacitive and vibration sensors. This surface allows the user to rest their fingers on the keys of an onscreen keyboard and type as they would on a regular keyboard. As the user places their fingers on the touch screen, the system relocates the onscreen keyboard to the location where the fingers are resting. The touch sensors report the signal strength level of each key touched to a processor, but no keystroke is issued by the processor until a corresponding “tap” (i.e., vibration) is detected. When a tap is detected, the processor references the status of the touch capacitance sensors before, during, and/or immediately after the moment in time the tap occurred. The size, position, and orientation of the onscreen keyboard keys are dynamically set as determined by the user initiating a home-row definition event by resting their fingers momentarily on a virtual home-row.

Claims

1. A method comprising:

at an electronic device having a processor, memory, a display, a plurality of touch sensors coupled to the display for generating touch signals, and one or more vibration sensors for generating vibration signals;

detecting a home-row definition event, the home-row definition event including at least one of;

(1) one or more sense signals based on one or more sensed user contacts with the display at one or more touch locations, and (2) one or more vibration signals based on one or more sensed vibrations at one or more tapped locations on the display;

in response to detecting the home-row definition event, generating an onscreen keyboard having a plurality of keys, wherein respective positions of the plurality of keys of the onscreen keyboard are based on the touch locations or tapped locations of the home-row event;

detecting a key activation event for a first key of the plurality of keys in accordance with a respective vibration signal based on a respective sensed vibration at a respective tapped location on the display in proximity to the first key;

determining a drift based at least partially on a difference between the respective position of the first key of the onscreen keyboard and the respective tapped location associated with the detected key activation event for the first key; and

modifying the respective position of at least one of the plurality of keys of the onscreen keyboard based at least partially on the drift exceeding a predetermined threshold.

2. The method of claim 1, wherein generating the onscreen keyboard further comprises defining a location of the keyboard on the display based on the one or more sense signals.

3. The method of claim 1, further comprising:

generating a vibration at a predefined frequency in response to detecting the home-row definition event.

4. The method of claim 1, further comprising:

detecting a user finger contact drift within a threshold distance of an edge of the display; and

outputting a signal to an output device in response to detecting the user finger contact drift within the threshold distance of the edge of the display.

5. The method of claim 1, further comprising:

determining at least one next most likely key to be activated based one or more previous key activation events; and

uniquely displaying the determined at least one next most likely key.

6. The method of claim 1, further comprising:

determining a size value for at least one key of the keyboard based on a statistical probability model and at least one previous key activation event; and

altering the keyboard based on the determined size value for the at least one key.

7. The method of claim 1, further comprising:

causing an image of the keyboard presented in the display to be invisible in an active state based on a sensed first user action; and

causing the presented image of the keyboard to be invisible in an inactive state based on a sensed second user action.

8. The method of claim 1, wherein determining the drift further comprises:

adding the difference between the respective position of the first key and the respective tapped location associated with the detected key activation event for the first key to a cumulative total difference calculated from a plurality of previous key activation events detected on the onscreen keyboard.

9. The method of claim 1, wherein detecting the drift further comprises:

adding the difference between the respective position of the first key and the respective tapped location associated with the detected key activation event for the first key to a cumulative key-specific difference calculated from a plurality of previous key activation events detected for the first key.

10. The method of claim 1, further comprising:

generating a vibration at a predefined frequency in response to detecting at least one user finger within a threshold distance from at least one home-row key.

11. The method of claim 10, wherein generating the vibration further comprises altering an intensity of the generated vibration based on a processor determined distance of the at least one user finger from the at least one home-row key.

12. The method of claim 1, further comprising

generating an audio signal at a predefined frequency in response to detecting at least one user finger within a threshold distance from at least one home-row key.

13. The method of claim 12, wherein generating the audio signal further comprises altering an intensity of the audio signal based on a processor determined distance of the at least one user finger from the at least one home-row key.

14. The method of claim 1, further comprising:

changing an image of the keyboard presented on the display to be at least one of semitransparent or invisible when no user typing action is sensed for at least a predefined amount of time.

15. The method of claim 14, wherein changing the image of the keyboard further comprises causing the image of the keyboard to appear at least less transparent when a user typing action has been sensed.

16. The method of claim 1, wherein detecting the key activation event for the first key of the plurality of keys further comprise:

determining relative movement of one or more user fingers from one or more home-row keys based on corresponding sense signals generated by the one or more user fingers; and

generating the key activation event based on the determined relative movement and the respective vibration signal generated by one of the one or more user fingers.

17. The method of claim 16, wherein detecting the key activation event for the first key of the plurality of keys further comprise:

generating one or more candidate keys based on at least a portion of the corresponding sense signals generated by the one or more user fingers and the respective vibration signal; and

generating the key activation event by disambiguating the generated one or more candidate keys using a statistical probability model.

18. The method of claim 1, wherein detecting the home-row definition event further comprises detecting two or more sense signals that are active for at least a predefined amount time.

19. The method of claim 18, wherein generating the onscreen keyboard further comprises:

determining positions of home-row keys of the keyboard based on the locations of the two or more sense signals; and

determining positions of non-home-row keys of the keyboard based on the determined position of at least one of the home-row keys.

20. The method of claim 18, wherein generating the onscreen keyboard further comprises:

determining sizes of home-row keys of the keyboard based on the locations of the two or more sense signals; and

determining sizes of non-home-row keys of the keyboard based on the determined size of at least one of the home-row keys.

21. The method of claim 18, wherein generating the onscreen keyboard further comprises:

determining orientations of home-row keys of the keyboard based on the locations of the two or more sense signals; and

determining orientations of non-home-row keys of the keyboard based on the determined orientation of at least one of the home-row keys.

22. An electronic device, comprising:

a processor;

a display;

a plurality of touch sensors coupled to the display for generating sense signals;

one or more vibration sensors for generating vibration signals; and

memory having instructions stored thereon, the instructions, when executed by the processor, cause the processor to perform operations comprising;

detecting a home-row definition event, the home-row definition event including at least one of;

(1) one or more sense signals based on one or more sensed user contacts with the display at one or more touch locations, and (2) one or more vibration signals based on one or more sensed vibrations at one or more tapped locations on the display;

in response to detecting the home-row definition event, generating an onscreen keyboard having a plurality of keys, wherein respective positions of the plurality of keys of the onscreen keyboard are based on the touch locations or tapped locations of the home-row event;

detecting a key activation event for a first key of the plurality of keys in accordance with a respective vibration signal based on a respective sensed vibration at a respective tapped location on the display in proximity to the first key;

determining a drift based at least partially on a difference between the respective position of the first key of the onscreen keyboard and the respective tapped location associated with the detected key activation event for the first key; and

modifying the respective position of at least one of the plurality of keys in the onscreen keyboard based at least partially on the drift exceeding a predetermined threshold.

23. A non-transitory computer-readable storage medium storing instructions, the instructions, when executed by a processor, cause the processor to perform operations comprising:

detecting a home-row definition event, the home-row definition event including at least one of;

(1) one or more sense signals based on one or more sensed user contacts with a display at one or more touch locations, and (2) one or more vibration signals based on one or more sensed vibrations at one or more tapped locations on the display;

in response to detecting the home-row definition event, generating an onscreen keyboard having a plurality of keys, wherein respective positions of the plurality of keys of the onscreen keyboard are based on the touch locations or tapped locations of the home-row event;

detecting a key activation event for a first key of the plurality of keys in accordance with a respective vibration signal based on a respective sensed vibration at a respective tapped location on the display in proximity to the first key;

determining a drift based at least partially on a difference between the respective position of the first key of the onscreen keyboard and the respective tapped location associated with the detected key activation event for the first key; and

modifying the respective position of at least one of the plurality of keys in the onscreen keyboard based at least partially on the drift exceeding a predetermined threshold.