Adaptive virtual keyboard for handheld device

• US 9,195,818 B2
• Filed: 05/13/2014
• Issued: 11/24/2015
• Est. Priority Date: 06/16/2009
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ferren, Bran
• Primary Examiner(s): Lee, Jr., Kenneth B
• Application Number:
  US14/276,661
• Publication Number:
  US 20140247221A1
• Time in Patent Office: 560 Days
• Field of Search: 345168-173
• US Class Current: 1/1
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G03B 3/00 Focusing arrangements of general interest for cameras, projectors or printers
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/40 by quorum, i.e. whereby two or more security principals are required
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06K 2207/1011 Aiming
• CPC Class Codes: G06K 7/10811 Focalisation
• CPC Class Codes: G06Q 20/3276 using a pictured code, e.g. barcode or QR-code, being read by the M-device
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 20/40145 : Biometric identity checks
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/165 : using facial parts and geometric relationships
• CPC Class Codes: G06V 40/19 : Sensors therefor
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/45 : Detection of the body part being alive
• CPC Class Codes: H04L 63/083 : using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04M 1/72448 : with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 2250/52 : including functional features of a camera
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 : using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 : for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

In various embodiments, the size, shape, and arrangement of keys on a virtual keyboard may be determined based on touchscreen contacts made by the user. Further, the actual contact patch made by the user may be analyzed to interpret which point of contact was intended, and other factors such as spelling and context may also be considered. These factors may be determined based on a calibration session and/or on continuing inputs during operation of the keyboard, and applied to future operational interpretations of the touchscreen contacts.

Claims

1. A handheld electronic device, comprising:

a touchscreen for displaying a virtual keyboard, the virtual keyboard having a plurality of rows of keys, the handheld electronic device including assumptions about the shape and orientation for a contact patch for each row of keys of the plurality of rows, wherein the handheld electronic device is to;

sense a touch on the touchscreen of the handheld electronic device, wherein sensing the touch includes sensing a contact patch for the touch on the touchscreen;

determine a location of a centroid for the contact patch of the touch; and

determine a location of an active point for the contact patch;

wherein the location of the active point is different than the location of the centroid; and

wherein determination of the location of the active point for the touch includes consideration of the contact patch and the assumptions about the shape and orientation for a contact patch for each row of keys of the plurality of rows.

2. The handheld electronic device of claim 1, wherein said determining the location of the centroid comprises:

receiving readings from multiple sensor points on the touchscreen located within the contact patch; and

calculating the centroid from the multiple sensor points.

3. The handheld electronic device of claim 1, wherein said determining the location of the active point is based at least in part on one or more factors selected from a list consisting of:

the centroid of the contact patch;

a size of the contact patch;

a shape of the contact patch;

a difference in pressure sensed at multiple points within the contact patch;

a location on the virtual keyboard of the contact patch;

a positional relationship between the centroid and a keyboard key that is at least partly overlapped by the contact patch; and

a previous history of the positional relationship between the contact patch and the keyboard key.

4. The handheld electronic device of claim 1, wherein said determining a location of the active point is to be performed during a calibration procedure for the virtual keyboard.

15. The handheld electronic device of claim 1, wherein the handheld electronic device is further to:

sense whether the handheld electronic device is in a one-handed or two-handed mode of operation for the virtual keyboard; and

adjust the determination of the active point based on the mode of operation.

16. The handheld electronic device of claim 1, wherein the handheld electronic device is further to make an assumption for a middle row of the plurality of rows of the virtual keyboard that is interpolated between an assumption for an inner row of the virtual keyboard and an assumption for an outer row of the virtual keyboard.

5. A method for adaptation of a virtual keyboard of a handheld electronic device, comprising:

sensing a touch on a touchscreen of a handheld electronic device, the handheld electronic device including a virtual keyboard having a plurality of rows of keys, the handheld electronic device including assumptions about the shape and orientation for a contact patch for each row of keys of the plurality of rows, wherein sensing the touch including sensing a contact patch for the touch on the touchscreen;

determining a location of a centroid for the contact patch of the touch; and

determining a location of an active point for the contact patch;

wherein the location of the active point is different than the location of the centroid; and

wherein determination of the location of the active point for the touch includes consideration of the contact patch and the assumptions about the shape and orientation for a contact patch for each row of keys of the plurality of rows.

6. The method of claim 5, wherein said determining the active point is additionally based at least in part on one or more factors selected from a list consisting of:

the centroid of the contact patch;

a size of the contact patch;

a shape of the contact patch;

a difference in pressure sensed at multiple points within the contact patch;

a location on the virtual keyboard of the contact patch;

a positional relationship between the contact patch and a keyboard key that is at least partly overlapped by the contact patch; and

a previous history of the positional relationship between the contact patch and the keyboard key.

7. The method of claim 5, wherein said determining the location of the active point is performed during an initial calibration procedure for the virtual keyboard.

8. The method claim 5, wherein said determining the location of the active point is performed during operational usage of the virtual keyboard.

17. The method of claim 5, further comprising:

sensing whether the handheld electronic device is in a one-handed or two-handed mode of operation for the virtual keyboard; and

adjusting the determination of the active point based on the mode of operation.

18. The method of claim 5, wherein an assumption for a middle row of the plurality of rows of the virtual keyboard is interpolated between an assumption for an inner row of the virtual keyboard and an assumption for an outer row of the virtual keyboard.

9. An article comprising:

a computer-readable storage medium that contains instructions, which when executed by one or more processors result in performing operations comprising;

sensing a touch on a touchscreen of a handheld electronic device, the handheld electronic device including a virtual keyboard having a plurality of rows of keys, the handheld electronic device including assumptions about the shape and orientation for a contact patch for each row of keys of the plurality of rows, wherein sensing the touch including sensing a contact patch for the touch on the touchscreen;

determining a location of a centroid for a contact patch for the touch; and

determining a location of an active point for the contact patch;

wherein the location of the active point is different than the location of the centroid; and

wherein determination of the location of the active point for the touch includes consideration of the contact patch and the assumptions about the shape and orientation for a contact patch for each row of keys of the plurality of rows.

10. The article of claim 9, wherein the operation of determining the active point is based at least in part on one or more factors selected from a list consisting of:

a centroid of the contact patch;

a size of the contact patch;

a shape of the contact patch;

a difference in pressure sensed at multiple points within the contact patch;

a location on the virtual keyboard of the contact patch;

a positional relationship between the contact patch and a keyboard key that is at least partly overlapped by the contact patch;

a previous history of the positional relationship between the contact patch and the keyboard key.

11. The article of claim 9, wherein the operation of determining the location of the active point is performed during an initial calibration procedure for the virtual keyboard.

12. The article of claim 9, wherein the operation of determining the location of the active point is performed during operational usage of the virtual keyboard.

13. The article of claim 9, wherein the operations further comprise:

sensing whether the handheld electronic device is in a one-handed or two-handed mode of operation for the virtual keyboard; and

adjusting the determination of the active point based on the mode of operation.

14. The article of claim 9, wherein an assumption for a middle row of the plurality of rows of the virtual keyboard is interpolated between an assumption for an inner row of the virtual keyboard and an assumption for an outer row of the virtual keyboard.