Methods, devices and systems for entering textual representations of words into a computing device by processing user physical and verbal interactions with the computing device

• US 9,189,158 B2
• Filed: 03/28/2013
• Issued: 11/17/2015
• Est. Priority Date: 03/28/2013
• Status: Active Grant

• Current Assignee: Vasan Sun
• Original Assignee: Vasan Sun
• Inventors: Sun, Vasan
• Primary Examiner(s): Bashore, William
• Assistant Examiner(s): ORR, HENRY W
• Application Number:
  US14/237,156
• Publication Number:
  US 20140298177A1
• Time in Patent Office: 964 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 3/0233 Character input methods
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

Example embodiments relate to processing user interactions with a computing device, comprising receiving a user-initiated action performed on a character button, the character button representing a character, determining whether the user-initiated action is performed in a normal or abnormal operating manner. When a normal operating manner is determined, displaying the character on a graphical display. When an abnormal operating manner is determined: identifying a previously entered character preceding the character, activating a microphone and receiving, by the microphone, a spoken word, searching a subset of a database for a textual form of the received spoken word, the subset based on one or more of the character and the previously entered character, and displaying a correct textual form of the spoken word on a graphical display by amending one or more of the character and the previously entered character when one or more of the character and the previously entered character is inconsistent with the textual form of the spoken word found in the searching.

Claims

1. A computing device comprising:

a touchscreen display;

a microphone;

a computer-readable medium comprising a database; and

a processor in communication with the touchscreen display, the microphone, and the database, the processor operable to;

display a virtual keyboard on the touchscreen display, the virtual keyboard comprising a character button representing a character;

when the character button representing the character of the virtual keyboard is pressed in a normal operating manner, display the character in an editable section displayed on the touchscreen display; and

when the character button representing the character is pressed in an abnormal operating manner;

activate the microphone to receive a spoken word;

receive the spoken word and convert the spoken word into a form searchable in the database;

identify the character of the pressed character button and all immediately preceding non-null characters displayed in the editable section of the touchscreen display before the receiving of the spoken word;

identify a first subset of the database, the first subset comprising words having all of the identified immediately preceding non-null characters and the identified character of the pressed character button in any order;

search the first subset for a textual form of the received spoken word;

when the search of the first subset yields a match;

perform a first amending of one or more of the identified immediately preceding non-null characters and/or the identified character of the pressed character button displayed in the editable section based on the textual form of the received spoken word found in the search; and

display the textual form of the received spoken word in the editable section based on the first amending;

when the search of the first subset does not yield a match;

select a second subset comprising words having one or more of the identified immediately preceding non-null characters in an order different from the order of the identified immediately preceding non-null characters used in the first subset and not having the identified character of the pressed character button;

search the second subset for the textual form of the received spoken word;

perform a second amending of one or more of the identified immediately preceding non-null characters and the identified character of the pressed character button displayed in the editable section based on the textual form of the spoken word found in the said searching of the second subset; and

display the textual form of the spoken word found in the searching of the second subset in the editable section based on the second amending.

2. The computing device of claim 1, wherein the abnormal operating manner comprises a pressing down on the character button for greater than a threshold duration.

3. The computing device of claim 1, wherein the abnormal operating manner comprises swiping from the character button to an area different from the character button.

4. The computing device of claim 1, further comprising displaying more than one textual forms of the spoken word on the touchscreen display when the searching of the second subset returns more than one match, wherein one of the textual forms being user-selectable from among the more than one displayed textual forms.

5. The computing device of claim 1, further comprising storing, in the database, information pertaining to matched spoken words.