Directional input system with automatic correction

• US 7,286,115 B2
• Filed: 10/01/2003
• Issued: 10/23/2007
• Est. Priority Date: 05/26/2000
• Status: Expired due to Term

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Tegic Communications, Inc. (Microsoft Corporation)
• Inventors: Longe, Michael R., Palmer, Brian, Hullfish, Keith Charles, Brams, Doug
• Primary Examiner(s): Hjerpe; Richard
• Assistant Examiner(s): Shapiro; Leonid
• Application Number:
  US10/677,890
• Publication Number:
  US 20040070567A1
• Time in Patent Office: 1,483 Days
• Field of Search: 715/834, 345/156, 345/161
• US Class Current: 345/168
• CPC Class Codes: G06F 3/018 Input/output arrangements for oriental characters
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/04897 Special input arrangements or commands for improving display capability
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32

Abstract

A system associated with a text entry application, such as email or instant messaging, comprises an optional onscreen representation of a circular keyboard, a list of potential linguistic object matches, and a message area where the selected words are entered. The circular keyboard is manipulated via a hardware joystick or game-pad with analog joystick or omni-directional rocker switch built therein. The user points the joystick in the general direction of the desired letter, and then continues pointing roughly to each letter in the desired word. Once all letters have been roughly selected, buttons are used to select a specific word from the list of potential matches and send the selected word to the message area.

Claims

1. A text entry input system, comprising:

a direction selector to individually point in a direction of letters to collectively form an intended linguistic object, where each letter comprises a linguistic object subcomponent;

a collection of linguistic objects;

an output device with a text display area;

a processor, comprising;

a difference calculation module configured to output, for each act of pointing, a letter and associated weight value based upon factors including at least an angular difference between an actual direction of pointing and pre-assigned directions of said letters;

an object search engine configured to construct a predicted list of linguistic objects based on the output letters and weight values;

a selection component to facilitate user selection of a desired linguistic object from the predicted list of linguistic objects.

2. The system of claim 1, further comprising an on-screen keyboard representation of a ring of the letters.

3. The system of claim 2, wherein said on-screen keyboard is of any shape selected from a group comprising circle, square, oval, polygon.

4. The system of claim 1, further comprising a set of compass point letters, said compass point letters being placed at positions around a linguistic object selection list, in a separate on-screen compass area, or around said directional selector.

5. The system of claim 2, wherein said letters have bottoms towards the center of said ring.

6. The system of claim 2, wherein said letters have an alphabetical order, QWERTY order, or Dvorak order.

7. The system of claim 2, wherein said letters start at the 12 o'"'"'clock or 9 o'"'"'clock position.

8. The system of claim 2, wherein said letters have a moving starting position.

9. The system of claim 2, wherein said letters have a clockwise or counterclockwise layout.

10. The system of claim 2, wherein each of said letters occupies different amount of radians depending upon use frequency.

11. The system of claim 1, wherein a number of characters are printed around said directional selector.

12. The system of claim 1, wherein said directional selector comprises a joystick or an omni-directional rocker switch.

13. The system of claim 1, wherein each letter comprises any subcomponent or combination of one or more of the following forming an incomplete part of one of the linguistic objects:

an alphabetic letter, numeric digit, symbol, character;

a sub-word component from a non-English language including one or more strokes, radicals, jamos, kana, punctuation symbols, digits.

14. The system of claim 1, further comprising at least four buttons independent of said directional selector.

15. The system of claim 1, where the weight values represent estimated probabilities that actually pointed to letters represent letters intended by the user.

16. The system of claim 12, wherein said joystick or omni-directional rocker switch is a component of a multi-function video game controller.

17. The system of claim 1, wherein said system provides auditory or visual feedback on each movement of said directional selector.

18. The system of claim 17, wherein said visual feedback is a solid or gradient-fill pie wedge shape appearing on said on-screen keyboard, said pie wedge shape being centered on a current selected direction.

19. The system of claim 1, wherein said linguistic objects are ordered according to a linguistic model.

20. The system of claim 19, where said linguistic model includes one or more of:

frequency of occurrence of a linguistic object in formal or conversational written text;

frequency of occurrence of a linguistic object when following a preceding linguistic object or linguistic objects;

proper or common grammar of the surrounding sentence;

application context of current linguistic object entry; and

recency of use or repeated use of the linguistic object by the user or within an application program.

21. The system of claim 1, wherein said list of predicted linguistic objects are ordered by a combination value of a calculated weighted difference value and a linguistic model.

22. The system of claim 21, wherein said linguistic model comprises one or more of:

frequency of occurrence of a linguistic object in formal or conversational written text;

frequency of occurrence of a linguistic object when following a preceding linguistic object or linguistic objects;

proper or common grammar of the surrounding sentence;

application context of current linguistic object entry; and

recency of use or repeated use of the linguistic object by the user or within an application program.

23. The system of claim 21, wherein the linguistic object with the highest combination value is selected by default.

24. The system of claim 1, wherein the linguistic objects longer than the number of actions of direction selector are included in said list of predicted linguistic objects.

25. The system of claim 1, further comprising a means for extending a selected linguistic object with completions.

26. The system of claim 25, wherein said completions are displayed in a pop-up list after a button press or directional input.

27. The system of claim 1, further comprising a means for precisely selecting said letters of said linguistic object.

28. The system of claim 1, wherein an exact spelling sequence is displayed in said text display area.

29. The system of claim 1, wherein the last entered letter is indicated in said exact spelling sequence.

30. The system of claim 2, wherein the last entered letter is indicated in said on-screen keyboard.

31. The system of claim 1, further comprising a means to change the last entered letter.

32. A text input method using a directional input device, wherein each direction entered corresponds, directly or indirectly, to one or more linguistic object subcomponents according to a predetermined mapping, said method comprising the steps of:

for each user act of pointing the directional input device in a direction, preparing an output of candidate linguistic object subcomponents and associated probability weightings based upon factors including at least an angular difference between the direction indicated by the directional input device and pre-assigned directions of said linguistic object subcomponents according to the predetermined mapping;

for a given sequence of user acts of pointing, utilizing the output to construct a list of predicted linguistic objects potentially representing a linguistic object intended by the user through entry of the given sequence;

facilitating user selection of a desired linguistic object from said list.

33. The method of claim 32, the directions indicated by the directional input device comprising angular direction, wherein the angular direction is derived from recording the X-Y offset of the directional input device and converting that offset into an angular notation comprising radians, gradients, or degrees.

34. The method of claim 33, wherein the conversion is a variation on the standard Cartesian to Polar formula of Angle=arctan(Y/X).

35. The method of claim 32, wherein the candidate linguistic object subcomponents can be one of any number of adjacent or nearby letters or symbols.

36. The method of claim 32, further comprising utilizing a linguistic model to order said list of predicted linguistic objects according to likelihood of intended selection by the user.

37. The method of claim 32, wherein the linguistic objects comprise words and the linguistic object subcomponents comprise letters.

38. The method of claim 36, wherein the order of said list of predicted linguistic objects is based on a combination of the weightings and the linguistic model.

39. The method of claim 36, wherein the linguistic model comprises one or more of:

frequency of occurrence of a linguistic object in formal or conversational written text;

frequency of occurrence of a linguistic object when following a preceding linguistic object or linguistic objects;

proper or common grammar of the surrounding sentence;

application context of current linguistic object entry; and

recency of use or repeated use of the linguistic object by the user or within an application program.

40. The method of claim 32, wherein said directional input device is associated with an on-screen keyboard.

41. The method of claim 40, wherein said on-screen keyboard comprises a ring of letters, numbers or other symbols.

42. The method of claim 40, wherein said on-screen keyboard is represented in Polar or Cartesian coordinate system for calculation.

43. The method of claim 32, wherein construction of said list of predicted linguistic objects includes retrieving linguistic objects from a vocabulary database, and wherein a plurality of linguistic objects stored in said vocabulary database is ordered according to a linguistic model.

44. The method of claim 43, where said linguistic model comprises one or more of:

frequency of occurrence of a linguistic object in formal or conversational written text;

frequency of occurrence of a linguistic object when following a preceding linguistic object or linguistic objects;

proper or common grammar of the surrounding sentence;

application context of current linguistic object entry; and

recency of use or repeated use of the linguistic object by the user or within an application program.

45. The method of claim 43, wherein said plurality of linguistic objects is stored in a mixed case format in said vocabulary database.

46. The method of claim 43, wherein said vocabulary database further comprises a user database which stores linguistic objects added by the user.

47. The method of claim 46, wherein said linguistic objects added by the user in said user database are ordered by length of linguistic object and recency of use.

48. The method of claim 43, wherein said vocabulary database further comprises a recency database to support linguistic object prediction based on recency of use.

49. The method of claim 36, where the order of each linguistic object in the list is based at least in part upon a mathematical function of probability weightings for all linguistic object subcomponents in the linguistic object.

50. The method of claim 32, where the factors additionally include, for each linguistic object subcomponent, a frequency of use of that linguistic object subcomponent.

51. The method of claim 32, wherein the user selects a partial linguistic object and continues with more directional inputs.

52. The method of claim 51, wherein said list of predicted linguistic objects is filtered to only include linguistic objects that begin with said selected partial linguistic object.

53. The method of claim 32, wherein the user selects an entry from said list of predicted linguistic objects and the highest-ranked linguistic object containing said selected entry remains selected upon the input of additional linguistic object subcomponents until said entry becomes invalid by further addition of linguistic object subcomponents.

54. The method of claim 32, wherein said directional input device includes a set of buttons.

55. The method of claim 32, wherein said directional input device includes a joystick or omni-directional rocker switch.

56. The method of claim 40, wherein said on-screen keyboard further comprises a smart punctuation symbol, said smart punctuation symbol when retrieved is automatically interpreted as a punctuation symbol, diacritic mark or tonal indication at the place in the input sequence where a matching punctuation symbol, diacritic mark or tonal indication occurs in predicted linguistic objects.

57. The method of claim 32, wherein a set of buttons or a second directional input device can be used alone or with said directional input device, separately or simultaneously, to switch or choose input modes, to change from input to word selection, or to invoke other functions.

58. The method of claim 32, further comprising the steps of:

invoking an undo means after selecting a linguistic object from said list of predicted linguistic objects; and

re-displaying said list.

59. The method of claim 32, further comprising the steps of:

selecting a linguistic object from a text message; and

displaying subcomponents of said linguistic object as if said subcomponents had been entered exactly and constructing a predicted list of linguistic objects based on the displayed subcomponents.

60. A text entry input module for use with user interface components including a direction indicator and a output device with a text display area, the text entry input module comprising:

a database of linguistic objects;

a predetermined set of linguistic object subcomponents;

where a predetermined relationship exists between said linguistic object subcomponents and different assigned angular directions of the direction indicator;

a calculation module to apply the predetermined relationship to each user-submitted direction entered via the direction indicator to provide an output, said output including;

(1) multiple predicted linguistic object subcomponents including a group of linguistic object subcomponents whose assigned angular directions are nearest the user-submitted directions, and (2) for each predicted linguistic object subcomponent, an associated proximity weighting proportional to an angular difference between the user submitted direction and the angular direction assigned to the predicted linguistic object subcomponent;

an object search engine configured to utilize the output to retrieve from the database a list of predicted linguistic objects potentially representative of the user-submitted directions;

a linguistic object module programmed to utilize at least one predetermined linguistic model to order said list of potential linguistic objects according to likelihood of intended selection by the user;

a selection component to facilitate user selection of a desired linguistic object from said ordered list of predicted linguistic objects.

61. A text entry input module for use with user interface components including a direction indicator and an output device with a display, the text entry input module comprising:

a vocabulary database of linguistic objects;

a mapping between angular directions of the direction indicator and linguistic object subcomponents;

a calculation module to apply the mapping to each user-submitted direction entered via the direction indicator to provide an output including;

multiple potentially user-intended linguistic object subcomponents and associated proximity weightings;

an object search engine configured to retrieve a list of predicted linguistic objects from the vocabulary database based upon said calculation module output;

a linguistic object module programmed to utilize a linguistic model to order said list of predicted linguistic objects according to likelihood of intended selection by the user; and

a selection component to facilitate user selection of a desired linguistic object from said ordered list of predicted linguistic objects.

62. A computer readable storage medium tangibly embodying a program of instructions executable by a digital data processing machine to perform text input operations comprising:

receiving machine-readable signals representing a series of user-submitted directional inputs entered via a directional input tool, the series having an order;

where directional inputs of the directional input tool correspond to different linguistic object subcomponents according to a predetermined mapping;

for each user-submitted directional input, based upon that directional input alone, estimating multiple corresponding subcomponents that the user might have intended by such directional input, and providing a weighting value that the user intended each such subcomponent;

assembling the different ones of the estimated subcomponents to construct multiple different proposed linguistic objects that the user might have intended by the series of directional inputs, where each proposed object includes one estimated subcomponent for each user-submitted directional input, the subcomponents occurring in the proposed object in the same order as the series of user-submitted directional inputs;

facilitating selection of a desired one of the proposed objects.

63. The medium of claim 62, where each linguistic object subcomponent comprises at least one of the following:

an alphabetic letter, numeric digit, symbol, character;

a sub-word component from a non-English language including one or more strokes, radicals, jamos, kana, punctuation symbols, digits;

a subcomponent or combination of one or more of the foregoing.

64. The medium of claim 62, the assembling operation further comprising:

ordering proposed linguistic objects according to considerations including one or more of the following;

presence in a predetermined vocabulary, presence in a user-defined list, frequency of general usage.

65. The medium of claim 62, the estimating operation further comprising:

for each user-submitted directional input, identifying linguistic object subcomponents within a predetermined angular range thereof according to the predetermined mapping;

limiting the estimated subcomponents to those identified.

66. The medium of claim 62, the operation of providing a weighting value comprising at least one of the following:

providing a weighting value for each linguistic object subcomponent according to angular proximity between the subcomponent'"'"'s corresponding actual user-submitted directional input and the directional input exactly mapped to the subcomponent;

providing a weighting value depending at least in part on frequency of general usage of the subcomponent.

67. The medium of claim 62, the operations further comprising:

responsive to receiving each machine-readable signal representing a user-submitted directional input, displaying a pie wedge indicating a current direction of the directional input tool and a range of linguistic object subcomponents within that range.

68. The medium of claim 67, the operations further comprising:

narrowing the displayed pie wedge and corresponding range in proportion to magnitude of the directional input.

69. The medium of claim 62, where the predetermined mapping is such that linguistic object subcomponents occupy greater angular ranges responsive to factors including greater frequency of general usage.

70. The medium of claim 62, where the assembling operation further comprises identifying supplemental linguistic objects that contain the proposed linguistic objects and include further subcomponents, and including the supplemental linguistic objects in the proposed linguistic objects.

71. The medium of claim 62, where the assembling operation further comprises ordering the proposed linguistic objects according to one or more of the following:

word length, ordinal ranking, weighting value of proposed linguistic object subcomponents, frequency of general usage, recency of use, appearance in a user-defined list.

72. A computer-assisted method of text entry, the operations comprising:

by using a joystick having predefined mapping between different actual radial directions and different text characters, a user sequentially pointing in an intended radial direction toward intended characters of at least part of an intended word;

the computer determining angular similarity between each actual pointing direction and radial directions of different characters in the mapping, and using the determined angular similarity to compute different possible combinations of intended characters that could form the intended word and their associated likelihoods, and presenting a list of potential matching words to the user, where the presentation favors potential matching words according to factors including an aggregate angular similarity of constituent characters of a potential matching word to the actual pointing directions;

the user selecting the Intended word from the list;

the computer displaying the selected word in a human-readable message composition display area.