Systems and Methods for Haptically-Enhanced Text Interfaces

• US 20110061017A1
• Filed: 09/09/2009
• Published: 03/10/2011
• Est. Priority Date: 09/09/2009
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Birnbaum, David, Fleming, Jason, Ullrich, Chris, Vachon, Craig
• Granted Patent: US 9,317,116 B2
• Time in Patent Office: Days
• US Class Current: 715/780
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs

Abstract

When units of serialized input are analyzed by an engine or algorithm that provides a plurality of entry suggestions for competing or replacing a unit of the serialized input, one or more haptic effects can be identified and provided in conjunction with display, selection, or use of one or more of the entry suggestions to aid in the use of the engine or algorithm. The haptic effects can represent a confidence level with respect to one or more of the entry suggestions or may otherwise indicate the relative success of the engine/algorithm. For example, the serialized input may comprise characters input to a mobile or other device. The engine or algorithm may suggest words or phrases with varying degrees of confidence, with the confidence with respect to some or all of the suggestions communicated by use of haptic effects. Additional embodiments include dynamic text input interfaces, haptically-enhanced serial presentation of content, and interfaces that provide haptic effects at a first device based on remote text entry, display, reading, or other activity at a different device.

Claims

1. A method, comprising:

receiving, at a computing device, input data representing a series of input selections, the input selections comprising characters, words, or phrases;

ascertaining, by the computing device, at least one text entry suggestion generated based on the received input data;

determining, by the computing device, a confidence level associated with each text entry suggestion;

identifying, by the computing device, a haptic effect from a plurality of potential haptic effects based at least in part on the confidence levels; and

outputting, by the computing device, at least one haptic signal configured to cause an actuator to generate the identified haptic effect.

2. The method set forth in claim 1, wherein the text entry suggestion comprises at least one of a completion suggestion based on the received input data or a correction suggestion based on the received input data.

3. The method set forth in claim 1, wherein the confidence level is based on a probability value and identifying comprises:

identifying the haptic effect from a function correlating probability values to the plurality of potential haptic effects.

4. The method set forth in claim 1, further comprising:

providing an interface, the interface configured to present a plurality of text entry suggestions and receive a selection of one of the plurality of text entry suggestions; and

wherein identifying the haptic effect comprises identifying the haptic effect associated with the selected one of the plurality of text entry suggestions.

5. The method set forth in claim 1,

wherein generating comprises generating a plurality of text entry suggestions;

wherein determining comprises determining a confidence level associated with each of the text entry suggestions; and

wherein identifying comprises identifying a haptic effect for each of the text entry suggestions.

6. The method set forth in claim 5, further comprising providing a display signal to display each of the text entry suggestions in sequence, wherein outputting at least one haptic signal comprises outputting a signal to generate the haptic effect corresponding to each text entry suggestion while the text entry suggestion is displayed.

7. The method set forth in claim 1,

wherein generating comprises generating a plurality of text entry suggestions;

wherein determining comprises determining a confidence level associated with each of the text entry suggestions; and

wherein identifying comprises identifying a single haptic effect based on evaluating the confidence levels of the text entry suggestions in aggregate.

8. The method set forth in claim 1, further comprising receiving input selecting the text entry suggestion for use as an input value;

wherein outputting occurs after the completion selection has been selected for use as an input value.

9. A system comprising a processor with access to a computer-readable medium embodying program components and an actuator, the program components comprising:

a user interface module configured to receive input data representing a series of characters, words, or phrases;

a text engine configured to generate a text entry suggestion based on the received input data and determine a confidence level associated with the text entry suggestion; and

a haptic effect module configured to identify a haptic effect from a plurality of potential haptic effects based at least in part on the confidence level and output at least one haptic signal configured to cause the actuator to generate the identified haptic effect.

10. The system set forth in claim 9, wherein the confidence level is based on a probability value and the haptic effect module is configured to identify the haptic effect from a function correlating probability values to the plurality of potential haptic effects.

11. The system set forth in claim 9, further comprising a display,

wherein the user interface module is configured to present a plurality of text entry suggestions on the display and to receive a selection of one of the plurality of text entry suggestions;

and

wherein the haptic effect module is configured to identify the haptic effect associated with the selected one of the plurality of text entry suggestions.

12. The system set forth in claim 9,

wherein generating comprises generating a plurality of text entry suggestions;

wherein determining comprises determining a confidence level associated with each of the text entry suggestions; and

identifying comprises identifying a single haptic effect based on evaluating the confidence levels of the text entry suggestions in aggregate.

13. The system set forth in claim 9,

wherein the haptic effect module is configured to output the at least one haptic signal after the user interface module has received selection of a text entry suggestion for use as an input value.

14. The system set forth in claim 9, wherein the processor, actuator, and computer-readable medium are comprised in a system comprising at least one of a mobile telephone, personal digital assistant, or a computer.

15. The system set forth in claim 9, wherein the processor and actuator are comprised in separate devices in communication with one another.

16. A tangible computer-readable medium embodying program code executable by a computing system, the program code comprising:

program code for receiving serialized input;

program code for generating an entry suggestion to replace a unit of the serialized input based on the received input data;

program code for determining a confidence level associated with the entry suggestion; and

program code for identifying a haptic effect from a plurality of potential haptic effects based at least in part on the confidence level.

17. The computer-readable medium set forth in claim 16, wherein the serialized input comprises input data representing a series of characters, words, or phrases and the entry suggestion comprises a text entry suggestion.

18. The computer readable medium set forth in claim 17, further comprising program code for causing the computing system to output at least one haptic signal configured to cause an actuator to generate the identified haptic effect.

19. The computer-readable medium set forth in claim 18, wherein the confidence level comprises a probability value and identifying comprises identifying the haptic effect from a function correlating probability values to the plurality of potential haptic effects.

20. The computer-readable medium set forth in claim 18, further comprising program code for providing an interface, the interface configured to present a plurality of text entry suggestions and receive a selection of one of the plurality of text entry suggestions,

wherein identifying the haptic effect comprises identifying the haptic effect associated with the selected one of the plurality of text entry suggestions.

21. The computer-readable medium set forth in claim 18,

wherein generating comprises generating a plurality of text entry suggestions;

wherein determining comprises determining a confidence level associated with each of the text entry suggestions; and

wherein identifying comprises identifying a haptic effect for each of the text entry suggestions.

22. The computer-readable medium set forth in claim 18,

wherein generating comprises generating a plurality of text entry suggestions;

wherein determining comprises determining a confidence level associated with each of the text entry suggestions; and

wherein identifying comprises identifying a single haptic effect based on evaluating the confidence levels of the text entry suggestions in aggregate.

23. The computer-readable medium set forth in claim 18, further comprising program code for receiving input selecting the text entry suggestion for use as an input value, wherein outputting occurs after the completion selection has been selected for use as an input value.

24. A system comprising an actuator and a processor with access to a computer-readable medium tangibly embodying program components, the program components comprising:

a module configured to access serialized data and determine a plurality of units of the serialized data, the units comprising at least one of words, punctuation, or characters;

a user interface module configured to display at least some of the units individually and in sequence; and

a haptic effect module configured to output at least one haptic signal in association with the display of a unit, wherein the haptic signal is output to cause the actuator to generate a haptic effect before, after, or while the unit is displayed.

25. A method comprising:

displaying, by a computing device comprising a display, an on-screen keyboard;

hiding at least a portion of the keyboard and displaying a keyboard restore control;

determining if the keyboard restore control is selected;

in response to selection of the keyboard restore control, displaying the on-screen keyboard;

determining if a discontinuity in selection of the keyboard or keyboard restore control has occurred;

identifying a key reached when the discontinuity occurred;

hiding at least a portion of the keyboard; and

storing data identifying the key in memory for use as an input value.

26. The method set forth in claim 25,

wherein displaying comprises displaying the keyboard on a touch-enabled screen, and

wherein determining a discontinuity comprises determining if contact with the screen is broken for a time period in excess of a threshold time period.

27. The method set forth in claim 25, further comprising:

before the discontinuity is reached, providing a haptic effect when a second key of the on-screen keyboard is reached after a first key of the on-screen keyboard has been reached.

28. A system comprising:

an actuator;

a display;

a sensor; and

a processor with access to a computer-readable medium embodying program components, the program components comprising;

a user interface module that configures the system to receive input data representing a series of characters, words, or phrases;

a text engine configured to generate a plurality of text entry suggestions based on the received input data and determine a confidence level associated with each text entry suggestion;

a communication application that configures the system to access a serialized data stream and status information indicating activity by a remote user; and

a haptic effect module that configures the system to identify a haptic effect from a plurality of potential haptic effects based at least in part on the confidence level and output at least one haptic signal configured to cause the actuator to generate the identified haptic effect,

wherein the user interface module configures the system to receive input data using a dynamic key or text input interface rendered on the display, the dynamic key or text input interface configured to selectively display an onscreen keyboard in response to selection of a keyboard restore control and hide the keyboard in response to selection of a key,

wherein the user interface module configures the system to display units of the serialized data stream individually and in sequence in response to selection of a serialized display mode, selection of the serialized data mode determined based on motion detected by the sensor, and

wherein the haptic effect module configures the system to provide a haptic effect in response to the status information.