Haptically enabled messaging

• US 8,315,652 B2
• Filed: 05/18/2007
• Issued: 11/20/2012
• Est. Priority Date: 05/18/2007
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Grant, Danny A., Ramsay, Erin B., Gregorio, Pedro, Fleming, Jason D., Flaherty, Natasha M.
• Primary Examiner(s): Hu, Jinsong
• Assistant Examiner(s): Nguyen, Michael
• Application Number:
  US11/750,694
• Publication Number:
  US 20080287147A1
• Time in Patent Office: 2,013 Days
• Field of Search: 455/466, 455/412.1, 455/412.2
• US Class Current: 455/466
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04M 1/72436 for text messaging, e.g. short messaging services [SMS] or e-mails
• CPC Class Codes: H04M 2250/22 including a touch pad, a touch sensor or a touch detector digitizers for touch screens G06F3/041
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/42042 Notifying the called party of information on the calling party details within substation equipment H04M1/57, signalling details H04Q3/72
• CPC Class Codes: H04M 3/42382 Text-based messaging services in telephone networks such as PSTN/ISDN, e.g. User-to-User Signalling or Short Message Service for fixed networks
• CPC Class Codes: H04W 4/14 Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD]

Abstract

A mobile terminal device receives a haptically enabled message that includes text and at least one haptic feedback command. The device determines the timing of the haptic feedback command within the message. When the device receives an indication that the message is being displayed by a user, the device synchronizes a haptic effect corresponding to the haptic feedback command. The synchronizing is based on the timing and the indication.

Claims

1. A method of operating a mobile terminal device comprising:

accessing a haptically enabled message, the message comprising text and a plurality of haptic feedback commands, wherein at least two haptic feedback commands of the plurality of haptic feedback commands occur in succession within the message;

determining a timing of each haptic feedback command of the plurality of haptic feedback commands within the message;

assigning a precedence order to the at least two haptic feedback commands;

receiving an indication that the message is being displayed on said mobile terminal device; and

synchronizing the initiation of a plurality of haptic effects corresponding to fewer than all of the plurality of haptic feedback commands based on the received indication, the assigned precedence order, and the determined timing associated with the plurality of haptic feedback commands,

wherein the synchronizing comprises;

determining a time delay for each haptic feedback command based on the timing; and

initiating a haptic effect corresponding to each haptic feedback command after an expiration of the corresponding time delay upon receiving the indication.

2. The method of claim 1, wherein the determination of timing of each haptic feedback command is based on a location of the haptic feedback command within the message.

3. The method of claim 2, wherein the location is based on a word count of the text.

4. The method of claim 2, wherein the location is based on a character count of the text.

5. The method of claim 2, wherein the location is based on a reading speed of a user of the device.

6. The method of claim 1, further comprising:

prior to accessing, generating the haptically enabled message by receiving a gesture from a user of the device and converting the gesture into at least one haptic feedback command in the haptically enabled message.

7. The method of claim 6, wherein the gesture is a movement of the device by the user.

8. The method of claim 6, wherein the gesture is a video of the user.

9. The method of claim 1, wherein at least one haptic feedback command of the plurality of haptic feedback commands is an emoticon.

10. The method of claim 1, wherein at least one haptic feedback command of the plurality of haptic feedback commands is not visible in the message.

11. The method of claim 1, further comprising:

generating a haptic effect separate from the plurality of haptic effects in conjunction with a receipt of the haptically enabled message at the device.

12. The method of claim 11, wherein said haptic effect separate from the plurality of haptic effects may indicate at least one of message content, sender and priority.

29. The method of claim 1, wherein at least one of the haptic effects is a function of the inferred context of the text of the message.

30. The method of claim 5, further comprising:

presenting the user of the device with one or more test message to read; and

estimating the reading speed of the user based on the user'"'"'s reading of the one or more test messages.

31. The method of claim 1, further comprising:

determining whether the text is being scrolled, wherein said determination provides a cue for initiating at least one of the haptic effects.

33. The method of claim 1, further comprising:

receiving a message, the message comprising text; and

inserting the plurality of haptic feedback commands in the message to produce the haptically enabled message;

wherein the receiving and the inserting are performed prior to the accessing of the haptically enabled message.

13. A mobile terminal device comprising:

a processor;

a memory coupled to the processor; and

an actuator coupled to the processor;

wherein the processor is configured to access a haptically enabled message, the message comprising text and a plurality of haptic feedback commands, wherein at least two haptic feedback commands of the plurality of haptic feedback commands occur in succession within the message, and wherein the processor is further configured to;

determine a timing of each haptic feedback command of the plurality of haptic feedback commands within the message;

assign a precedence order to the at least two haptic feedback commands;

receive an indication that the message is being displayed on said mobile terminal device; and

synchronize the initiation of a plurality of haptic effects corresponding to fewer than all of the plurality of haptic feedback commands based on the received indication, the determined timing associated with the plurality of haptic feedback commands, and the assigned precedence order,

wherein synchronizing the initiation of the plurality of haptic effects comprises;

determining a time delay for each haptic feedback command based on the timing; and

initiating a haptic effect corresponding to each haptic feedback command after an expiration of the corresponding time delay upon receiving the indication.

14. The mobile terminal device of claim 13, wherein the plurality of haptic effects are generated by the actuator.

15. The mobile terminal device of claim 13, wherein the determination of timing of each haptic feedback command is based on a location of the haptic feedback command within the message.

16. The mobile terminal device of claim 15, wherein the location is based on a word count of the text.

17. The mobile terminal device of claim 15, wherein the location is based on a character count of the text.

18. The mobile terminal device of claim 15, wherein the location is based on a reading speed of a user of the device.

19. The mobile terminal device of claim 13, wherein the processor is further configured to:

prior to accessing, generate the haptically enabled message by receiving a gesture from a user of the device and converting the gesture into at least one haptic feedback command in the haptically enabled message.

20. The mobile terminal device of claim 19, further comprising an accelerometer coupled to the processor, wherein the gesture is a movement of the device by the user.

21. The mobile terminal device of claim 19, wherein the gesture is a video of the user.

22. The mobile terminal device of claim 13, wherein at least one haptic feedback command of the plurality of haptic feedback commands is an emoticon.

23. The mobile terminal device of claim 13, wherein at least one haptic feedback command of the plurality of haptic feedback commands is not visible in the message.

34. The mobile terminal device of claim 13, wherein the processor is further configured to:

receive a message, the message comprising text; and

insert the plurality of haptic feedback commands in the message to produce the haptically enabled message;

wherein the processor is configured to receive the message and insert the plurality of haptic effects prior to accessing the haptically enabled message.

24. A non-transitory computer readable storage medium having instructions stored thereon that, when executed by a processor, cause the processor to:

determine a timing of each haptic feedback command within a haptically enabled message, the message comprising text and a plurality of haptic feedback commands, wherein at least two haptic feedback commands of the plurality of haptic feedback commands occur in succession within the message;

assign a precedence order to the at least two haptic feedback commands;

receive an indication that the message is being displayed on said mobile terminal device; and

synchronize the initiation of a plurality of haptic effects corresponding to fewer than all of the plurality of haptic feedback commands based on the received indication, the determined timing associated with the plurality of haptic feedback commands, and the assigned precedence order,

wherein the synchronizing comprises;

determining a time delay for each haptic feedback command based on the timing; and

initiating a haptic effect corresponding to each haptic feedback command after an expiration of the corresponding time delay upon receiving the indication.

25. The non-transitory computer readable storage medium of claim 24, wherein the determination of timing for each haptic feedback command is based on a location of the haptic feedback command within the message.

26. The non-transitory computer readable storage medium of claim 25, wherein the location is based on a word count of the text.

27. The non-transitory computer readable storage medium of claim 25, wherein the location is based on a character count of the text.

28. The non-transitory computer readable storage medium of claim 25, wherein the location is based on a reading speed of a user of the device.

32. The non-transitory computer readable storage medium of claim 24, having instructions that further configure the processor to, prior to determining the timing of each haptic feedback command within the haptically enabled message:

generate the haptically enabled message by receiving a gesture from a user of the device and converting the gesture into at least one haptic feedback command in the haptically enabled message.

35. The non-transitory computer readable medium of claim 24, having further instructions stored thereon that, when executed by the processor, cause the processor to:

receive a message, the message comprising text; and

insert the plurality of haptic feedback commands in the message to produce the haptically enabled message;

wherein the execution of the instructions by the processor cause the processor to receive the message and insert the plurality of haptic feedback commands prior to determining the timing of each haptic feedback command within the haptically enabled message.