Systems and methods for mapping message contents to virtual physical properties for vibrotactile messaging

• US 10,198,078 B2
• Filed: 01/29/2018
• Issued: 02/05/2019
• Est. Priority Date: 07/15/2008
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Birnbaum, David M., Ullrich, Christopher J., Grant, Danny
• Primary Examiner(s): Pham, Toan N
• Application Number:
  US15/882,229
• Publication Number:
  US 20180348870A1
• Time in Patent Office: 372 Days
• Field of Search: 3404071, 345173, 715702
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72457 : according to geographic location
• CPC Class Codes: H04M 19/047 : Vibrating means for incoming calls
• CPC Class Codes: H04M 2250/12 : including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04W 4/21 : for social networking applications

Abstract

Systems and methods for mapping message contents to virtual physical properties for vibrotactile messaging are disclosed. For example, one disclosed method includes the steps of displaying a first visual representation of a virtual message object having an associated virtual physical parameter at a first device, generating a haptic effect based on the virtual physical parameter at the first device, and sending a message to a second device, the message being configured to display a second visual representation of the virtual message object at the second device and generate the haptic effect based on the virtual physical parameter at the second device.

Claims

1. A method comprising:

generating, at a messaging device, a virtual message object based on one or more user inputs;

determining and mapping, by the messaging device, a virtual physical parameter to the virtual message object based on one or more properties of the virtual message object;

determining, by the messaging device, a behavior of a virtual message object in a virtual message environment based on the virtual physical parameter and a physical interaction with a messaging device;

determining and imparting, by the messaging device, a haptic effect to the virtual message object based on the determined behavior; and

transmitting, by the messaging device, the virtual message object to a remote device based on the behavior, the virtual message object comprising the virtual physical parameter and the haptic effect.

2. The method of claim 1, wherein determining the virtual physical parameter is further based on a user interaction with the messaging device.

3. The method of claim 2, wherein the user interaction comprises a gesture, and wherein determining the behavior comprises determining a collision behavior of the virtual message object based on the gesture.

4. The method of claim 3, wherein the gesture comprises dragging or throwing the virtual message object or flicking, jabbing, or moving the messaging device.

5. The method of claim 1, wherein the virtual physical parameter comprises a size or a mass, and wherein the size or the mass is based on a data size of the virtual message object.

6. The method of claim 5, wherein the data size is a file size of a message attachment, a length of a text message, an area or resolution of a picture attachment, a length of a song attachment, or a length of a video attachment.

7. A messaging device comprising:

a haptic output device;

a display;

a non-transitory computer-readable medium; and

a processor in communication with the non-transitory computer-readable medium, the processor configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to;

generate a first virtual message object based on one or more user inputs;

determine and map a first virtual physical parameter to the first virtual message object based on one or more properties of the first virtual message object;

determine a first behavior of the first virtual message object in a virtual message environment based on the first virtual physical parameter and a physical interaction with the messaging device;

determine and imparting a first haptic effect to the first virtual message object based on the determined first behavior;

transmit the first virtual message object to a remote device based on the first behavior, the first virtual message object comprising the first virtual physical parameter and the first haptic effect;

receive a second virtual message object from a second remote device, the second virtual message object comprising a second virtual physical parameter and a second haptic effect;

determine a second behavior of the second virtual message object based on the second virtual physical parameter and a virtual message environment;

display, using the display, the second behavior of the second virtual message object; and

output, using the haptic output device, the second haptic effect corresponding to the displayed second behavior of the second virtual message object.

8. The messaging device of claim 7, wherein the second remote device is the remote device.

9. The messaging device of claim 7, wherein determining the first virtual physical parameter is further based on a user interaction with the messaging device.

10. The messaging device of claim 9, wherein the user interaction comprises a gesture, and wherein determining the first behavior comprises determining a collision behavior of the first virtual message object based on the gesture.

11. The messaging device of claim 10, wherein the gesture comprises dragging or throwing the first virtual message object or flicking, jabbing, or moving the messaging device.

12. The messaging device of claim 7, wherein the first or second virtual physical parameter comprises a size or a mass, and wherein the size or the mass is based on a data size of the respective first or second virtual message object.

13. The messaging device of claim 12, wherein the data size is a file size of an attachment to the first or second virtual message object, a length of a text message of the first or second virtual message object, an area or resolution of a picture attachment to the first or second virtual message object to the first or second virtual message object, a length of a song attachment to the first or second virtual message object, or a length of a video attachment to the first or second virtual message object.

14. A method comprising:

receiving, by a messaging device, a virtual message object from a remote device, the virtual message object comprising a virtual physical parameter and a haptic effect;

determining a behavior of the virtual message object based on the virtual physical parameter and a virtual message environment;

displaying, by a display of the messaging device, the behavior of virtual message object; and

outputting, using a haptic output device of the messaging device, the haptic effect corresponding to the displayed behavior of the virtual message object.

15. The method of claim 14, wherein the virtual physical parameter comprises a size or a mass, and wherein the size or the mass is based on a data size of the virtual message object.

16. The method of claim 15, wherein the data size is a file size of a message attachment, a length of a text message, an area or resolution of a picture attachment, a length of a song attachment, or a length of a video attachment.

17. The method of claim 14, wherein displaying the behavior comprises displaying the virtual message object bouncing or rolling within the virtual message environment, and wherein outputting the haptic effect corresponds to a bounce or rolling.

18. The method of claim 14, wherein displaying the behavior comprises displaying the virtual message object colliding with another virtual message object within the virtual message environment, and wherein outputting the haptic effect corresponds to the colliding.

19. The method of claim 14, wherein displaying the behavior comprises displaying the virtual message object breaking within the virtual message environment, and wherein outputting the haptic effect corresponds to the breaking.

20. The method of claim 14, further comprising:

determining a sound effect based on an interaction of the virtual message object with the virtual message environment based on the behavior; and

outputting, using a speaker of the messaging device, the sound effect corresponding to the interaction.