Method and apparatus for haptic flex gesturing

• US 10,133,401 B2
• Filed: 07/26/2016
• Issued: 11/20/2018
• Est. Priority Date: 02/08/2012
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Heubel, Robert W.
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): Mathews, Crystal A
• Application Number:
  US15/219,824
• Publication Number:
  US 20160334924A1
• Time in Patent Office: 847 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: G06F 1/1652 the display being flexible, e.g. mimicking a sheet of paper, or rollable
• CPC Class Codes: G06F 2203/04102 Flexible digitiser, i.e. constructional details for allowing the whole digitising part of a device to be flexed or rolled like a sheet of paper
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

An electronic interactive device having a user interface with a flexible surface, a sensor configured to sense a flex gesture applied to the flexible surface, a haptic device configured to generate a haptic effect in response to the flex gesture, and a controller in signal communication with the sensor and the haptic device. The controller is configured to trigger the haptic device to provide haptic confirmation of the flex gesture and to trigger an application action associated with the flex gesture.

Claims

1. An electronic interactive device comprising:

a user interface comprising a flexible surface deformable by a flex gesture;

a first sensor configured to sense the flex gesture applied to the flexible surface;

a haptic device configured to generate a haptic effect in response to the flex gesture; and

a controller in signal communication with the first sensor and the haptic device, the controller configured, upon the first sensor detecting the flex gesture applied to the flexible surface, to trigger an action associated with the flex gesture, wherein the haptic effect is generated by the haptic device based on a rate of flexing of the flex gesture.

2. The electronic interactive device of claim 1, further comprising a second sensor configured to sense a touch gesture applied to the flexible surface, wherein the controller is configured to trigger the action associated with the flex gesture only in response to a determination that the touch gesture was applied to the flexible surface before the flex gesture was applied to the flexible surface, such that the touch gesture has to be applied to the flexible surface before the flex gesture is able to trigger the action, wherein the flexible surface is deformable by flex gestures in a plurality of respective directions, wherein a direction of the flex gesture is a first direction of the plurality of respective directions, and the first sensor is configured to sense the flex gesture as being applied to the flexible surface in at least the first direction of the plurality of directions.

3. The electronic interactive device of claim 2, wherein, upon the first sensor detecting the flex gesture being applied to the flexible surface in the first direction of the plurality of directions, the controller operates to trigger one of a plurality of actions associated with the detected flex gesture when the touch gesture was applied before the detected flex gesture.

4. The electronic interactive device of claim 3, further comprising:

an accelerometer configured to sense an accelerometer-based gesture, wherein the controller is configured to trigger one of the plurality of actions based on the accelerometer-based gesture and the flex gesture.

5. The electronic interactive device of claim 2, wherein the controller is configured to trigger the action associated with the flex gesture in response to determining that the touch gesture was applied to the flexible surface within a threshold amount of time before the flex gesture.

6. The electronic interactive device of claim 1, wherein the controller is configured to determine, upon the first sensor sensing the flex gesture applied to the flexible surface, whether the flex gesture sensed by the first sensor is valid, the flex gesture being valid when there is an action mapped in a storage device to the flex gesture, and wherein upon determining that the flex gesture is valid, the controller is configured to trigger the haptic device to provide the haptic effect as a haptic confirmation of the flex gesture, and wherein the action is an action in a virtual or simulated environment of an application, or an action to control a multimedia platform.

7. The electronic interactive device of claim 1, wherein the flex gesture comprises at least one of bending, twisting, and rolling of the flexible surface, and the haptic effect is further based on which one of the bending, twisting, and rolling of the flexible surface forms the flex gesture.

8. The electronic interactive device of claim 7, wherein the controller is configured to trigger the haptic device to provide the haptic effect as a first haptic effect in response to the flex gesture comprising bending of the flexible surface and to trigger the haptic device to provide the haptic effect as a second haptic effect in response to the flex gesture comprising twisting of the flexible surface, the first haptic effect being different than the second haptic effect.

9. The electronic interactive device of claim 1, wherein the flex gesture deforms only one corner of the flexible surface, and wherein the controller is configured to trigger the action upon the flex gesture sensor detecting the flex gesture that deforms only one corner of the flexible surface.

10. The electronic interactive device of claim 1, wherein the haptic device is one of a plurality of haptic devices of the electronic interactive device, and wherein the controller is configured to select a haptic device of the plurality of haptic devices to activate to generate the haptic effect, wherein selection of the haptic device is based on a direction of the flex gesture that is applied to the flexible surface.

11. The electronic interactive device of claim 1, wherein the controller is configured to generate the haptic effect with an intensity that is based on a direction of the flex gesture.

12. The electronic interactive device of claim 11, wherein the direction is one of a first direction and a second direction that is opposite to the first direction, wherein the controller associates the first direction with a first application action, and associates the second direction with a second application action that is opposite to the first application action.

13. The electronic interactive device of claim 1, wherein the haptic effect is generated further based on a duration of the flex gesture.

14. An electronic interactive device comprising:

a user interface comprising a flexible surface deformable by each flex gesture of a plurality of flex gestures,

a first sensor that is a strain gauge or a piezoelectric transducer configured to sense a degree of flexing of each flex gesture of the plurality of flex gestures,

a second sensor that is an accelerometer configured to sense a rate of flexing of each flex gesture of the plurality of flex gestures, and

a third sensor that is a gyroscope configured to sense a direction of each flex gesture of the plurality of flex gestures,

a haptic device configured to generate respective haptic effects in response to the plurality of flex gestures, and

a controller in signal communication with the first sensor, the second sensor, the third sensor, and the haptic device, wherein the controller is configured

to trigger an application action for each flex gesture of the plurality of flex gestures based on at least one of the direction of the flex gesture, the degree of flexing of the flex gesture, and the rate of flexing of the flex gesture, and

to cause the haptic device to generate a haptic effect for at least one flex gesture of the plurality of flex gestures based on the rate of flexing of the at least one flex gesture.

15. The electronic interactive device of claim 14, wherein the controller is configured to cause the haptic device to generate another haptic effect for another flex gesture of the plurality of flex gestures based on a duration of the flex gesture.

16. A haptic gesture recognition method comprising:

sensing a first flex gesture applied to a flexible surface of a user interface with a flex gesture sensor;

generating a first haptic effect on the user interface with at least one haptic device in response to said sensing the first flex gesture, wherein a frequency of the first haptic effect is based on a rate of flexing of the first flex gesture; and

triggering a first action in response to the first flex gesture with a controller.

17. The method of claim 16, further comprising:

sensing a second flex gesture applied to the flexible surface of the user interface with the flex gesture sensor;

generating a second haptic effect to the user interface with at least one haptic device in response to said sensing the second flex gesture, wherein a duration of the second haptic effect is based on a degree of flexing of the second flex gesture, wherein a degree of flexing of the first flex gesture is different than the degree of flexing of the second flex gesture; and

triggering a second action in response to the second flex gesture with the controller,

wherein said triggering the first action comprises outputting a signal associated with the first flex gesture to a computing device executing an application associated with the first action or presenting the first action on a display screen of the user interface, and wherein the first haptic effect and the second haptic effect are different in duration.

18. The method of claim 16, wherein said sensing the first flex gesture comprises sensing bending of the flexible surface, or twisting of the flexible surface.

19. The method of claim 16, further comprising:

sensing a swinging gesture with an accelerometer, wherein the first action is triggered in response to the swinging gesture and the first flex gesture.

20. The electronic interactive device of claim 17, wherein a type of the first haptic effect is based on an application action associated with the first flex gesture, wherein a type of the second haptic effect is based on an application action associated with the second flex gesture, wherein the application action associated with the first flex gesture is different than the application action associated with the second flex gesture, and wherein the type of the first haptic effect is different than the type of the second haptic effect.