Simulation of tangible user interface interactions and gestures using array of haptic cells

• US 9,983,676 B2
• Filed: 06/28/2016
• Issued: 05/29/2018
• Est. Priority Date: 04/26/2013
• Status: Expired due to Fees

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Modarres, Ali, Cruz-Hernandez, Juan Manuel, Grant, Danny A., Levesque, Vincent
• Primary Examiner(s): Chang, Kent
• Assistant Examiner(s): Brittingham, Nathan
• Application Number:
  US15/195,705
• Publication Number:
  US 20160306426A1
• Time in Patent Office: 700 Days
• Field of Search: None
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1641 the display being formed by a plurality of foldable display components G06F1/1647 takes precedence
• CPC Class Codes: G06F 1/1643 the display being associated to a digitizer, e.g. laptops that can be used as penpads details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615
• CPC Class Codes: G06F 1/1652 the display being flexible, e.g. mimicking a sheet of paper, or rollable
• CPC Class Codes: G06F 2203/04102 Flexible digitiser, i.e. constructional details for allowing the whole digitising part of a device to be flexed or rolled like a sheet of paper
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/045 using resistive elements, e.g. a single continuous surface or two parallel surfaces put in contact
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A user interface device includes a flexible layer comprising a touch surface configured to receive a touch by a user, a plurality of haptic cells covered by the flexible layer, each haptic cell comprising a haptic output device, a sensor configured to sense an amount and/or rate of deformation of the flexible layer when a user touches the touch surface, and a processor configured to receive an output signal from the sensor, generate a haptic control signal based on the output signal from the sensor, and output the haptic control signal to at least one haptic output device of the plurality of haptic cells to cause the haptic output device to deform an associated haptic cell in response to the sensed deformation of the flexible layer.

Claims

1. A user interface device comprising:

a plurality of haptic cells;

a deformable layer comprising a touch surface;

each haptic cell covered by the deformable layer and comprising a haptic output device;

each cell comprising a first sensor positioned to sense deformation of the deformable layer proximal to the haptic cell, the first sensor configured to output a first sensor signal in response to sensing the deformation, the first sensor signal having an electrical characteristic; and

a processor in electrical communication with the first sensor of each haptic cell, and for each haptic cell;

the processor programmed to generate a haptic control signal in response to receiving the first sensor signal and communicate the haptic control signal to the haptic cell, the haptic control signal having an electrical characteristic defined at least in part by the electrical characteristic of the first sensor signal, the electrical characteristic of the haptic control signal corresponding to a desired amount of resistance to display by the haptic output device against deformation of the deformable layer.

2. The user interface device of claim 1 wherein a value of the electrical characteristic of the first sensor signal of each haptic cell varies with an amount of the deformation.

3. The user interface device of claim 2 wherein, upon receiving the haptic control signal, the haptic output device in the plurality of haptic cells delivers a haptic effect selected from the group consisting essentially of:

a wave, a ripple, movement of a joystick, and movement of a slider.

4. The user interface device of claim 1 wherein the electrical characteristic of the haptic signal is selected from the group consisting of:

magnitude, frequency, and duration.

5. The user interface device of claim 1, wherein:

each haptic cell further comprises a second sensor positioned to sense deformation of the deformable layer proximal to the haptic cell, the second sensor configured to output a second sensor signal in response to sensing the deformation, the second sensor signal having an electrical characteristic; and

the processor is further programmed to generate the haptic control signal in response to the first sensor signal and the second sensor signal, the haptic control signal having an electrical characteristic defined at least in part on the characteristic of the first sensor signal and the characteristic of the sensor second signal.

6. The user interface device of claim 5 wherein the first sensor and the second sensor are selected from the group consisting essentially of a strain gauge, force sensitive resistor, multi-touch sensor, pressure sensor, and multi-touch pressure sensor.

7. The user interface device of claim 6 wherein one of the first sensor or the second sensor is a pressure sensor.

8. The user interface device of claim 1 wherein the processor is further programmed to generate the haptic control signal only upon the electrical characteristic of the first sensor signal having a value above a threshold value.

9. A method of controlling a user interface having a plurality of haptic cells, each cell having a first sensor, the method comprising:

for each cell proximal to a deformed portion of a deformable layer, outputting a first sensor signal from the first sensor in response to deformation of the deformable layer proximal to the first sensor, the first sensor signal having an electrical characteristic;

for each cell proximal to the deformed portion of the deformable layer, generating a haptic control signal in response to the first sensor signal, the haptic control signal having an electrical characteristic defined at least in part on the characteristic of the first sensor signal, the electrical characteristic of the haptic signal corresponding to a desired amount of resistance to display by the haptic output device against deformation of the deformable layer;

delivering the haptic control signal to the haptic output device; and

in response to receiving the haptic control device, the haptic output device resisting deformation of the deformable layer, the amount of resistance corresponding to the electrical characteristic of the first sensor signal.

10. The method of claim 9 wherein:

outputting the first sensor signal comprises outputting a plurality of first sensor signals, each first sensor signal being output from a different first sensor; and

generating the haptic control signal comprises generating the haptic control signal in response to the plurality of first sensor signals.

11. The method of claim 10 further comprising:

varying a value of the electrical characteristic of at least one of the first sensor signals in response to changes in deformation of the deformable layer.

12. The method of claim 11 further comprising:

varying a value of the electrical characteristic of the haptic control signal in response to varying of the electrical characteristic of the first sensor signals.

13. The method of claim 12 wherein the electrical characteristics of the first sensor signals and the haptic control signal is selected from the group consisting of magnitude, frequency, and duration.

14. The method of claim 10, wherein the haptic actuators deliver a haptic effect selected from the group consisting of:

a wave, a ripple, movement of a joystick, and movement of a slider.

15. The method of claim 9 further comprising:

outputting a second sensor signal from a second sensor in response to deformation of the deformable layer proximal to the first sensor, the second sensor signal having an electrical characteristic; and

generating the haptic control signal in response to the first sensor signal and the second sensor signal, the electrical characteristic of the haptic control signal defined at least in part on the characteristic of the first sensor signal and the characteristic of the second sensor signal.

16. The method of claim 15 wherein:

the first sensor signal is associated with an amount and/or rate of deformation of the deformable layer; and

the second sensor signal is associated with a pressure exerted against the deformable layer.

17. The method of claim 9 wherein generating the haptic control signal occurs only upon the electrical characteristic of the first sensor signal having a value above a threshold value.

18. A user interface device comprising:

a deformable layer comprising a touch surface;

a plurality of haptic cells covered by the deformable layer, at least one of the haptic cells comprising a haptic output device, a first sensor, and a second sensor;

each first sensor positioned to sense deformation of the deformable layer proximal to the haptic cell, the first sensor configured to output a first sensor signal in response to sensing the deformation, the first sensor signal being associated with an amount and/or rate of deformation of the deformable layer, the first sensor signal having an electrical characteristic, the electrical characteristic of the first sensor signal having a value that varies with the deformation;

each second sensor positioned to sense deformation of the deformable layer proximal to the haptic cell, the second sensor configured to output a second sensor signal in response to sensing the deformation, the second sensor signal associated with a pressure exerted against the deformable layer, the second pressure sensor having an electrical characteristic, the electrical characteristic of the second sensor signal having a value that varies with the deformation; and

a processor in electrical communication with the first sensor and the haptic cell, the processor programmed to generate a haptic control signal in response to receiving the first and second sensor signals and communicate the haptic control signal to the haptic cell, the haptic control signal having an electrical characteristic defined at least in part by the characteristics of the first and second sensor signals, the electrical characteristic of the haptic signal corresponding to a determined amount of resistance to display against deformation of the deformable layer.