Force feedback computer input and output device with coordinated haptic elements

• US 20020044132A1
• Filed: 11/01/2001
• Published: 04/18/2002
• Est. Priority Date: 07/21/1999
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Tactiva Incorporated
• Inventors: Fish, Daniel E.
• Granted Patent: US 6,819,312 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user

Abstract

A set of haptic elements (haptels) are arranged in a grid. Each haptel is a haptic feedback device with linear motion and a touchable surface substantially perpendicular to the direction of motion. In a preferred embodiment, each haptel has a position sensor which measures the vertical position of the surface within its range of travel, a linear actuator which provides a controllable vertical bi-directional feedback force, and a touch location sensor on the touchable surface. All haptels have their sensors and effectors interfaced to a control processor. The touch location sensor readings are processed and sent to a computer, which returns the type of haptic response to use for each touch in progress. The control processor reads the position sensors, derives velocity, acceleration, net force and applied force measurements, and computes the desired force response for each haptel. The haptels are coordinated such that force feedback for a single touch is distributed across all haptels involved. This enables the feel of the haptic response to be independent of where touch is located and how many haptels are involved in the touch. As a touch moves across the device, haptels are added and removed from the coordination set such that the user experiences an uninterrupted haptic effect. Because the touch surface is comprised of a multiple haptels, the device can provide multiple simultaneous interactions, limited only by the size of the surface and the number of haptels. The size of the haptels determines the minimum distance between independent touches on the surface, but otherwise does not affect the properties of the device. Thus, the device is a pointing device for graphical user interfaces which provides dynamic haptic feedback under application control for multiple simultaneous interactions.

Claims

1. An input/output device comprising:

a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces of said plurality of haptic elements define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements in response to said contact surface being touched.

2. The input/output device of claim 1, further comprising:

a sensor coupled to said contact surface of said at least one of said plurality of haptic elements;

a processor, coupled to said sensor and said at least one of said plurality of haptic elements, wherein

said sensor is configured to generate information in response to said contact surface being touched, and

said processor is configured to control said haptic effect provided by said at least one of said plurality of haptic elements based on information received from said sensor;

3. The input/output device of claim 2, wherein said sensor is further configured to generate information based upon a proximity of a pointing device to said contact surface.

4. The input/output device of claim 2, wherein said sensor is a resistive film sensor.

5. The input/output device of claim 2, wherein said sensor is a capacitive proximity sensor.

6. The input/output device of claim 1, wherein each haptic element of said plurality of haptic elements is configured to provide a haptic effect at a contact surface of said haptic element upon said contact surface of said haptic element being touched, and further comprising:

a plurality of sensors, each one of said plurality of sensors coupled to a contact surface of a corresponding one of said plurality of haptic elements, wherein said at least one of said plurality of haptic elements said plurality of sensors; and

a processor, coupled to said plurality of sensors and said plurality of haptic elements, wherein

each one of said plurality of sensors is configured to generate information upon said contact surface of said corresponding one of said plurality of haptic elements being touched,

said processor is configured to control a haptic effect provided by certain ones of said plurality of haptic elements based on information received from ones of said plurality of sensors corresponding to said certain ones of said plurality of haptic elements, and

said certain ones of said plurality of haptic elements are ones of said plurality of haptic elements being touched.

7. The input/output device of claim 6, wherein said processor is further configured to cause contiguous haptic elements of said certain ones of said plurality of haptic elements to collectively produce a haptic effect.

8. The input/output device of claim 6, wherein said processor is configured to cause said contiguous haptic elements to collectively produce a haptic effect by virtue of being configured to calculate a weighted average of a force experienced by each one of said contiguous haptic elements.

9. The input/output device of claim 6, wherein new haptic elements of said plurality of haptic elements are added to a set of haptic elements in response to said new haptic elements also being touched and old haptic elements in said set of haptic elements are removed from said set of haptic elements in response to said old haptic elements no longer being touched, wherein said set of haptic elements represents said contiguous haptic elements.

10. The input/output device of claim 1, further comprising:

a display, coupled to said plurality of haptic elements, wherein

said each one of said plurality of haptic elements correspond to one of a plurality of areas displayed on said display, and

said haptic effect provided by said at least one of said plurality of haptic elements is related to a one of said plurality of areas displayed on said display corresponding to said at least one of said plurality of haptic elements.

11. The input/output device of claim 1, wherein

said at least one of said plurality of haptic elements corresponds to one of a plurality of areas displayed on a display;

and

said haptic effect provided by said at least one of said plurality of haptic elements is related to information displayed in said one of said plurality of areas.

12. The input/output device of claim 1, wherein certain ones of said plurality of haptic elements are configured to provide a haptic response at a respective contact surface of said certain ones of said plurality of haptic elements in response to said respective contact surface of said certain ones of said plurality of haptic elements being touched.

13. The input/output device of claim 12, wherein each one of said certain ones of said plurality of haptic elements is configured to provide concurrent input and output when touched concurrently.

14. The input/output device of claim 12, wherein said certain ones of said plurality of haptic elements are configured to provide input from and output to concurrent touches in a plurality of locations.

15. The input/output device of claim 12, wherein each one of said certain ones of said plurality of haptic elements further comprise an actuator coupled to a corresponding contact surface and configured to provide force feedback to said corresponding contact surface.

16. The input/output device of claim 15, wherein said actuator comprises a linear electromagnetic actuator, wherein said contact surface is coupled to said linear electromagnetic actuator.

17. The input/output device of claim 16, wherein said actuator further comprises a spring, wherein said spring is coupled to said contact surface to maintain said actuator in an extended position when said contact surface is not being touched.

18. The input/output device of claim 15, wherein said each one of said plurality of haptic elements further comprises a force sensor coupled between said contact surface and said actuator.

19. The input/output device of claim 18, wherein said force sensor is a piezoelectric force sensor.

20. The input/output device of claim 12, wherein said certain ones of said plurality of haptic elements are further configured to provide a given haptic effect in a cooperative manner.

21. The input/output device of claim 12, wherein said certain ones of said plurality of haptic elements provide a given haptic effect in a cooperative manner by summing forces on ones of said certain ones of said plurality of haptic elements being touched, taking a weighted average of positions of said ones of said certain ones of said plurality of haptic elements being touched, and combining this information in order to determine a desired feedback force for each one of said ones of said certain ones of said plurality of haptic elements being touched.

22. A computer system comprising:

a processor;

a display, coupled to said processor; and

an input/output device, coupled to said processor and comprising a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements in response to said contact surface being touched.

23. The computer system of claim 22, further comprising:

a sensor coupled to said contact surface of said at least one of said plurality of haptic elements;

a processor, coupled to said sensor and said at least one of said plurality of haptic elements, wherein

said sensor is configured to generate information upon said contact surface being touched, and

said processor is configured to control said haptic effect provided by said at least one of said plurality of haptic elements based on information received from said sensor;

24. The computer system of claim 22, further comprising:

a display, coupled to said plurality of haptic elements, wherein

said each one of said plurality of haptic elements correspond to one of a plurality of areas displayed on said display, and

said haptic effect provided by said at least one of said plurality of haptic elements is related to a one of said plurality of areas displayed on said display corresponding to said at least one of said plurality of haptic elements.

25. The computer system of claim 22, wherein each haptic element of said plurality of haptic elements is configured to provide a haptic effect at a contact surface of said haptic element upon said contact surface of said haptic element being touched.

26. The computer system of claim 25, wherein multiple ones of said plurality of haptic elements can provide input and output when touched simultaneously.

27. The computer system of claim 25, wherein said plurality of haptic elements can provide input from concurrent touches in a plurality of locations.

28. The computer system of claim 25, wherein said each haptic element is further configured to provide said haptic effect in a cooperative manner.

29. The computer system of claim 25, further comprising:

a display; and

a processor, coupled to said display and to said plurality of haptic elements, wherein

said each one of said plurality of haptic elements corresponds to a corresponding one of a plurality of areas displayed on said display, and

said haptic effect provided by said each one of said plurality of haptic elements is related to said corresponding one of said plurality of areas displayed on said display.

30. A method of communicating information to and from a computer comprising:

reading movement information from a motion sensor of at least one of a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

said movement information represents motion of a contact surface of said at least one of a plurality of haptic elements; and

causing said at least one of said plurality of haptic elements to produce a haptic effect in response to said motion.

31. The method of claim 30, further comprising:

displaying information in a plurality of areas displayed on a display, wherein

each one of said plurality of haptic elements corresponds to one of said plurality of areas.

32. The method of claim 31, wherein said each one of said plurality of haptic elements further comprises an actuator coupled to said contact surface of said each one of said plurality of haptic elements, the method further comprising:

applying force to said contact surface of said each one of said plurality of haptic elements, using said actuator of said each one of said plurality of haptic elements, based on information displayed in a corresponding one of said plurality of areas displayed on said display.

33. The method of claim 30, further comprising:

sensing a touch on said contact surface of said at least one of said plurality of haptic elements using a sensor coupled to said contact surface of said at least one of said plurality of haptic elements;

generating information based on said contact surface being touched; and

controlling said haptic effect provided by said at least one of said plurality of haptic elements based on information received from said sensor.

34. The method of claim 30, wherein each haptic element of said plurality of haptic elements is configured to provide a haptic effect at a contact surface of said haptic element in response to said contact surface of said haptic element being touched and each contact surface has a sensor coupled thereto, the method further comprising:

sensing a touch on a contact surface of certain ones of said plurality of haptic elements using corresponding ones of said sensors;

generating information corresponding to said certain ones of said plurality of haptic elements based on said touch; and

controlling said haptic effect provided by said certain ones of said plurality of haptic elements based on said information.

35. The method of claim 34, further comprising:

causing said certain ones of said plurality of haptic elements to provide a coordinated haptic effect regardless of a distribution of force of said touch among said certain ones of said plurality of haptic elements.