Force feedback computer input and output device with coordinated haptic elements

• US 6,337,678 B1
• Filed: 07/21/1999
• Issued: 01/08/2002
• Est. Priority Date: 07/21/1999
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Tactiva Incorporated
• Inventors: Fish, Daniel E.
• Primary Examiner(s): Jankus, Almis R.
• Assistant Examiner(s): AWAD, AMR A
• Application Number:
  US09/357,727
• Time in Patent Office: 902 Days
• Field of Search: 345/161, 345/156, 345/157, 345/158, 345/173-179, 345/163
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user

Abstract

A set of haptic elements (haptels) are arranged in a grid. Each haptel is a haptic feedback device with linear motion and a touchable surface substantially perpendicular to the direction of motion. In a preferred embodiment, each haptel has a position sensor which measures the vertical position of the surface within its range of travel, a linear actuator which provides a controllable vertical bi-directional feedback force, and a touch location sensor on the touchable surface. All haptels have their sensors and effectors interfaced to a control processor. The touch location sensor readings are processed and sent to a computer, which returns the type of haptic response to use for each touch in progress. The control processor reads the position sensors, derives velocity, acceleration, net force and applied force measurements, and computes the desired force response for each haptel. The haptels are coordinated such that force feedback for a single touch is distributed across all haptels involved. This enables the feel of the haptic response to be independent of where touch is located and how many haptels are involved in the touch. As a touch moves across the device, haptels are added and removed from the coordination set such that the user experiences an uninterrupted haptic effect. Because the touch surface is comprised of a multiple haptels, the device can provide multiple simultaneous interactions, limited only by the size of the surface and the number of haptels. The size of the haptels determines the minimum distance between independent touches on the surface, but otherwise does not affect the properties of the device. Thus, the device is a pointing device for graphical user interfaces which provides dynamic haptic feedback under application control for multiple simultaneous interactions.

Claims

1. An input/output device comprising:

a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

each haptic element of said plurality of haptic elements is configured to provide a haptic effect at the contact surface of said each haptic element upon said contact surface of said each haptic element being touched;

a plurality of sensors, each one of said plurality of sensors coupled to the contact surface of a corresponding one of said plurality of haptic elements, wherein said sensor is a one of said plurality of sensors, wherein each one of said sensor is configured to generate information in response to said contact surface being touched and each of said plurality of haptic elements is configured to produce said haptic effect based on said information; and

a processor, coupled to said plurality of sensors and said plurality of haptic elements, wherein

said processor is configured to control a haptic effect produced by certain ones of said plurality of haptic elements based on information received from ones of said plurality of sensors corresponding to said certain ones of said plurality of haptic elements,

said certain ones of said plurality of haptic elements are ones of said plurality of haptic elements being touched, and

said processor is configured to cause contiguous haptic elements of said certain ones of said plurality of haptic elements to collectively produce a haptic effect by virtue of being configured to calculate a weighted average of a force experienced by each one of said contiguous haptic elements.

2. An input/output device comprising:

a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements; and

a sensor, wherein

said sensor is coupled to said contact surface of said at least one of said plurality of haptic elements,

said sensor is configured to generate information in response to said contact surface being touched,

said at least one of said plurality of haptic elements is configured to produce said haptic effect based on said information,

certain ones of said plurality of haptic elements are configured to produced a haptic response at a respective contact surface of said certain ones of said plurality of haptic elements in response to said respective contact surface of said certain ones of said plurality of haptic elements being touched,

said certain ones of said plurality of haptic elements produce a given haptic effect in a cooperative manner by summing forces on ones of said certain ones of said plurality of haptic elements being touched, taking a weighted average of positions of said ones of said certain ones of said plurality of haptic elements being touched, and combining this information in order to determine a desired feedback force for each one of said ones of said certain ones of said plurality of haptic elements being touched.

3. An input/output device comprising:

a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface,

each haptic element of said plurality of haptic elements is configured to provide a haptic effect at a contact surface of said haptic element upon said contact surface of said haptic element being touched;

a plurality of sensors, each one of said plurality of sensors coupled to a contact

surface of a corresponding one of said plurality of haptic elements; and

a processor, coupled to said plurality of sensors and said plurality of haptic elements, wherein

each one of said plurality of sensors is configured to generate information upon said contact surface of said corresponding one of said plurality of haptic elements being touched,

said processor is configured to control a haptic effect provided by certain ones of said plurality of haptic elements based on information received from ones of said plurality of sensors corresponding to said certain ones of said plurality of haptic elements,

said certain ones of said plurality of haptic elements are ones of said plurality of haptic elements being touched, and

said processor is configured to cause said contiguous haptic elements to collectively produce a haptic effect by virtue of being configured to calculate a weighted average of a force experienced by each one of said contiguous haptic elements.

4. An input/output device comprising:

a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface,

certain ones of said plurality of haptic elements are configured to produce a haptic effect at a respective contact surface of said certain ones of said plurality of haptic elements in response to said respective contact surface of said certain ones of said plurality of haptic elements being touched, and

said certain ones of said plurality of haptic elements provide said haptic effect in a cooperative manner by

summing forces on ones of said certain ones of said plurality of haptic elements being touched,

taking a weighted average of positions of said ones of said certain ones of said plurality of haptic elements being touched, and

combining this information in order to determine a desired feedback force for each one of said ones of said certain ones of said plurality of haptic elements being touched.

5. An input/output device comprising:

a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements;

a sensor, wherein

said sensor is coupled to said contact surface of said at least one of said plurality of haptic elements,

said sensor is configured to generate information in response to said contact surface being touched, and

said at least one of said plurality of haptic elements is configured to produce said haptic effect based on said information; and

a processor, coupled to said sensor and said at least one of said plurality of haptic elements, wherein

said processor is configured to control said haptic effect produced by said at least one of said plurality of haptic elements based on information received from said sensor, and

said sensor is configured to act as said contact surface.

6. An input/output device comprising:

a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements;

a sensor, wherein

said sensor is coupled to said contact surface of said at least one of said plurality of haptic elements,

said sensor is configured to generate information in response to said contact surface being touched, and

said at least one of said plurality of haptic elements is configured to produce said haptic effect based on said information; and

a processor, coupled to said sensor and said at least one of said plurality of haptic elements, wherein

said processor is configured to control said haptic effect produced by said at least one of said plurality of haptic elements based on information received from said sensor, and

said sensor is a touch sensor.

7. An input/output device comprising:

a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements;

a sensor, wherein

said sensor is coupled to said contact surface of said at least one of said plurality of haptic elements,

said sensor is configured to generate information in response to said contact surface being touched, and

said at least one of said plurality of haptic elements is configured to produce said haptic effect based on said information; and

a processor, coupled to said sensor and said at least one of said plurality of haptic elements, wherein

said processor is configured to control said haptic effect produced by said at least one of said plurality of haptic elements based on information received from said sensor, and

said sensor is a position sensor.

8. An input/output device comprising:

a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements;

a sensor, wherein

said sensor is coupled to said contact surface of said at least one of said plurality of haptic elements,

said sensor is configured to generate information in response to said contact surface being touched, and

said at least one of said plurality of haptic elements is configured to produce said haptic effect based on said information; and

a processor, coupled to said sensor and said at least one of said plurality of haptic elements, wherein

said processor is configured to control said haptic effect produced by said at least one of said plurality of haptic elements based on information received from said sensor, and

said sensor is a motion sensor.

9. A computer system comprising:

a first processor;

a display, coupled to said first processor;

an input/output device, coupled to said first processor and comprising a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements; and

a sensor, wherein

said sensor is coupled to said contact surface of said at least one of said plurality of haptic elements,

said sensor is configured to generate information in response to said contact surface being touched,

said at least one of said plurality of haptic elements is configured to produce said haptic effect based on said information, and

said sensor is configured to act as said contact surface.

10. A computer system comprising:

a first processor;

a display, coupled to said first processor;

an input/output device, coupled to said first processor and comprising a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements;

a sensor, wherein

said sensor is coupled to said contact surface of said at least one of said plurality of haptic elements,

said sensor is configured to generate information in response to said contact surface being touched,

said at least one of said plurality of haptic elements is configured to produce said haptic effect based on said information; and

a second processor, coupled to said sensor and said at least one of said plurality of haptic elements, wherein

said second processor is configured to control said haptic effect produced by said at least one of said plurality of haptic elements based on information received from said sensor, and

said sensor is a touch sensor.

11. A computer system comprising:

a first processor;

a display, coupled to said first processor;

an input/output device, coupled to said first processor and comprising a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements;

a sensor, wherein

said sensor is coupled to said contact surface of said at least one of said plurality of haptic elements,

said sensor is configured to generate information in response to said contact surface being touched,

said at least one of said plurality of haptic elements is configured to produce said haptic effect based on said information; and

a second processor, coupled to said sensor and said at least one of said plurality of haptic elements, wherein

said second processor is configured to control said haptic effect produced by said at least one of said plurality of haptic elements based on information received from said sensor, and

said sensor is a position sensor.

12. A computer system comprising:

a first processor;

a display, coupled to said first processor;

an input/output device, coupled to said first processor and comprising a plurality of haptic elements, wherein

each one of said plurality of haptic elements comprises a contact surface,

said contact surfaces define a surface, and

at least one of said plurality of haptic elements is configured to produce a haptic effect at a contact surface of said at least one of said plurality of haptic elements;

a sensor, wherein

said sensor is coupled to said contact surface of said at least one of said plurality of haptic elements,

said sensor is configured to generate information in response to said contact surface being touched,

said at least one of said plurality of haptic elements is configured to produce said haptic effect based on said information; and

a second processor, coupled to said sensor and said at least one of said plurality of haptic elements, wherein

said second processor is configured to control said haptic effect produced by said at least one of said plurality of haptic elements based on information received from said sensor, and

said sensor is a motion sensor.