Haptic sensations as a function of eye gaze

• US 10,220,317 B2
• Filed: 11/21/2017
• Issued: 03/05/2019
• Est. Priority Date: 03/11/2013
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Grant, Danny A., Levesque, Vincent
• Primary Examiner(s): McClellan, James S
• Assistant Examiner(s): Carter, Kevin
• Application Number:
  US15/818,952
• Publication Number:
  US 20180111050A1
• Time in Patent Office: 469 Days
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user

Abstract

Systems, electronic devices, and methods provide haptic sensations as a function of eye gaze. A system may include a detector configured to determine a direction of an eye gaze of a user of the system, a processor configured to generate signal representative of a haptic effect based on the direction of the eye gaze, and a haptic output device configured to receive the signal from the processor and output the haptic effect to the user. A method for providing a haptic effect to a user of a system may include determining a direction of an eye gaze of the user of the system, generating a haptic effect based on the direction of the eye gaze, and outputting the haptic effect to the user.

Claims

1. A system comprising:

a sensor;

a processor communicatively coupled to the sensor and configured to;

receive a sensor signal from the sensor;

determine a user eye gaze direction relative to an object on a display device associated with the system based on the sensor signal from the sensor;

determine a first parameter of a first haptic effect based on the user eye gaze direction relative to the object on the display device and a type of the object;

determine a second parameter of a second haptic effect based on the user eye gaze direction and the type of the object, wherein the second parameter is different from the first parameter, the first parameter or the second parameter comprising an amplitude, a frequency, or a duration of the haptic effect;

generate a first haptic output signal representative of the first haptic effect when the user eye gaze direction is determined to be directed at the object on the display device; and

generate a second haptic output signal representative of the second haptic effect when the user eye gaze direction is determined to be directed away from the object on the display device; and

a haptic output device configured to receive the first haptic output signal and the second haptic output signal from the processor and output the first haptic effect or the second haptic effect to the user based on the user eye gaze direction relative to the object on the display device.

2. The system according to claim 1, wherein the haptic output device comprises an actuator configured to output the first haptic effect or the second haptic effect.

3. The system according to claim 1, wherein the haptic output device comprises a non-mechanical device configured to output the first haptic effect or the second haptic effect.

4. The system according to claim 1, wherein the sensor comprises a camera configured to capture an image and an image processor configured to determine the user eye gaze direction based on the image.

5. The system according to claim 1, wherein the sensor is operable for sensing movements of muscles near an eye of the user, and a second processor is configured to determine the user eye gaze direction based on the movements of muscles near the eye of the user.

6. The system according to claim 1, wherein the sensor and the processor are part of a first device and the haptic output device is part of a second device that is separate from the first device.

7. The system according to claim 6, further comprising a communication port configured to establish a wired or wireless communication channel between the first device and the second device.

8. The system according to claim 1, wherein the sensor is part of a first device and the processor and the haptic output device are part of a second device that is separate from the first device.

9. The system according to claim 8, further comprising a communication port configured to establish a wired or wireless communication channel between the first device and the second device.

10. The system according to claim 1, wherein the processor is further configured to generate an augmented reality space on the display device, and use the user eye gaze direction to determine when the object the user is looking at is an object in the augmented reality space.

11. The system according to claim 1, wherein the display device includes a touch screen configured to receive input from the user.

12. The system according to claim 1, wherein the system is a gaming system.

13. The system according to claim 1, wherein the system is part of a vehicle.

14. A method for providing a haptic effect to a user of a system, the method comprising:

receiving, by a processor, a sensor signal from a sensor;

determining, by the processor, a user eye gaze direction relative to an object on a display device based on the sensor signal from the sensor;

determining, by the processor, a first parameter of a first haptic effect based on the user eye gaze direction relative to the object on the display device and a type of the object;

determining, by the processor, a second parameter of a second haptic effect based on the user eye gaze direction and the type of the object, wherein the second parameter is different from the first parameter, the first or second parameter comprising an amplitude, a frequency, or a duration of the haptic effect;

generating, by the processor, a first haptic output signal representative of the first haptic effect when the user eye gaze direction is determined to be directed at the object on the display device;

generating, by the processor, a second haptic output signal representative of the second haptic effect when the user eye gaze direction is determined to be directed away from the object on the display device;

transmitting, by the processor, the first haptic signal when the user eye gaze direction is determined to be directed at the object on the display device;

transmitting, by the processor, the second haptic signal when the user eye gaze direction is determined to be directed away from the object on the display device; and

outputting, by a haptic output device, the first haptic effect or the second haptic effect to the user based on the user eye gaze direction relative to the object on the display device.

15. The method according to claim 14, wherein said determining the user eye gaze direction comprises:

capturing, by the sensor, an image; and

determining, by the processor, the user eye gaze direction based on the image.

16. The method according to claim 14, wherein said determining the user eye gaze direction comprises:

monitoring, by the sensor, movements of muscles near an eye of the user; and

determining the user eye gaze direction based on the movements of muscles near the eye of the user with the processor.

17. The method according to claim 14, further comprising:

generating, by the processor, an augmented reality space that includes the object;

displaying the augmented reality space on the display device; and

generating the first haptic output signal when the user eye gaze direction is determined to be directed at the object in the augmented reality space.

18. A system comprising:

a processing device; and

a non-transitory computer-readable medium communicatively coupled to the processing device comprising processor-executable instructions, wherein the processing device is configured to execute the processor-executable instructions stored in the non-transitory computer-readable medium to;

receive, from a sensor, a sensor signal;

determine a user eye gaze direction relative to an object on a display device based on the sensor signal from the sensor;

determine a first parameter of a first haptic effect based on the user eye gaze direction relative to the object on the display device and a type of the object;

determine a second parameter of a second haptic effect based on the user eye gaze direction and the type of the object, wherein the second parameter is different from the first parameter, the first or second parameter comprising an amplitude, a frequency, or a duration of the haptic effect;

generate a first haptic output signal representative of the first haptic effect when the user eye gaze direction is determined to be directed at the object on the display device;

generate a second haptic output signal representative of the second haptic effect when the user eye gaze direction is determined to be directed away from the object on the display device; and

output the first haptic output signal or the second haptic output to signal to a haptic output device configured to receive the first haptic output signal and the second haptic output signal from the processing device and output the first haptic effect or the second haptic effect to the user based on the user eye gaze direction relative to the object on the display device.

19. The system according to claim 18, wherein the sensor comprises a camera configured to capture an image and wherein the processing device is further configured to execute the processor-executable instructions stored in the non-transitory computer-readable medium to determine the user eye gaze direction based on the image.

20. The system according to claim 18, wherein the sensor is operable for sensing movements of muscles near an eye of the user and wherein the processing device is configured to execute the processor-executable instructions stored in the non-transitory computer-readable medium to determine the user eye gaze direction based on the movements of 0 muscles near the eye of the user.