Haptic feedback for opportunistic displays

• US 10,198,072 B2
• Filed: 06/08/2016
• Issued: 02/05/2019
• Est. Priority Date: 06/08/2016
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Levesque, Vincent
• Primary Examiner(s): Sharifi-Tafreshi, Koosha
• Application Number:
  US15/176,692
• Publication Number:
  US 20170357315A1
• Time in Patent Office: 972 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers

Abstract

One illustrative system disclosed herein includes a computing device in communication with a sensor configured to detect a display device near the sensor, the computing device, or within a field of view of a user of the computing device. The sensor can also be configured to transmit a signal associated with the display device to a processor in communication with the sensor. The processor is configured to determine an availability of the display device to display or output content (e.g., texts, images, sounds, videos, etc.) and a location of the display device based on the signal. The processor is also configured to determine a haptic effect based on the availability and the location of the display device and transmit a haptic signal associated with the haptic effect. The illustrative system also includes a haptic output device configured to receive the haptic signal and output the haptic effect.

Claims

1. A system comprising:

a computing device;

a sensor communicatively coupled to the computing device and configured to transmit a first signal and a second signal;

a processor communicatively coupled to the sensor for receiving, the first signal and the second signal, the processor configured to;

detect a display device proximate to the sensor based on the first signal and determine that the display device is within a field of view of a user of the computing device based on the second signal;

determine, while the field of view of the user is directed away from the computing device, an availability of the display device to output content and a location of the display device based on the first signal or the second signal;

determine a first haptic effect based at least in part on the availability and the location of the display device; and

transmit a first haptic signal associated with the first haptic effect; and

a haptic output device configured to receive the first haptic signal and output the first haptic effect.

2. The system of claim 1, wherein the processor is further configured to transmit data that includes the content to the display device and the content includes text, an image, a sound, or a video.

3. The system of claim 2, wherein the processor is further configured to determine:

a second haptic effect based at least in part on the content and transmit a second haptic signal to the haptic output device configured to output the second haptic effect.

4. The system of claim 1, wherein the first haptic effect comprises a horizontal haptic effect associated with a horizontal position of the display device relative to the user and a vertical haptic effect associated with a vertical position of the display device relative to the user.

5. The system of claim 4, wherein the haptic output device comprises a first haptic output device and a second haptic output device, the first haptic output device configured to output the horizontal haptic effect and the second haptic output device configured to output the vertical haptic effect.

6. The system of claim 1, wherein the processor is further configured to:

determine a size, shape, or type of the display device based on the first signal or the second signal;

determine a second haptic effect based at least in part on the size, shape, or type of the display device; and

transmit a second haptic signal to the haptic output device configured to output the second haptic effect.

7. The system of claim 1, wherein the processor is communicatively coupled to the display device and the processor is further configured to:

receive, from the display device, a display signal, wherein the display device comprises a display device processor configured to determine a direction of the field of view of the user based on data from a display device sensor configured to detect the field of view of the user; and

determine the availability and the location of the display device based on the display signal.

8. The system of claim 7, wherein the display device sensor comprises a camera configured to capture an image associated with an eye of the user and the display device processor is configured to determine the direction of the field of view of the user based on the image.

9. The system of claim 7, wherein the display device sensor is configured to monitor a movement of a muscle in an eye of the user and the display device processor is configured to determine the field of view of the user based on the monitored movement.

10. The system of claim 7, wherein the display device comprises a display haptic output device and the display device processor is configured to:

determine a display device haptic effect based at least in part on the direction of the field of view of the user; and

transmit a display device haptic signal associated with the display device haptic effect to the display haptic output device configured to output the display device haptic effect.

11. The system of claim 1, wherein the sensor is configured to transmit location signal associated with the user of the system or the sensor and receive data about the location and the availability of the display device based on the location signal.

12. A method comprising:

receiving, by a processor of a computing device from a sensor communicatively coupled to the computing device, a first sensor signal and a second sensor signal;

detecting, by the processor, a display device proximate to the sensor and within a field of view of a user of the computing device based on the first sensor signal and the second sensor signal;

determining, by the processor and while the field of view of the user is directed away from the computing device, an availability of the display device to output content to the user and a location of the display device based at least in part on the first sensor signal or the second sensor signal;

determining, by the processor, a first haptic effect based at least in part on the availability and the location of the display device;

transmitting, by the processor, a first haptic signal associated with the first haptic effect to a haptic output device; and

outputting, by the haptic output device, the first haptic effect.

13. The method of claim 12, wherein the first haptic effect comprises a horizontal haptic effect associated with a horizontal position of the display device relative to the user and a vertical haptic effect associated with a vertical position of the display device relative to the user.

14. The method of claim 12, further comprising:

transmitting, by the processor, data that includes the content to the display device and the content includes an image, a sound, a video, or text.

15. The method of claim 14, further comprising:

determining, by the processor, a second haptic effect based at least in part on the content;

transmitting, by the processor, a second haptic signal associated with the second haptic effect to the haptic output device; and

outputting, by the haptic output device, the second haptic effect.

16. The method of claim 12, further comprising:

determining, by the processor, a size, shape, or type of the display device based on the first sensor signal or the second sensor signal;

determining, by the processor, a second haptic effect based at least in part on the size, shape, or type of the display device;

transmitting, by the processor, a second haptic signal associated with the second haptic effect to the haptic output device; and

outputting, by the haptic output device, the second haptic effect.

17. The method of claim 12, wherein determining the availability of the display device to output content to the user and the location of the display device further comprises:

receiving, by the processor, a display signal from the display device, wherein the display device comprises a display device processor configured to determine a direction of the field of view of the user based on data from a display device sensor configured to detect the field of view of the user, wherein the processor is communicatively coupled to the display device; and

determining the availability and the location of the display device based at least in part on the display signal.

18. The method of claim 17, wherein the display device sensor comprises a camera configured to capture an image associated with an eye of the user and the display device processor is configured to determine the direction of the field of view of the user based on the image.

19. The method of claim 17, wherein the display device sensor is configured to monitor a movement of an eye of the user and the display device processor is configured to determine the field of view of the user based on the monitored movement.

20. The method of claim 17, wherein the display device comprises a display haptic output device and the method further comprises:

determining, by the display device processor, a display device haptic effect based at least in part on the direction of the field of view of the user; and

transmitting, by the display device processor, a display device haptic signal associated with the display device haptic effect to the display haptic output device configured to output the display device haptic effect.