Systems and methods for providing dynamic haptic playback for an augmented or virtual reality environments

• US 10,572,017 B2
• Filed: 04/20/2018
• Issued: 02/25/2020
• Est. Priority Date: 04/20/2018
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Wu, Liwen, Grant, Danny A., Cruz-Hernandez, Juan Manuel
• Primary Examiner(s): Sherwin, Ryan W
• Application Number:
  US15/958,987
• Publication Number:
  US 20190324539A1
• Time in Patent Office: 676 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

One illustrative system disclosed herein includes a computing device that comprises a memory and a processor in communication with the memory. The system also includes an xPC target machine that is capable of achieving sampling rates of at least 100 khz and in communication with the computing device and a user device that includes a sensor and a haptic output device. The processor generates a simulate reality environment and determines a haptic effect based on the simulated reality environment or a sensor signal from the sensor. The processor transmits data about a parameter of the haptic effect or the sensor signal to the xPC target machine, which determines the parameter of the haptic effect and generates, in substantially real time, a haptic signal. The xPC target machine transmits the haptic signal to the haptic output device, which is configured to receive the haptic signal and output the haptic effect.

Claims

1. A computing device comprising:

a processor and a non-transitory computer-readable medium communicatively coupled to the processor, wherein the processor is configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to;

generate a simulated reality environment;

obtain sensor data indicating an interaction associated with the simulated reality environment, the sensor data obtained at a sampling rate;

determine, based on the interaction, whether to output a static haptic effect or a dynamic haptic effect;

responsive to a determination to output a dynamic haptic effect, iteratively during the interaction substantially in real time and substantially at the sampling rate;

determine, using the obtained sensor data, one or more parameters of the dynamic haptic effect;

generate one or more haptic signals associated with the dynamic haptic effect based on the one or more parameters; and

transmit the one or more haptic signals associated with the dynamic haptic effect to a haptic output device to output the dynamic haptic effect in substantially real time.

2. The computing device of claim 1, wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

receive, from a sensor configured to capture information about a user, a sensor signal indicating information about the user; and

determine whether to output the dynamic haptic effect based at least in part on the sensor signal, wherein the one or more haptic signals associated with the dynamic haptic effect are based on the dynamic haptic effect.

3. The computing device of claim 2, wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

determine, based on the one or more parameters of the dynamic haptic effect, that the dynamic haptic effect is a periodic dynamic haptic effect, wherein the periodic dynamic haptic effect is based on a periodic waveform, and the one or more haptic signals associated with the dynamic haptic effect are based on the periodic waveform.

4. The computing device of claim 2, wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

determine, based on the one or more parameters of the dynamic haptic effect, that the dynamic haptic effect is an aperiodic dynamic haptic effect, wherein the aperiodic dynamic haptic effect is based on an aperiodic waveform, and the one or more haptic signals associated with the dynamic haptic effect are based on the aperiodic waveform.

5. The computing device of claim 2, wherein the information about a user comprises biological or biometric information about the user.

6. The computing device of claim 1, wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

determine, based on the interaction, to output a static haptic effect, wherein the static haptic effect is a predetermined haptic effect, and one or more haptic signals are associated with the static haptic effect.

7. The computing device of claim 6, wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

determine one or more parameters of the static haptic effect, wherein the static haptic effect is a periodic static haptic effect, the static haptic effect is a periodic static haptic effect, the periodic static haptic effect is based on a periodic waveform, and the one or more haptic signals associated with the static haptic effect are based on the periodic waveform.

8. The computing device of claim 6, wherein the processor is further configured to execute processor executable instructions stored in the non-transitory computer-readable medium to:

determine one or more parameters of the static haptic effect, wherein the static haptic effect is an aperiodic static haptic effect, the aperiodic static haptic effect is based on an aperiodic waveform, and the one or more haptic signals associated with the static haptic effect are based on the aperiodic waveform.

9. A method comprising:

generating, by a processor, a simulated reality environment;

obtaining, by the processor, sensor data indicating an interaction associated with the simulated reality environment, the sensor data obtained at a sampling rate;

determining, by the processor and based on the interaction, whether to output a static haptic effect or a dynamic haptic effect;

responsive to determining to output a dynamic haptic effect, iteratively during the interaction substantially in real time and substantially at the sampling rate;

determining, by the processor and using the obtained sensor data, based on the one or more parameters of the dynamic haptic effect;

generating one or more haptic signals associated with the dynamic haptic effect; and

transmitting, by the processor, the one or more haptic signals associated with the dynamic haptic effect to a haptic output device to output the dynamic haptic effect in substantially real time.

10. The method of claim 9, further comprising:

receiving, by the processor, a sensor signal indicating information about a user; and

determining, by the processor, whether to output the dynamic haptic effect based at least in part on the sensor signal, wherein the one or more haptic signals associated with the dynamic haptic effect are based on the dynamic haptic effect.

11. The method of claim 10, further comprising:

determining, by the processor and based on the one or more parameters of the dynamic haptic effect, that the dynamic haptic effect is a periodic dynamic haptic effect, wherein the periodic dynamic haptic effect is based on a periodic waveform, and the one or more haptic signals associated with the dynamic haptic effect are based on the periodic waveform.

12. The method of claim 10, further comprising:

determining, by the processor and based on the one or more parameters of the dynamic haptic effect, that the dynamic haptic effect is an aperiodic dynamic haptic effect, wherein the aperiodic dynamic haptic effect is based on an aperiodic waveform, and the one or more haptic signals associated with the dynamic haptic effect are based on the aperiodic waveform.

13. The method of claim 9, further comprising:

determining, by the processor and based on the interaction, to output a static haptic effect, wherein the static haptic effect is a predetermined haptic effect, and one or more haptic signals are associated with the static haptic effect.

14. The method of claim 13, further comprising:

determining, by the processor, one or more parameters of the static haptic effect, wherein the static haptic effect is a periodic static haptic effect, the periodic static haptic effect is based on a periodic waveform, and the one or more haptic signals associated with the static haptic effect are based on the periodic waveform.

15. The method of claim 13, further comprising:

determining, by the processor, one or more parameters of the static haptic effect, wherein the static haptic effect is an aperiodic static haptic effect, the aperiodic static haptic effect is based on an aperiodic waveform, and the one or more haptic signals associated with the static haptic effect are based on the aperiodic waveform.

16. A non-transitory computer-readable medium comprising processor-executable program code configured to cause a processor to:

generate a simulated reality environment;

obtain sensor data indicating an interaction associated with the simulated reality environment, the sensor data obtained at a sampling rate;

determine, based on the interaction, whether to output a static haptic effect or a dynamic haptic effect;

responsive to a determination to output a dynamic haptic effect, iteratively during the interaction substantially in real time and substantially at the sampling rate;

determine, using the obtained sensor data, one or more parameters of the dynamic haptic effect;

generate one or more a-haptic signals associated with the dynamic haptic effect based on the one or more parameters; and

transmit the one or more haptic signals associated with the dynamic haptic effect to a haptic output device to output the dynamic haptic effect in substantially real time.

17. The non-transitory computer-readable medium of claim 16, wherein the processor is further configured to execute processor-executable program code stored in the non-transitory computer-readable medium to:

receive a sensor signal indicating information about a user; and

determine whether to output the dynamic haptic effect based at least in part on the sensor signal, wherein the one or more haptic signals associated with the dynamic haptic effect are based on the dynamic haptic effect.

18. The non-transitory computer-readable medium of claim 17, wherein the processor is further configured to execute processor-executable program code stored in the non-transitory computer-readable medium to:

determine, based on the one or more parameters of the dynamic haptic effect, that the dynamic haptic effect is a periodic dynamic haptic effect or an aperiodic dynamic haptic effect, wherein the periodic dynamic haptic effect is based on a periodic waveform and the aperiodic dynamic haptic effect is based on an aperiodic waveform, and wherein the one or more haptic signals associated with the dynamic haptic effect are based on the periodic waveform or the aperiodic waveform.

19. The non-transitory computer-readable medium of claim 16, wherein the processor is further configured to execute processor-executable program code stored in the non-transitory computer-readable medium to:

determine, based on the interaction, to output a static haptic effect, wherein the static haptic effect is a predetermined haptic effect, and one or more haptic signals are associated with the static haptic effect.

20. The non-transitory computer-readable medium of claim 19, wherein the processor is further configured to execute processor-executable program code stored in the non-transitory computer-readable medium to:

determine one or more parameters of the static haptic effect, wherein the static haptic effect is a periodic static haptic effect or an aperiodic static haptic effect, the periodic static haptic effect is based on a periodic waveform and the aperiodic static haptic effect is based on an aperiodic waveform, and the one or more haptic signals associated with the static haptic effect are based on the periodic waveform or the aperiodic waveform.