Systems and methods for viewport-based augmented reality haptic effects

• US 10,444,829 B2
• Filed: 03/13/2018
• Issued: 10/15/2019
• Est. Priority Date: 05/05/2014
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Levesque, Vincent, Grant, Danny
• Primary Examiner(s): Abdulselam, Abbas I
• Application Number:
  US15/919,260
• Publication Number:
  US 20180239418A1
• Time in Patent Office: 581 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

One illustrative system disclosed herein includes a display configured to receive a display signal and output an image, and an image capture device configured to capture an area image and transmit an image signal. The illustrative system also includes a processor in communication with the image capture device and the display, the processor configured to: receive the image signal; determine a virtual object based in part on the image signal; determine the display signal based in part on the image signal, wherein the display signal includes data associated with the virtual object; determine a haptic effect based at least in part on the virtual object; and transmit a haptic signal associated with the haptic effect. The illustrative system further includes a haptic output device configured to receive the haptic signal and output the haptic effect.

Claims

1. A system comprising:

a processor; and

a memory comprising program code that is executable by the processor to cause the processor to;

receive image content from an image capture device;

identify a body part in the image content;

determine a virtual wearable device to be generated in an augmented reality environment, the virtual wearable device determined independently of the image content;

display the augmented reality environment on a display device, the augmented reality environment comprising a graphical representation of the virtual wearable device coupled to the body part;

detect an event in the augmented reality environment, wherein the event comprises a manipulation of the virtual wearable device; and

cause a haptic output device to output a haptic effect based on the event.

2. The system of claim 1, wherein the virtual wearable device is an animation that moves in the augmented reality environment.

3. The system of claim 1, wherein the event comprises movement or operation of the virtual wearable device in the augmented reality environment.

4. The system of claim 1, wherein the virtual wearable device is depicted as being positioned around the body part in the augmented reality environment.

5. The system of claim 1, wherein the virtual wearable device is virtual clothing, and wherein the haptic effect is configured to simulate wearing the virtual clothing.

6. The system of claim 5, wherein the virtual wearable device is a virtual weapon, and wherein the haptic effect is configured to simulate operation of the virtual weapon.

7. The system of claim 1, wherein the haptic effect is a remote haptic effect comprising at least one of a pressure wave, solid emission, liquid emission, gas emission, plasma emission, vibration, or texture.

8. The system of claim 1, wherein the haptic effect is a remote haptic effect comprising at least two remote haptic effects, the at least two remote haptic effects further comprising any combination of a pressure wave, solid emission, liquid emission, gas emission, plasma emission, vibration, or texture, and wherein the haptic output device is further configured to output the remote haptic effects in concert.

9. A method comprising:

receiving, by a processor, image content from an image capture device;

identifying, by the processor, a body part in the image content;

determining, by the processor, a virtual wearable device to be generated in an augmented reality environment, the virtual wearable device determined independently of the image content;

displaying, by the processor, the augmented reality environment on a display device, the augmented reality environment comprising a graphical representation of the virtual wearable device coupled to the body part;

detecting, by the processor, an event in the augmented reality environment, wherein the event comprises a manipulation of the virtual wearable device; and

causing, by the processor, a haptic output device to output a haptic effect based on the event.

10. The method of claim 9, wherein the virtual wearable device is an animation that moves in the augmented reality environment.

11. The method of claim 9, wherein the event comprises movement or operation of the virtual wearable device in the augmented reality environment.

12. The method of claim 9, wherein the body part is depicted as wearing the virtual wearable device in the augmented reality environment.

13. The method of claim 9, wherein the virtual wearable device is virtual clothing, and wherein the haptic effect is configured to simulate wearing the virtual clothing.

14. The method of claim 9, wherein the virtual wearable device is a virtual weapon, and wherein the haptic effect is configured to simulate operation of the virtual weapon.

15. The method of claim 9, wherein the haptic effect is a remote haptic effect that comprises at least one of a pressure wave, solid emission, liquid emission, gas emission, plasma emission, vibration, or texture.

16. The method of claim 9, wherein the haptic effect is a remote haptic effect comprising at least two remote haptic effects, the at least two remote haptic effects further comprising any combination of a pressure wave, solid emission, liquid emission, gas emission, plasma emission, vibration, or texture, and wherein the haptic output device is further configured to output the remote haptic effects in concert.

17. A non-transitory computer-readable medium comprising program code that is executable by a processor to cause the processor to:

receive image content from an image capture device;

identify a body part in the image content;

determine a virtual wearable device to be generated in an augmented reality environment, the virtual wearable device determined independently of the image content;

display the augmented reality environment on a display device, the augmented reality environment comprising a graphical representation of the virtual wearable device coupled to the body part;

detect an event in the augmented reality environment, wherein the event comprises a manipulation of the virtual wearable device; and

cause a haptic output device to output a haptic effect based on the event.

18. The non-transitory computer-readable medium of claim 17, wherein the virtual wearable device is an animation that moves in the augmented reality environment.

19. The non-transitory computer-readable medium of claim 17, wherein the event comprises movement or operation of the virtual wearable device in the augmented reality environment.

20. The non-transitory computer-readable medium of claim 17, wherein the body part is depicted as wearing the virtual wearable device in the augmented reality environment.