Systems and methods for viewport-based augmented reality haptic effects

• US 9,946,336 B2
• Filed: 05/19/2017
• Issued: 04/17/2018
• Est. Priority Date: 05/05/2014
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Levesque, Vincent, Grant, Danny
• Primary Examiner(s): ABDULSELAM, ABBAS I
• Application Number:
  US15/600,141
• Publication Number:
  US 20170255255A1
• Time in Patent Office: 333 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

One illustrative system disclosed herein includes a display configured to receive a display signal and output an image, and an image capture device configured to capture an area image and transmit an image signal. The illustrative system also includes a processor in communication with the image capture device and the display, the processor configured to: receive the image signal; determine a virtual object based in part on the image signal; determine the display signal based in part on the image signal, wherein the display signal includes data associated with the virtual object; determine a haptic effect based at least in part on the virtual object; and transmit a haptic signal associated with the haptic effect. The illustrative system further includes a haptic output device configured to receive the haptic signal and output the haptic effect.

Claims

1. A system comprising:

a processor in communication with an image capture device;

a memory comprising program code that is executable by the processor to cause the processor to;

receive an image of a physical object from an image capture device;

identify the physical object in the image;

determine information about the physical object;

determine a virtual object based on the information about the physical object, wherein the virtual object is different from a virtual representation of the physical object;

cause a display device to display an augmented reality environment having the virtual object;

detect an event in the augmented reality environment, wherein the event is associated with the virtual object; and

cause a haptic output device to output a haptic effect based on the event.

2. The system of claim 1, wherein the information about the physical object comprises a make, model, manufacturer, or contents of the physical object.

3. The system of claim 2, wherein the information about the physical object comprises the contents of the physical object, and wherein the virtual object is a virtual representation of the contents of the physical object.

4. The system of claim 1, wherein the information about the physical object comprises a company related to the physical object.

5. The system of claim 4, wherein the virtual object comprises a logo of the company related to the physical object.

6. The system of claim 1, wherein the memory further comprises program code that is executable by the processor to cause the processor to output another haptic effect indicating the information about the physical object.

7. The system of claim 1, wherein the augmented reality environment is configured to have the virtual object positioned on a virtual representation of a body part.

8. The system of claim 7, wherein the event comprises a movement of the virtual object along the virtual representation of the body part in the augmented reality environment.

9. A method comprising:

receiving, by a processor, an image of a physical object from an image capture device;

identifying, by the processor, the physical object in the image;

determining, by the processor, information about the physical object;

determining, by the processor, a virtual object based on the information about the physical object, wherein the virtual object is different from a virtual representation of the physical object;

causing, by the processor, a display device to display an augmented reality environment having the virtual object;

detecting, by the processor, an event in the augmented reality environment, wherein the event is associated with the virtual object; and

causing, by the processor, a haptic output device to output a haptic effect based on the event.

10. The method of claim 9, wherein the information about the physical object comprises a make, model, manufacturer, or contents of the physical object.

11. The method of claim 10, wherein the information about the physical object comprises the contents of the physical object, and wherein the virtual object is a virtual representation of the contents of the physical object.

12. The method of claim 9, wherein the information about the physical object comprises a company related to the physical object, and wherein the virtual object comprises a logo of the company related to the physical object.

13. The method of claim 9, further comprising output another haptic effect indicating the information about the physical object.

14. The method of claim 9, wherein the augmented reality environment is configured to have the virtual object positioned on a virtual representation of a body part, and wherein the event comprises a movement of the virtual object along the virtual representation of the body part in the augmented reality environment.

15. The method of claim 9, wherein the haptic output device is coupled to an article configured to be worn by a user.

16. A non-transitory computer-readable medium comprising program code that is executable by a processor to cause the processor to:

receive an image of a physical object from an image capture device;

identify the physical object in the image;

determine information about the physical object;

determine a virtual object based on the information about the physical object, wherein the virtual object is different from a virtual representation of the physical object;

cause a display device to display an augmented reality environment having the virtual object;

detect an event in the augmented reality environment, wherein the event is associated with the virtual object; and

cause a haptic output device to output a haptic effect based on the event.

17. The non-transitory computer-readable medium of claim 16, wherein the information about the physical object comprises a make, model, manufacturer, or contents of the physical object.

18. The non-transitory computer-readable medium of claim 16, wherein the information about the physical object comprises the contents of the physical object, and wherein the virtual object is a virtual representation of the contents of the physical object.

19. The non-transitory computer-readable medium of claim 16, wherein the information about the physical object comprises a company related to the physical object, and wherein the virtual object comprises a logo of the company related to the physical object.

20. The non-transitory computer-readable medium of claim 16, wherein the augmented reality environment is configured to have the virtual object positioned on a virtual representation of a body part, and wherein the event comprises a movement of the virtual object along the virtual representation of the body part in the augmented reality environment.