Synchronous interpersonal haptic communication system

• US 8,294,557 B1
• Filed: 06/09/2009
• Issued: 10/23/2012
• Est. Priority Date: 06/09/2009
• Status: Active Grant

• Current Assignee: University of Ottawa
• Original Assignee: University of Ottawa
• Inventors: El Saddik, Abdulmotaleb, Cha, Jongeun, Eid, Mohamad Ahmad, A Alsulaiman, Fawaz Abdulaziz, Alamri, Atif Mohammad, Rahal, Lara, Martin, Daniel J.
• Primary Examiner(s): Nguyen, Hung T.
• Application Number:
  US12/481,274
• Time in Patent Office: 1,232 Days
• Field of Search: 340/407.1, 340/407.2, 340/7.6, 340/7.61, 340/815.4, 340/425.5, 340/691.6, 345/156, 345/158, 345/173, 345/178
• US Class Current: 340/407.1
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user

Abstract

A method, apparatus and system of synchronous haptic communications between an active user and a passive user through a network is provided. An image from an imaging device is captured of the passive user. Geometry or position data is determined from the captured image by utilizing a depth component of the capture image. Haptic feedback is generated to a haptic interface used by the active user based upon the determined geometry or position data. Haptic data is captured from the haptic interface used by the active user and tactile output is generated to a tactile display associated with the passive user wherein the tactile output is based upon the haptic data in correlation with the geometry or position data to determine collisions therein.

Claims

1. A method of synchronous haptic communications between an active user and a passive user through a network, the method comprising:

capturing an image from an imaging device of the passive user and determining geometry or position data from the captured image by utilizing a depth component of the captured image;

capturing haptic data from a haptic interface used by the active user based upon interaction with a generated image based on the captured image;

generating tactile output to a tactile display associated with the passive user wherein the tactile output is based upon the haptic data in correlation with the geometry or position data to determine collisions therein; and

generating haptic feedback to the haptic interface used by the active user based upon the haptic data in correlation with the geometry or position data to determine collisions therein is controlled/processed by a processor.

2. The method of claim 1 wherein the geometry or position data is transmitted through the network at a rate less than 60 Hz.

3. The method of claim 1 wherein the haptic data is represented in an RGB (Red, Green, Blue)-like pixel-based frames to describe the haptic properties associated with each pixel in the RGB frame wherein the haptic properties.

4. The method of claim 3 wherein the haptic properties are selected from stiffness, static friction, and dynamic friction.

5. The method of claim 1 wherein generating tactile output to the tactile display simulates touch by using a funneling illusion to a desired stimulus location by activating actuators in proximity of the desired stimulus location in the tactile display to provide a continuous sensation.

6. The method of claim 1 wherein the imaging device generates the depth image associated with the capture image of the passive user by using a 2.5-Dimensional or 3-Dimensional imaging.

7. The method of claim 1 further comprising determining a pose of the haptic device from the active user input to calculate if a collision has occurred with the captured image to generate an haptic feedback to the active user.

8. The method of claim 1 further comprising continuously performing active user identification and verification to determine in real time an identity of the active user.

9. An apparatus for synchronous haptic communications through a network, the apparatus comprising:

a processor;

a memory containing instructions for execution by the processor, the instructions comprising;

capturing an image from an imaging device of the passive user and determining geometry or position data from the captured image by utilizing a depth component of the captured image;

capturing haptic data from a haptic interface used by the active user based upon interaction with a generated image based on the captured image;

generating tactile output to a tactile display associated with the passive user wherein the tactile output is based upon the haptic data in correlation with the geometry or position data to determine collisions therein; and

generating haptic feedback to the haptic interface used by the active user based upon the haptic data in correlation with the geometry or position data to determine collisions therein.

10. The apparatus of claim 9 wherein the geometry or position is data is transmitted through the network at a rate less than 60 Hz.

11. The apparatus of claim 9 wherein the representation of the remote user is a computer generated avatar.

12. The apparatus of claim 9 wherein the tactile display is a device selected from a group comprising a tactile jacket, tactile vest, tactile glove, tactile suit and furniture incorporating addressable actuators.

13. The apparatus of claim 9 wherein the tactile device has visual indicators for determining positions of the actuators within the device, device location is determined based on the image of the passive user.

14. The apparatus of claim 9 further comprising an avatar database for providing an avatar representation of a user for display in a virtual environment.

15. A system for synchronous haptic communications between an active user and a passive user through a network, the system comprising:

an image coding module for capturing images of the passive user and for generating a depth image to generate geometry or position data of the passive user;

at least one network manager for facilitating networking and communication between the active user and passive user devices;

a haptic render for converting a haptic position input and generating haptic feedback based on a haptic input device for receiving position input from the active user and providing haptic feedback to the active user;

a tactile render for providing control commands to a tactile display for providing tactile output to the passive user based upon received contact data from the network manager;

a video render for rendering an image or representation of the passive user to an video display to the active user; and

a virtual space manager for processing geometry or position data of the passive user with haptic position input of the active user to generate the contact data, and provide the contact data to the tactile render and to the haptic render to simulate touch between the active user and passive user.

16. The system of claim 15 wherein the geometry or position is data is transmitted through the network at a rate less than 60 Hz.

17. The system of claim 15 further comprising an avatar repository for providing representation of one or more avatars to be utilized in place of an image of the user.

18. The system of claim 15 wherein the tactile display is a device selected from a group comprising a tactile jacket, tactile vest, tactile glove, tactile suit or furniture incorporating addressable actuators.

19. The system of claim 18 wherein the tactile rendered simulates touch by using a funneling illusion to a desired stimulus location by activating actuators in proximity of the desired stimulus location in the tactile device to provide a continuous sensation.

20. The system of claim 15 further comprising a haptic authentication module for continuously performing active user identification and verification to determine in real time an identity of the active user.