FINGERTIP VISUAL HAPTIC SENSOR CONTROLLER

• US 20080226134A1
• Filed: 03/07/2008
• Published: 09/18/2008
• Est. Priority Date: 03/12/2007
• Status: Active Grant

• Current Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Original Assignee: University of Pittsburgh of The Commonwealth System of Higher Education
• Inventors: STETTEN, George DeWitt, Klatzky, Roberta
• Granted Patent: US 9,024,874 B2
• Time in Patent Office: Days
• US Class Current: 382/114
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037

Abstract

Systems, methods and devices for capturing environmental image data and providing tactile feedback about the visual environment to a user. One or more image sensing devices are mounted on various appendages of a user and are used to sense the environment. The image data is sent to a computer vision system where it is processed in order to identify specific objects within the environment. When the computer vision system makes such an identification it may send a signal to a haptic stimulator mounted on a user'"'"'s appendage or may be used to control an object in the environment.

Claims

4. A visual sensing and tactile stimulation system, comprising:

a first image sensing device;

a first haptic stimulator associated with said first image sensing device; and

a computer vision system in communication with said first image sensing device and first haptic stimulator, wherein said computer vision system receives image data from said first image sensing device and sends a signal to said first haptic stimulator when said received image data corresponds to a recognized object.

1. A visual sensing and control system, comprising:

a first image sensing device;

a first controller associated with said first image sensing device; and

a computer vision system in communication with said first image sensing device and first controller, wherein said computer vision system receives image data from said first image sensing device and sends a signal through said controller to control a recognized object.

2. The system according to claim 1, further comprising a haptic stimulator.

3. The system according to claim 1, wherein the image sensing device is a video camera.

5. The system according to claim 4, further comprising:

a second image sensing device; and

a second haptic stimulator associated with said second image sensing device, wherein said second image sensing device and second haptic stimulator are in communication with said computer vision system.

6. The system according to claim 4, wherein the image sensing device is a video camera.

7. The system according to claim 4, wherein the image sensing device is a laser range finder.

8. The system according to claim 4, wherein the haptic stimulator is a vibratory transducer.

10. The system according to claim 4, further comprising a plurality of navigational beacons.

8-1. The system according to claim 4, wherein the haptic stimulator is a pneumatic device.

11. A method for remotely sensing the environment, comprising the steps of:

gathering image data using an image sensing device mounted on a user;

communicating the image data to a computer vision system;

processing the image data by the computer vision system to identify a specific feature in the environment; and

transmitting a signal to a haptic stimulator mounted on a user which provides tactile sensations when the specific feature in the environment is identified by the computer vision system.

12. A method according to claim 11, wherein the image sensing device and haptic stimulator are mounted on the dorsal aspect of a fingertip.

13. A method according to claim 11, wherein said processing step identifies edges or obstructions in the environment, and further wherein said transmitting step includes transmitting signals to the haptic stimulator when edges or obstructions in the environment are identified.

14. A method according to claim 11, wherein the computer vision system includes a database housing a plurality of predefined features in the environment.

15. A method according to claim 14, wherein the specific feature in said processing step is selected from said database via voice command.

16. A method according to claim 11, further comprising transmitting a command signal from said computer vision system to a controller associated with an object in the environment.

17. A method according to claim 11, wherein the image sensing device is mounted on the dorsal aspect of a fingertip.

18. A method according to claim 16, wherein the specific object identified by the computer vision system is a cursor on a computer screen, and further wherein the controller transmits signals to move the cursor on the computer screen.

19. A method according to claim 11, wherein said step of gathering image data includes gathering image data using a plurality of image sensing devices mounted on a user.

20. A method according to claim 19, wherein said step of processing the image data includes the computer vision system identifying a different specific feature in the environment for each of said plurality of image sensing devices.