PORTABLE ELECTRONIC HAPTIC VISION DEVICE

• US 20190332175A1
• Filed: 03/15/2019
• Published: 10/31/2019
• Est. Priority Date: 04/25/2018
• Status: Active Grant

• Current Assignee: Seventh Sense OU
• Original Assignee: Seventh Sense OU
• Inventors: VAANANEN, Johannes, PUKARI, Mika
• Granted Patent: US 10,795,446 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

The invention provides a portable electronic haptic vision device configured to be attached to or worn by a user. The portable electronic haptic vision device is arranged to provide haptic feedback with pressurized air on skin of the user based upon detected objects in the surroundings of the user. The information about the detected objects in the surroundings is captured with a digital camera, radar and/or sonar device, and/or a 3-D capture device such as a 3-D scanner or 3-D camera attached to the portable electronic haptic vision device. The portable electronic haptic vision device is realised as a helmet with at least two cameras placed in the position of the eyes of the user, or as a t-shirt or other wearable.

Claims

1. A portable electronic haptic vision device, configured to be attached to or worn by a user, and arranged to provide haptic feedback with pressurized air on skin of the user based upon detected objects in the surroundings of the user.

2. A portable electronic haptic vision device, as claimed in claim 1, information about the detected objects in the surroundings is captured with a digital camera, a 3-D capturing device, 3-D scanner, radar and/or sonar device attached to the portable electronic haptic vision device, and optionally recorded as video or 3D point cloud.

3. A portable electronic haptic vision device, as claimed in claim 2, information about the detected objects in the surroundings are captured within the field of view of the user, creating a natural sense-based vision.

4. A portable electronic haptic vision device, as claimed in claim 2, information about the detected objects in the surroundings is captured with a 360 degree field of view attached to the portable electronic haptic vision device, capturing spatial surroundings around the user.

5. A portable electronic haptic vision device, as claimed in claim 1, characterized in that, the portable electronic haptic vision device is realized as any of the following:

helmet wherein optionally at least two cameras are placed in front of the position of the eyes of the user, as a t-shirt, belt or other wearable.

6. A portable electronic haptic vision device, as claimed in claim 1, characterized in that, the portable electronic haptic vision device features a light to illuminate the surroundings of the user.

7. A portable electronic haptic vision device, as claimed in claim 6, characterized in that, the light is a low power light such as LED, narrow emission band LED, infrared light and/or a flash.

8. A portable electronic haptic vision device, as claimed in claim 1, characterized in that, the user is blind or a gaming user playing a virtual reality game.

9. A portable electronic haptic vision device, as claimed in claim 1, characterized in that, the pressurized air is configured to be directed to the skin of the user via at least one nozzle, straw, pipe and/or funnel that may be movable.

10. A portable electronic haptic vision device, as claimed in claim 1, characterized in that, based upon an image, video and/or 3-D point cloud obtained from the surroundings, any of the following is drawn on the skin of the user with pressurized air:

routes, pedestrian walkways, streets, stairs, corridors, furniture, walls, obstacles;

impressions and expressions from another person'"'"'s face; and

video signal obtained from a screen, such as a movie or computer game.

11. A portable electronic haptic vision device, as claimed in claim 1, characterized in that, the portable electronic haptic vision device is configured to track the eyes of the user, zoom at least one camera in or out, and/or photo- and/or videograph surroundings based upon eye tracking of the user.

12. A portable electronic haptic vision device, as claimed in claim 1, characterized in that, the portable electronic haptic vision device is connected to another peripheral device on the same user providing haptic feedback, optionally with pressurized air.

13. A software program product for a portable electronic haptic vision device, configured to be attached to or worn by a user, and arranged to provide haptic feedback with pressurized air on the skin of the user based upon the detected objects in the surroundings of the user, characterized by the software program product comprising:

an image capturing module and/or a 3D capturing device to capture images and/or a 3D point cloud from the surroundings of the user;

a processing module to process information gathered from the image capturing module and/or the 3D capturing device; and

a driving module to direct at least one nozzle to spray pressurized air on skin of the user based upon processed information from the processing module.

14. A software program product as claimed in claim 13, characterized in that, the driving module is configured to comprise at least one pump and valve controller.

15. A software program product as claimed in claim 13, wherein information about the detected objects in the surroundings is captured using the image capturing module and/or the 3D capturing device associated with the portable electronic haptic vision device and recorded as images, video and/or 3D-point cloud, wherein the image capturing module is a digital camera, radar and/or sonar device, and/or the 3D capturing device is a 3D scanner or 3D camera, which are configured to be attached, or attached, to the portable electronic haptic vision device.

16. A software program product as claimed in claim 15, wherein information about the detected objects in the surroundings is captured within a field of view of the user, creating a natural sense based vision.

17. A software program product as claimed in claim 15, information about the detected objects in the surroundings and the spatial surroundings around the user is captured with a 360 degree field of view.

18. A software program product as claimed in claim 13, characterized in that, the software program product implemented in the portable electronic haptic vision device is realized as any of the following:

helmet wherein optionally at least two cameras are placed in front of the position of the eyes of the user, as a t-shirt, belt or other wearable.

19. A software program product as claimed in claim 13, characterized in that, the software program product implemented in the portable electronic haptic vision device features a light to illuminate the surroundings of the user.

20. A software program product as claimed in claim 13, characterized in that, the light is a low power light such as LED, narrow emission band LED, infrared light and/or a flash.

21. A software program product as claimed in claim 13, characterized in that, the user is a blind individual or a gaming individual playing a virtual reality computer game.

22. A software program product as claimed in claim 13, characterized in that, the driving module directs the pressurized air to the skin of the user via at least one nozzle, straw, pipe and/or funnel that may be movable.

23. A software program product as claimed in claim 13, characterized in that, based upon images, video or a 3-D point cloud captured from the surroundings, the driving module draws symbols pertaining to any of the following on the skin of the user with pressurized air:

routes, pedestrian walkways, streets, stairs, corridors, furniture, walls, obstacles;

impressions and expressions from another person'"'"'s face; and

video signal obtained from a screen, such as a movie or computer game.

24. A software program product as claimed in claim 13, characterized in that, the portable electronic haptic vision device is configured to track the eyes of the user, zoom at least one camera in or out, and/or photo- and/or videograph surroundings based upon eye tracking of the user.

25. A software program product as claimed in claim 13, characterized in that, the portable electronic haptic vision device is connected to another peripheral device on the same user providing haptic feedback also, optionally with pressurized air.

26. A method for operating a portable electronic haptic vision device, configured to be attached to or worn by a user, and arranged to provide haptic feedback with pressurized air on the skin of the user based upon the detected objects in the surroundings of the user, characterized by the following steps,

capturing images, video and/or a 3-D point cloud from the surroundings of the user using an image capturing module and/or a 3D capturing device;

processing information gathered from the image capturing module and/or the 3D capturing device with a processing module; and

directing pressurized air on skin of the user, by a driving module, based upon processed information in the processing module via at least one nozzle.

27. A method as claimed in claim 26, characterized in that, the driving module comprises at least one pump and valve controller to produce the haptic feedback with pressurized air based upon the received processed information from the processing module.

28. A method as claimed in claim 26, characterized in that, capturing information about the detected objects in the surroundings within the field of view of the user, creates a natural sense based vision for the user.

29. A method as claimed in claim 28, characterized in that, capturing information about the visual objects and the spatial surroundings around the user with a 360 degree field of view creates sensing in all directions for the user.

30. A method as claimed in claim 26, characterized in that, realizing the portable electronic haptic vision device as any of the following:

a helmet, wherein optionally at least two cameras are placed in front of the position of the eyes of the user, as a t-shirt, belt or other wearable.

31. A method as claimed in claim 26, characterized in that, featuring a light in the portable electronic haptic vision device to illuminate the surroundings of the user.

32. A method as claimed in claim 26, characterized in that, the light is a low power light such as LED, narrow emission band LED, infrared light and/or a flash.

33. A method as claimed in claim 26, characterized in that, the user is a blind individual or a gaming individual playing a virtual reality game.

34. A method as claimed in claim 26, characterized in that, directing, by the driving module, the pressurized air to the skin of the user via at least one nozzle, straw, pipe and/or funnel that may be movable.

35. A method as claimed in claim 26, characterized in that, based upon images, video and/or a 3-D point cloud obtained from the surroundings, any of the following is drawn on the skin of the user with pressurized air:

routes, pedestrian walkways, streets, stairs, corridors, furniture, walls, obstacles;

impressions and expressions from another person'"'"'s face; and

video signal obtained from a screen, such as a movie or computer game.

36. A method as claimed in claim 26, characterized in that, tracking the eyes of the user, zoom at least one camera in or out, and/or photo- and/or videograph surroundings based upon eye tracking of the user.

37. A method as claimed in claim 26, characterized in that, connecting the portable electronic haptic vision device to another peripheral device on the same user providing haptic feedback, optionally with pressurized air.