INTELLIGENT GLASSES FOR THE VISUALLY IMPAIRED

• US 20160372007A1
• Filed: 09/01/2016
• Published: 12/22/2016
• Est. Priority Date: 02/07/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cervantes, Humberto Orozco
• Granted Patent: US 9,805,619 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted
• CPC Class Codes: G09B 5/125 the stations being mobile
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices

Abstract

An approach for communicating navigation information on a physical environment to a user. The approach includes a computer receiving digital images of a physical environment of the user captured by digital video devices and converting the digital images into a three-dimensional image. The approach includes the computer analyzing the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment and determining at least one device associated with the user. The approach includes the computer formatting the output data for use with at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by an electrical stimulation pad, and receiving a touch from a user on a surface of the electrical stimulation pad corresponding to an object represented in the spatial map of the physical environment.

Claims

1. A method for communicating navigation information on a physical environment to a user, the method comprising:

receiving, by a server computer, at least one digital image of a physical environment of the user captured by one or more digital video devices;

converting, by the server computer, the at least one digital image into a three-dimensional image;

analyzing, by the server computer, the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment;

determining, by the server computer, at least one device associated with the user; and

formatting, by the server computer, the output data for use with the at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by an electrical stimulation pad, and receiving a touch from a user on a surface of the electrical stimulation pad corresponding to an object represented in the spatial map of the physical environment.

2. The method of claim 1, wherein formatting the output data for use with the at least one device, further comprises:

receiving, by the server computer, from the at least one device, a request for information on the object based, at least in part, on the touch from the user on the surface of the electrical stimulation pad corresponding to the object; and

retrieving, by the server computer, a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object.

3. The method of claim 1, wherein analyzing the three dimensional image using object analysis to generate output data further comprises:

determining, by the server computer, a distance to an object, based, at least in part, on at least one of a focal point of the user'"'"'s gaze and a touch by the user on a surface of the at least one device corresponding to the object; and

determining, by the server computer, output data as a series of beeps that correspond to the distance to the object, based, at least in part, on the distance to the object wherein a tone of the beeps provides information on a type of object and information on a motion of the object.

4. The method of claim 1, wherein formatting the output data for use with the at least one device further comprises:

transmitting, by the server computer, the output data to the at least one device associated with the user, wherein the output data is formatted to create the spatial map on the electrical stimulation pad that depicts the physical environment, the output data is formatted to create audio responses from one of a smartphone and an intelligent computing device, and the output data is formatted for a series of beeps representing information on the object in the physical environment from one of a smartphone and an intelligent computing device.

5. The method of claim 1, wherein the at least one device associated with the user includes at least one of:

an audio device, a smart phone, and an intelligent computing device.

6. The method of claim 1, wherein formatting the output data for use with the at least one device further comprises:

determining, by the server computer, a level of intensity of the electrical stimulation associated with the object represented in the spatial map of the physical environment, the level corresponding to a motion of the object, a proximity of the object to the user, and a terrain in the physical environment.

7. The method of claim 1, further comprising:

creating, by the server computer, a representation of the physical environment using a matrix of electrical stimulation electrodes in the electrical stimulation pad providing varying levels of intensity of electrical stimulation to represent a terrain, one or more objects in the terrain, a distance of the one or more objects to the user, and motion of the one or more objects in the physical environment.

8. The method of claim 1, further comprising:

receiving, by the server computer, the touch from the user on the surface of the electrical stimulation pad for requesting at least one of an estimated distance to the object and an identification of the object.

9. A computer program product for communicating navigation information on a physical environment to a user, the computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions executable by a processor, the program instructions comprising instructions for;

receiving at least one digital image of a physical environment of the user captured by one or more digital video devices;

converting the at least one digital image into a three-dimensional image;

analyzing the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment;

determining at least one device associated with the user; and

formatting the output data for use with the at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by an electrical stimulation pad, and receiving a touch from a user on a surface of the electrical stimulation pad corresponding to an object represented in the spatial map of the physical environment.

10. The computer program product of claim 9, wherein formatting the output data for use with the at least one device, further comprises:

receiving, by the server computer, from the at least one device, a request for information on the object based, at least in part, on the touch from the user on the surface of the electrical stimulation pad corresponding to the object; and

retrieving, by the server computer, a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object.

11. The computer program product of claim 9, wherein analyzing the three dimensional image using object analysis to generate output data further comprises:

determining a distance to an object, based, at least in part, on at least one of a focal point of the user'"'"'s gaze and a touch by the user on a surface of the at least one device corresponding to the object; and

determining output data as a series of beeps that correspond to the distance to the object, based, at least in part, on the distance to the object wherein a tone of the beeps provides information on a type of object and information on a motion of the object.

12. The computer program product of claim 9, wherein formatting the output data for use with the at least one device further comprises:

transmitting the output data to the at least one device associated with the user, wherein the output data is formatted to create the spatial map on the electrical stimulation pad that depicts the physical environment, the output data is formatted to create audio responses from one of a smartphone and an intelligent computing device, and the output data is formatted for a series of beeps representing information on the object in the physical environment from one of a smartphone and an intelligent computing device.

13. The computer program product of claim 9, wherein the at least one device associated with the user includes at least one of:

an audio device, a smart phone, and an intelligent computing device.

14. The computer program product of claim 9, wherein formatting the output data for use with the at least one device further comprises:

determining a level of intensity of the electrical stimulation associated with the object represented in the spatial map of the physical environment, the level corresponding to a motion of the object, a proximity of the object to the user, and a terrain in the physical environment.

15. The computer program product of claim 9, further comprising:

creating a representation of the physical environment using a matrix of electrical stimulation electrodes in the electrical stimulation pad providing varying levels of intensity of electrical stimulation to represent a terrain, one or more objects in the terrain, a distance of the one or more objects to the user, and motion of the one or more objects in the physical environment.

16. The computer program product of claim 9, further comprising:

receiving the touch from the user on the surface of the electrical stimulation pad for requesting at least one of an estimated distance to the object and an identification of the object.

17. A computer system for communicating navigation information on a physical environment to a user, the computer system comprising:

one or more computer processors;

one or more computer readable storage media;

program instructions stored on at least one of the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising instructions to perform;

receiving at least one digital image of a physical environment of the user captured by one or more digital video devices;

converting the at least one digital image into a three-dimensional image;

analyzing the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment;

determining at least one device associated with the user; and

formatting the output data for use with the at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by an electrical stimulation pad, and receiving a touch from a user on a surface of the electrical stimulation pad corresponding to an object represented in the spatial map of the physical environment.

18. The computer system of claim 17, wherein formatting the output data for use with the at least one device, further comprises:

receiving, by the server computer, from the at least one device, a request for information on the object based, at least in part, on the touch from the user on the surface of the electrical stimulation pad corresponding to the object; and

retrieving, by the server computer, a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object.

19. The computer system of claim 17, wherein analyzing the three dimensional image using object analysis to generate output data further comprises:

determining a distance to an object, based, at least in part, on at least one of a focal point of the user'"'"'s gaze and a touch by the user on a surface of the at least one device corresponding to the object; and

determining output data as a series of beeps that correspond to the distance to the object, based, at least in part, on the distance to the object wherein a tone of the beeps provides information on a type of object and information on a motion of the object.

20. The computer system of claim 17, wherein formatting the output data for use with the at least one device further comprises:

transmitting the output data to the at least one device associated with the user, wherein the output data is formatted to create the spatial map on the electrical stimulation pad that depicts the physical environment, the output data is formatted to create audio responses from one of a smartphone and an intelligent computing device, and the output data is formatted for a series of beeps representing information on the object in the physical environment from one of a smartphone and an intelligent computing device.