Intelligent glasses for the visually impaired

• US 9,805,619 B2
• Filed: 09/01/2016
• Issued: 10/31/2017
• Est. Priority Date: 02/07/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cervantes, Humberto Orozco
• Primary Examiner(s): VO, TUNG T
• Application Number:
  US15/254,486
• Publication Number:
  US 20160372007A1
• Time in Patent Office: 425 Days
• Field of Search: 348 47
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted
• CPC Class Codes: G09B 5/125 the stations being mobile
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices

Abstract

An approach for communicating navigation information on a physical environment to a user. The approach includes a computer receiving digital images of a physical environment of the user captured by digital video devices and converting the digital images into a three-dimensional image. The approach includes the computer analyzing the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment and determining at least one device associated with the user. The approach includes the computer formatting the output data for use with at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by an electrical stimulation pad, and receiving a touch from a user on a surface of the electrical stimulation pad corresponding to an object represented in the spatial map of the physical environment.

Claims

1. A method for communicating navigation information on a physical environment to a user, the method comprising:

receiving, by a server computer, at least one digital image of a physical environment of the user captured by one or more digital video devices;

converting, by the server computer, the at least one digital image into a three-dimensional image;

analyzing, by the server computer, the three-dimensional image using object analysis to generate output data corresponding to the physical environment, wherein the object analysis includes determining a motion of one or more objects based, at least in part, on a comparison of a digital image to a previous digital image from digital image data captured by the one or more digital video devices in the physical environment for identification to a user, where the motion of the one or more objects is at least one of a pulse of a plurality of electrical stimulation electrodes moving across an area of the electrical stimulation pad of the at least one device corresponding to the motion of each of the one or more objects in the physical environment or is a pitch of one or more beeps that corresponds to the motion of each of the one or more object in the physical environment;

determining, by the server computer, at least one device associated with the user, wherein the at least one device includes a wearable device with a first side of the at least one device including an electrical stimulation pad and a second side of the at least one device including a touch pad;

formatting, by the server computer, the output data for use with the at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by the electrical stimulation pad on the first side of the at least one device and providing the identification of the motion of the one or more objects on the first side of the at least one device; and

receiving, by the server computer, a user query for an identification of an object from a user touch on the second side of the at least one device.

2. The method of claim 1, wherein receiving the user query for the identification of the object from the user touch on the second side of the at least one device, further comprises:

receiving, by the server computer, from the second side of the at least one device, an indication of a user request for a name of the object based, at least in part, on a touch from the user on a location on a surface of the second side of the at least one device corresponding to a representation of the object in the spatial map of the physical environment provided on the first side of the at least one device;

retrieving, by the server computer, a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object; and

providing, by the server computer, the name of the object to the user by one of an audio device, a smartphone, or an intelligent computing device.

3. The method of claim 1, wherein formatting the output data for use with the at least one device further comprises:

transmitting, by the server computer, the output data to the at least one device associated with the user, wherein the output data is formatted to create the spatial map of the physical environment using the electrical stimulation pad on the first side of the at least one device and the output data is formatted to create audio responses from one of an audio device, a smartphone and an intelligent computing device representing information on the object in the physical environment.

4. The method of claim 1, wherein the at least one device associated with the user includes at least one of:

an audio device, a smart phone, and an intelligent computing device.

5. The method of claim 1, wherein formatting the output data for use with the at least one device further comprises:

determining, by the server computer, a level of intensity of the electrical stimulation associated with the object represented in the spatial map of the physical environment, the level corresponding to a motion of the object, a proximity of the object to the user, and a terrain in the physical environment.

6. The method of claim 1, further comprising:

creating, by the server computer, a representation of the physical environment using a matrix of electrical stimulation electrodes in the electrical stimulation pad providing varying levels of intensity of electrical stimulation to represent a terrain, one or more objects in the terrain, a distance of the one or more objects to the user, and motion of the one or more objects in the physical environment.

7. A computer program product for communicating navigation information on a physical environment to a user, the computer program product comprising:

one or more non-transitory computer readable storage device and program instructions stored on the one or more non-transitory computer readable storage device, the program instructions executable by a processor, the program instructions comprising instructions for;

receiving at least one digital image of a physical environment of the user captured by one or more digital video devices;

converting the at least one digital image into a three-dimensional image;

analyzing the three-dimensional image using object analysis to generate output data corresponding to the physical environment, wherein the object analysis includes determining a motion of one or more objects based, at least in part, on a comparison of a digital image to a previous digital image from digital image data captured by the one or more digital video devices in the physical environment for identification to a user, where the motion of the one or more objects is at least one of a pulse of a plurality of electrical stimulation electrodes moving across an area of the electrical stimulation pad of the at least one device corresponding to the motion of each of the one or more objects in the physical environment or is a pitch of one or more beeps that corresponds to the motion of each of the one or more object in the physical environment;

determining at least one device associated with the user, wherein the at least one device includes a wearable device with a first side of the at least one device including an electrical stimulation pad and a second side of the at least one device including a touch pad;

formatting the output data for use with the at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by the electrical stimulation pad on the first side of the at least one device and providing the identification of the motion of the one or more objects on the first side of the at least one device; and

receiving a user query for an identification of an object from a user touch on the second side of the at least one device.

8. The computer program product of claim 7, wherein receiving the user query for the identification of the object from the user touch on the second side of the at least one device, further comprises:

receiving from the second side of the at least one device, an indication of a user request for a name of the object based, at least in part, on a touch from the user on a location on a surface of the second side of the at least one device corresponding to a representation of the object in the spatial map of the physical environment provided on the first side of the at least one device;

retrieving a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object; and

providing the name of the object to the user by one of an audio device, a smartphone, or an intelligent computing device.

9. The computer program product of claim 7, wherein formatting the output data for use with the at least one device further comprises:

transmitting, by the server computer, the output data to the at least one device associated with the user, wherein the output data is formatted to create the spatial map of the physical environment using the electrical stimulation pad on the first side of the at least one device and the output data is formatted to create audio responses from one of an audio device, a smartphone and an intelligent computing device representing information on the object in the physical environment.

10. The computer program product of claim 7, wherein the at least one device associated with the user includes at least one of:

an audio device, a smart phone, and an intelligent computing device.

11. The computer program product of claim 7, wherein formatting the output data for use with the at least one device further comprises:

determining a level of intensity of the electrical stimulation associated with the object represented in the spatial map of the physical environment, the level corresponding to a motion of the object, a proximity of the object to the user, and a terrain in the physical environment.

12. The computer program product of claim 7, further comprising:

creating a representation of the physical environment using a matrix of electrical stimulation electrodes in the electrical stimulation pad providing varying levels of intensity of electrical stimulation to represent a terrain, one or more objects in the terrain, a distance of the one or more objects to the user, and motion of the one or more objects in the physical environment.

13. A computer system for communicating navigation information on a physical environment to a user, the computer system comprising:

one or more computer processors;

one or more computer readable storage media;

program instructions stored on at least one of the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising instructions to perform;

receiving at least one digital image of a physical environment of the user captured by one or more digital video devices;

converting the at least one digital image into a three-dimensional image;

analyzing the three-dimensional image using object analysis to generate output data corresponding to the physical environment, wherein the object analysis includes determining a motion of one or more objects based, at least in part, on a comparison of a digital image to a previous digital image from digital image data captured by the one or more digital video devices in the physical environment for identification to a user, where the motion of the one or more objects is at least one of a pulse of a plurality of electrical stimulation electrodes moving across an area of the electrical stimulation pad of the at least one device corresponding to the motion of each of the one or more objects in the physical environment or is a pitch of one or more beeps that corresponds to the motion of each of the one or more object in the physical environment;

determining at least one device associated with the user, wherein the at least one device includes a wearable device with a first side of the at least one device including an electrical stimulation pad and a second side of the at least one device including a touch pad;

formatting the output data for use with the at least one device, wherein the at least one device is capable of providing to the user a spatial map of the physical environment created by the electrical stimulation pad on the first side of the at least one device and providing the identification of the motion of the one or more objects on the first side of the at least one device; and

receiving a user query for an identification of an object from a user touch on the second side of the at least one device.

14. The computer system of claim 13, wherein receiving the user query for the identification of the object from the user touch on the second side of the at least one device, further comprises:

receiving from the second side of the at least one device, an indication of a user request for a name of the object based, at least in part, on a touch from the user on a location on a surface of the second side of the at least one device corresponding to a representation of the object in the spatial map of the physical environment provided on the first side of the at least one device;

retrieving a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object; and

providing the name of the object to the user by one of an audio device, a smartphone, or an intelligent computing device.

15. The computer system of claim 13, wherein formatting the output data for use with the at least one device further comprises:

transmitting the output data to the at least one device associated with the user, wherein the output data is formatted to create the spatial map of the physical environment using the electrical stimulation pad on the first side of the at least one device and the output data is formatted to create audio responses from one of an audio device, a smartphone and an intelligent computing device representing information on the object in the physical environment.