Intelligent glasses for the visually impaired

• US 9,488,833 B2
• Filed: 02/07/2014
• Issued: 11/08/2016
• Est. Priority Date: 02/07/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cervantes, Humberto Orozco
• Primary Examiner(s): Vo, Tung
• Application Number:
  US14/175,732
• Publication Number:
  US 20150227778A1
• Time in Patent Office: 1,005 Days
• Field of Search: 348/47
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted
• CPC Class Codes: G09B 5/125 the stations being mobile
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices

Abstract

An embodiment of the present invention is a method for communicating navigation information on a physical environment to a user. The method comprises receiving, by one or more computer processors, at least one digital image of a physical environment of a user that is captured by one or more digital video devices and converting the at least one digital image into a three-dimensional image. Furthermore, the method includes analyzing the three-dimensional image using object analysis to generate output data where the output data corresponds to the physical environment. Additionally, the method includes determining a device associated with the user and formatting the output data for use with the device.

Claims

1. A method for communicating navigation information on a physical environment to a user, the method comprising:

receiving, by a server computer, at least one digital image of a physical environment of the user captured by one or more digital video devices;

converting, by the server computer, the at least one digital image into a three-dimensional image;

analyzing, by the server computer, the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment;

determining, by the server computer, at least one device associated with the user;

formatting, by the server computer, the output data for use with the at least one device, wherein the least one device is capable of providing to a user both a spatial map of the physical environment created by an electrical stimulation pad and receiving from a user a tactile user input corresponding to an object represented in the spatial map of the physical environment to request an identification of an object; and

receiving, by the server computer, user inputs requesting an estimated distance to the object using gaze focal point, to create a tactile representation of the physical environment using a matrix of electrical stimulation electrodes providing various levels of intensity of electrical stimulation to represent a terrain, a distance to an object, and motion of an object in the physical environment, to provide the identification of the object based, at least in part, on the received user input on the surface of the electrical stimulation pad, and to provide a series of beeps representing a distance to the object, a type of object, and a motion of the object in the physical environment.

2. The method of claim 1, wherein formatting the output data for use with the at least one device, further comprises:

receiving, by the server computer, from the at least one device, information for a touch on an area of the electrical stimulation pad corresponding to the object in the spatial map of the physical environment to receive the request for the identification of the object; and

retrieving, by the server computer, a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object.

3. The method of claim 1, wherein analyzing, by the server computer, the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment, further comprises:

determining, by the server computer, a distance to an object, based, at least in part, on a focal point of the user'"'"'s gaze; and

determining, by the server computer, output data as a series of beeps that correspond to the distance to the object, based, at least in part, on the distance to the object wherein a tone of the beeps provides information on a type of object and information on a motion of the object.

4. The method of claim 1, wherein formatting, by the server computer, the output data for use with the device further comprises:

transmitting, by a computer device, the output data to the at least one device associated with the user;

wherein the output data is formatted to create the spatial map on the electrical stimulation pad that depicts the physical environment, the output data is formatted to create verbal responses from one of a smartphone and an intelligent computing device, and the output data is formatted for a series of beeps representing information on an object in the physical environment from one of a smartphone and an intelligent computing device.

5. The method of claim 1, wherein the at least one device associated with the user includes at least one of:

an electrical stimulation pad with a matrix of embedded electrical stimulation electrodes that provide varying levels of an electrical stimulation representing the spatial map of the physical environment, an audio device including one of a smart phone and an intelligent computing device, and an electrical stimulation pad that includes a touch pad accessible to the user to request information on an object corresponding to the matrix of embedded electrodes.

6. The method of claim 1, wherein formatting, by the server computer, the output data for use with the device further comprises:

determining, by a computer device, a level of intensity of the electrical stimulation associated with the physical environment, the level corresponding to a motion of an object and a terrain in the physical environment.

7. The method of claim 1, further comprising:

receiving, by the server computer, simultaneously, a tactile user input corresponding to the object in the spatial map of the physical environment provided on the at least one device using a matrix of electrical stimulation electrodes with a user accessible touch pad and a user input provided on one of an intelligent computing device and a smartphone using a user'"'"'s gaze focal point.

8. A computer program product for communicating navigation information on a physical environment to a user, the computer program product comprising:

one or more non-transitory computer readable storage devices and program instructions stored on the one or more non-transitory computer readable storage devices when executed by a server computer perform a method comprising;

receiving at least one digital image of a physical environment of the user captured by one or more digital video devices;

converting the at least one digital image into a three-dimensional image;

analyzing the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment;

determine determining at least one device associated with the user;

formatting the output data for use with the at least one device, wherein the least one device is capable of providing to a user both a spatial map of the physical environment created by an electrical stimulation pad, and receiving from a user a tactile user input corresponding to an object represented in the spatial map of the physical environment to request an identification of an object; and

receiving user inputs requesting an estimated distance to the object using gaze focal point, to create a tactile representation of the physical environment using a matrix of electrical stimulation electrodes providing various levels of intensity of electrical stimulation to represent a terrain, a distance to an object, and motion of an object in the physical environment, to provide the identification of the object based, at least in part, on the received user input on the surface of the electrical stimulation pad, and to provide a series of beeps representing a distance to the object, a type of object, and a motion of the object in the physical environment.

9. The computer program product of claim 8, wherein receiving from a user a tactile user input corresponding to an object represented in the spatial map, further comprises:

receiving a touch on an area of the electrical stimulation pad corresponding to the object in the spatial map of the physical environment to receive the request for the identification of the object; and

retrieving a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object.

10. The computer program product of claim 8, wherein analyzing the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment, further comprises:

determining a distance to an object, based, at least in part, on a focal point of the user'"'"'s gaze; and

determining output data as a series of beeps that correspond to the distance to the object, based, at least in part, on the distance to the object wherein a tone of the beeps provides information on a type of object and information on a motion of the object.

11. The computer program product of claim 8, wherein formatting the output data for use with the device further comprises:

transmitting the output data to the at least one device associated with the user;

wherein the output data is formatted to create the spatial map on the electrical stimulation device that depicts the physical environment, the output data is formatted to create verbal responses from one of a smartphone and an intelligent computing device, and the output data is formatted for a series of beeps representing information on an object in the physical environment from one of a smartphone and an intelligent computing device.

12. The computer program product of claim 8, wherein the at least one device associated with the user includes at least one of:

an electrical stimulation pad with a matrix of embedded electrical stimulation electrodes that provide varying levels of an electrical stimulation representing the spatial map of the physical environment, an audio device including one of a smart phone and an intelligent computing device, and an electrical stimulation pad that includes a touch pad accessible to the user to request information on an object corresponding to the matrix of embedded electrodes.

13. The computer program product of claim 8, wherein formatting the output data for use with the device further comprises:

determining a level of intensity of the electrical stimulation associated with the physical environment, the level corresponding to a motion of an object and a terrain in the physical environment.

14. A computer system for communicating navigation information on a physical environment to a user, the system comprising:

a server computer;

one or more computer non-transitory readable storage media;

program instructions stored on at least one of the one or more non-transitory computer readable storage media when executed by the server computer perform a method comprising;

receiving at least one digital image of a physical environment of the user captured by one or more digital video devices;

converting the at least one digital image into a three-dimensional image;

analyzing the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment;

determining at least one device associated with the user;

formatting the output data for use with the at least one device, wherein the least one device is capable of providing to a user both a spatial map of the physical environment created by an electrical stimulation pad, and receiving from a user a tactile user input corresponding to an object represented in the spatial map of the physical environment to request an identification of an object; and

receiving user inputs requesting an estimated distance to the object using gaze focal point, to create a tactile representation of the physical environment using a matrix of electrical stimulation electrodes providing various levels of intensity of electrical stimulation to represent a terrain, a distance to an object, and motion of an object in the physical environment, to provide the identification of the object based, at least in part, on the received user input on the surface of the electrical stimulation pad, and to provide a series of beeps representing a distance to the object, a type of object, and a motion of the object in the physical environment.

15. The computer system of claim 14, wherein receiving from a user a tactile user input corresponding to an object represented in the spatial map, further comprises:

receiving a request to identify a touch on an area of the electrical stimulation pad corresponding to the object in the spatial map of the physical environment to receive the request for the identification of the object; and

retrieving a name of the object in the physical environment from a database, the database containing the at least one digital image associated with the name of the object.

16. The computer system of claim 14, wherein analyzing the three-dimensional image using object analysis to generate output data, wherein the output data corresponds to the physical environment, further comprises:

determining a distance to an object, based, at least in part, on a focal point of the user'"'"'s gaze; and

determining output data as a series of beeps that correspond to the distance to the object, based, at least in part, on the distance to the object wherein a tone of the beeps provides information on a type of object and information on a motion of the object.

17. The computer system of claim 14, wherein formatting the output data for use with the device further comprises:

transmitting the output data to the at least one device associated with the user;

wherein the output data is formatted to create a spatial map on the electrical stimulation device that depicts the physical environment, the output data is formatted to create verbal responses from one of a smartphone and an intelligent computing device, and the output data is formatted for a series of beeps representing information on an object in the physical environment from one of a smartphone and an intelligent computing device.

18. The computer system of claim 14, wherein the at least one device associated with the user includes at least one of:

an electrical stimulation pad with a matrix of embedded electrical stimulation electrodes that provide varying levels of an electrical stimulation representing the spatial map of the physical environment, an audio device including one of a smart phone and an intelligent computing device, and an electrical stimulation pad that includes a touch pad accessible to the user to request information on an object corresponding to the matrix of embedded electrodes.

19. The computer system of claim 14, wherein formatting the output data for use with the device further comprises:

determining a level of intensity of the electrical stimulation associated with the physical environment, the level corresponding to a motion of an object and a terrain in the physical environment.