Augmented auditory perception for the visually impaired
First Claim
Patent Images
1. A computer-implemented process for providing a person with the ability to auditorily determine the spatial geometry of his current physical environment, comprising:
- using a computer to perform the following process actions;
generating a depth map of the current physical environment of the person;
analyzing the depth map to identify objects and passageways therein;
analyzing the depth map to determine the current spatial coordinates, from the perspective of an environment sensing device being held by the person, of each of the identified objects and passageways;
using a first spatial orientation sensor within the sensing device to determine the current spatial orientation of the sensing device;
using a second spatial orientation sensor within a stereo listening device being worn on the head of the person to determine the current spatial orientation of the head of the person, wherein the second spatial orientation sensor is located at a distance from the first spatial orientation sensor;
comparing the current spatial orientation of the sensing device to the current spatial orientation of the head of the person to determine a relative current pose between the sensing device and head;
determining the current distance between the sensing device and head;
using said current pose and current distance to transform the current spatial coordinates of each of the identified objects and passageways to be from the perspective of the head of the person;
using the depth map to generate a spatialized audio representation of the current physical environment; and
outputting said spatialized audio representation to the stereo listening device.
2 Assignments
0 Petitions
Accused Products
Abstract
A person is provided with the ability to auditorily determine the spatial geometry of his current physical environment. A spatial map of the current physical environment of the person is generated. The spatial map is then used to generate a spatialized audio representation of the environment. The spatialized audio representation is then output to a stereo listening device which is being worn by the person.
-
Citations
17 Claims
-
1. A computer-implemented process for providing a person with the ability to auditorily determine the spatial geometry of his current physical environment, comprising:
-
using a computer to perform the following process actions; generating a depth map of the current physical environment of the person; analyzing the depth map to identify objects and passageways therein; analyzing the depth map to determine the current spatial coordinates, from the perspective of an environment sensing device being held by the person, of each of the identified objects and passageways; using a first spatial orientation sensor within the sensing device to determine the current spatial orientation of the sensing device; using a second spatial orientation sensor within a stereo listening device being worn on the head of the person to determine the current spatial orientation of the head of the person, wherein the second spatial orientation sensor is located at a distance from the first spatial orientation sensor; comparing the current spatial orientation of the sensing device to the current spatial orientation of the head of the person to determine a relative current pose between the sensing device and head; determining the current distance between the sensing device and head; using said current pose and current distance to transform the current spatial coordinates of each of the identified objects and passageways to be from the perspective of the head of the person; using the depth map to generate a spatialized audio representation of the current physical environment; and outputting said spatialized audio representation to the stereo listening device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 16, 17)
-
-
13. A computer-implemented process for providing a person with the ability to auditorily determine the spatial geometry of his current physical environment, comprising:
-
using a computer to perform the following process actions; using an infrared projector to project a known infrared pattern into the current physical environment of the person; using an infrared camera that is matched to the infrared projector to capture an ongoing series of images of a reflected infrared pattern being reflected off of objects in said environment; generating a depth map of said environment by comparing the known infrared pattern to the reflected infrared pattern in each captured image; analyzing the depth map to identify objects and passageways therein; analyzing the depth map to determine the current spatial coordinates, from the perspective of an environment sensing device being held by the person, of each of the identified objects and passageways; using a first spatial orientation sensor within the sensing device to determine the current spatial orientation of the sensing device; using a second spatial orientation sensor within a stereo listening device being worn by the person to determine the current spatial orientation of the head of the person, wherein the second spatial orientation sensor is located at a distance from the first spatial orientation sensor; comparing the current spatial orientation of the sensing device to the current spatial orientation of the head of the person to determine a relative current pose between the sensing device and head; determining the current distance between the sensing device and head; using said current pose and current distance to transform the current spatial coordinates of each of the identified objects and passageways to be from the perspective of the head of the person; using a head-related transfer function for the person to generate a spatialized audio representation of said environment by spatializing a different virtual sound source at the transformed current spatial coordinates of each of the identified objects and passageways, wherein each sound source generates an identifiable sound; and outputting said spatialized audio representation to the listening device. - View Dependent Claims (14)
-
Specification