IMAGE TO SOUND CONVERSION DEVICE

• US 20090122161A1
• Filed: 11/08/2007
• Published: 05/14/2009
• Est. Priority Date: 11/08/2007
• Status: Abandoned Application

• Current Assignee: Technical Vision, Inc.
• Original Assignee: Technical Vision, Inc.
• Inventors: Bolkhovitinov, Igor
• Application Number:
  US11/936,797
• Publication Number:
  US 20090122161A1
• Time in Patent Office: Days
• US Class Current: 348/234
• CPC Class Codes: A61H 2201/165 Wearable interfaces
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: H04R 1/10 Earpieces; Attachments therefor ; Earphones; Monophonic headphones H04R1/28 takes precedence; stereophonic headphones H04R5/033
• CPC Class Codes: H04R 5/033 Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10

Abstract

A device for creating a sound map of a three dimensional view area is provided. The device comprises a first camera configured to capture and transmit a first image and a second camera positioned a predetermined distance from the first camera configured to capture and transmit a second image. An image processing system is connected to the first camera and the second camera and is configured to create a three dimensional topographic plan of the three dimensional view area based on a comparison of the first image with the second image and the predetermined distance between the first camera and the second camera. The image processing system is further configured to transform the three dimensional topographic plan into a sound map comprising volume gradients and tone gradients. The present invention further provides methods of creating a sound map of a three dimensional view area.

Claims

1. A device for creating a sound map of a three dimensional view area, the device comprising:

a first camera configured to capture and transmit a first image;

a second camera positioned a predetermined distance from the first camera configured to capture and transmit a second image; and

an image processing system connected to the first camera and the second camera configured to create a three dimensional topographic plan of the three dimensional view area based at least on a comparison of the first image with the second image and the predetermined distance between the first camera and the second camera, and configured to transform the three dimensional topographic plan into a sound map comprising volume gradients and tone gradients.

2. The device of claim 1, further comprising:

at least one frame member connected to the first camera and the second camera for attaching the device to a user; and

at least one audio output connected to the image processing system for transmitting the sound map to the user.

3. The device of claim 2, wherein the at least one frame member comprises a pair of spectacle frames for wearing on the head of a user, and the at least one audio output comprises a pair of speakers.

4. The device of claim 1, wherein the image processing system comprises:

at least one brightness determining engine connected to the first camera and the second camera configured to identify and quantify localized extreme points of the first image and the second image;

at least one parallax field forming engine connected to the at least one brightness determining engine configured to determine parallaxes between corresponding localized extreme points of the first image and the second image;

at least one block of cuts forming engine connected to the at least one parallax field forming engine configured to determine physical positions of portions of the three dimensional view area represented by the localized extreme points of the first image and the second image relative to at least one of the first camera and the second camera based on the determination of the parallaxes and the predetermined distance between the first camera and the second camera;

at least one topographical plan building engine configured to create the three dimensional topographic plan based at least on the physical positions of the portions of the three dimensional view area represented by the localized extreme points determined by the at least one block of cuts forming engine; and

at least one sound synthesizing engine connected to the topographical plan building engine configured to transform the three dimensional topographic plan into a sound map comprising volume gradients and tone gradients.

5. The device of claim 4, wherein the at least one topographical plan building engine comprises:

at least one brightness matrix forming engine connected to the at least one brightness determining engine, the at least one brightness matrix forming engine configured to create a brightness gradient matrix based on the relative positioning and the brightness magnitude of the localized extreme points;

at least one volume matrix forming engine connected to the at least one block of cuts forming engine and the at least one brightness matrix forming engine configured to create a volume gradient matrix based at least on the physical positions of the portions of the three dimensional view area represented by the localized extreme points and the relative brightness magnitude of the localized extreme points; and

at least one tone matrix forming engine connected to the at least one block of cuts forming engine configured to create a tone gradient matrix based at least on the physical positions of the portions of the three dimensional view area represented by the localized extreme points.

6. The device of claim 4, wherein the at least one topographical plan building engine comprises:

at least one brightness matrix forming engine connected to the at least one brightness determining engine, the at least one brightness matrix forming engine configured to create a brightness gradient matrix based on the relative positioning and the brightness magnitude of the localized extreme points;

a plurality of volume matrix forming engines connected to the at least one block of cuts forming engine and the at least one brightness matrix forming engine, the plurality of volume matrix forming engines configured to create a plurality of color-specific volume gradient matrices based at least on the physical positions of the portions of the three dimensional view area represented by the localized extreme points and the color-specific relative brightness magnitude of the localized extreme points; and

a plurality of tone matrix forming engines connected to the at least one block of cuts forming engine configured to create a plurality of color-specific tone gradient matrices based at least on the physical positions of the portions of the three dimensional view area represented by the localized extreme points;

wherein the at least one sound synthesizing engine is connected to the plurality of tone matrix forming engines and the plurality of volume matrix forming engines and is configured to transform the color-specific tone gradient matrices and the color-specific volume gradient matrices into a sound map comprising volume gradients and tone gradients.

7. The device of claim 1, wherein the image processing system comprises:

at least one brightness determining engine connected to the first camera and the second camera configured to at least one of create a plurality of coplanar pairs of line brightness functions from the first and second images and receive the first and second images as a plurality of coplanar pairs of line brightness functions, wherein the plurality of coplanar pairs of line brightness functions comprise a first plurality of line brightness functions from the first camera and a second plurality of line brightness functions from the second camera, wherein each of the first plurality of line brightness functions represents a cut of the three dimensional view area which is substantially coplanar with a cut of the three dimensional view area represented by at least one of the second plurality of line brightness functions, and wherein the at least one brightness determining engine is configured to identify localized extreme points of the line brightness functions;

at least one parallax field forming engine connected to the at least one brightness determining engine configured to determine parallaxes between corresponding ones of the localized extreme points of the coplanar pairs of brightness functions;

at least one block of cuts forming engine connected to the at least one parallax field forming engine configured to determine physical positions of the portions of the three dimensional view area represented by the localized extreme points relative to the first camera and the second camera based on the determination of the parallaxes and the predetermined distance between the first camera and the second camera;

at least one topographical plan building engine connected to the at least one block of cuts forming engine configured to create the three dimensional topographic plan based at least on the physical positions of the portions of the three dimensional view area represented by the localized extreme points determined by the at least one block of cuts forming engine; and

at least one sound synthesizing engine connected to the topographical plan building engine for transforming the three dimensional topographic plan into a sound map comprising volume gradients and tone gradients.

8. The device of claim 1, wherein the image processing system is configured to identify localized extreme points of the first image and the second image and to determine physical positions of the portions of the three dimensional view area represented by the localized extreme points relative to the first camera and the second camera to create the three dimensional topographic plan of the three dimensional view area.

9. A method of creating a sound map of a three dimensional view area comprising:

providing a first camera directed toward the three dimensional view area;

providing a second camera directed toward the three dimensional view area and positioned a predetermined distance from the first camera;

providing a processing system connected to the first camera and the second camera;

transmitting a first image of the three dimensional view area from the first camera to the processing system;

transmitting a second image of the three dimensional view area from the second camera to the processing system;

comparing the first image with the second image and creating a three dimensional topographic plan with the processing system based on the comparison of the first image and the second image and the predetermined distance between the first camera and the second camera; and

transforming using the processing system the three dimensional topographic plan into a sound map comprising volume gradients and tone gradients.

10. The method of claim 9, further comprising:

transmitting the first image and the second image to the processing system as a plurality of coplanar pairs of line brightness functions of the three dimensional view area, wherein each of the plurality of coplanar pairs of line brightness functions comprises a first line brightness function from the first camera and a second line brightness function from the second camera, wherein the first line brightness function represents a cut of the three dimensional view area which is substantially coplanar with a cut of the three dimensional view area represented by the second line brightness function;

identifying localized extreme points on the first line brightness functions and the second line brightness functions using the processing system;

determining parallaxes between corresponding ones of the localized extreme points of the plurality of coplanar pairs of line brightness functions using the processing system;

determining physical positions of the portions of the three dimensional view area represented by the localized extreme points relative to the first camera and the second camera based on the determination of the parallaxes and the predetermined distance between the first camera and the second camera using the processing system; and

creating the three dimensional topographic plan based at least on the physical positions of the portions of the three dimensional view area represented by the localized extreme points relative to the first camera and the second camera using the processing system.

11. The method of claim 10, further comprising using the processing system to normalize the determined physical positions of the portions of the three dimensional view area represented by the localized extreme points relative to the first camera and the second camera with respect to a ground plane.

12. The method of claim 10, further comprising using the processing system to triangulate the physical positions of the portions of the three dimensional view area represented by the localized extreme points relative to the first camera and the second camera.

13. The method of claim 10, further comprising using the processing system to determine physical positions of portions of the three dimensional view area represented by points between the localized extreme points in each of the plurality of coplanar pairs of line brightness functions by interpolating between the determined physical positions of portions of the three dimensional view area represented by the localized extreme points at a predetermined resolution to create an interpolation of the cuts of the three dimensional view area.

14. The method of claim 13, further comprising using the processing system to determine physical positions of portions of the three dimensional view area between the cuts of the three dimensional view area by interpolating between the determined interpolation of the cuts of the three dimensional view area at a predetermined resolution.

15. The method of claim 9, further comprising:

creating the three dimensional topographic plan with components of surface brightness and surface height using the processing system; and

modeling surface brightness as volume and modeling surface height as tone in transforming the three dimensional topographic plan into the sound map using the processing system.

16. The method of claim 9, further comprising providing at least two audio outputs connecting to the processing system and emitting the sound map stereophonically through the at least two audio outputs using the processing system.

17. A method for creating a sound map of a three dimensional view area comprising:

capturing a first image of the three dimensional view area including a surface from a first vantage point, whereby the first image comprises a first plurality of line brightness functions;

capturing a second image of the three dimensional view area including the surface from a second vantage point a predetermined distance from the first vantage point, whereby the second image comprises a second plurality of line brightness functions;

comparing the first plurality of line brightness functions with the second plurality of line brightness functions and creating a three dimensional topographic plan based at least on the comparison of the first and second plurality of line brightness functions and based on the predetermined distance between the first vantage point and the second vantage point; and

creating a sound map based on the three dimensional topographic plan, wherein the sound map comprises volume gradients and tone gradients.

18. The method of claim 17, wherein the creating the sound map comprises modeling brightness levels of at least one of the first image and the second image within a first frequency range, and modeling at least one color of the at least one of the first image and the second image within a second frequency range outside of the first frequency range.

19. The method of claim 17, wherein the creating the sound map comprises modeling a distance from at least one of the first vantage point and the second vantage point to the surface as a series of spaced sound pulses at a frequency of delivery of less than about 20 Hz, wherein the frequency of delivery of the sound pulses is dependent on the distance from the at least one of the first vantage point and the second vantage point to the surface.

20. The method of claim 17, wherein the comparing the first plurality of line brightness functions with the second plurality of brightness functions comprises:

determining matching points of coplanar pairs of the first plurality of line brightness functions and the second plurality of line brightness functions;

determining a plurality of parallaxes between the matching points of the coplanar pairs of the first plurality of line brightness functions and the second plurality of line brightness functions; and

determining physical positions of portions of the three dimensional view area represented by the matching points relative to at least one of the first vantage point and the second vantage point based on the determination of the plurality of parallaxes and the predetermined distance between the first vantage point and the second vantage point.