Method and device for visualizing images through sound

• US 6,963,656 B1
• Filed: 05/12/1999
• Issued: 11/08/2005
• Est. Priority Date: 05/12/1998
• Status: Expired due to Fees

• Current Assignee: University of Manchester
• Original Assignee: University of Manchester Institute of Science and Technology
• Inventors: Cronly-Dillon, John Ronald, Persaud, Krishna Chandra
• Primary Examiner(s): Miriam, Daniel
• Assistant Examiner(s): GORADIA, SHEFALI DINESH
• Application Number:
  US09/700,087
• Time in Patent Office: 2,372 Days
• Field of Search: 382/100, 382/190, 382/114, 340/825.19, 348/62, 348/70, 434/112, 434/116
• US Class Current: 382/100
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2220/131 for abstract geometric visualisation of music, e.g. for interactive editing of musical parameters linked to abstract geometric figures

Abstract

A method enabling a person to visualize images including encoding spatial information relating to a feature or features contained within an image into the form of one or more musical sequences, and playing the musical sequence or sequences to the person.

Claims

1. A method enabling a person to visualise images comprising the steps of:

encoding spatial information relating to at least one feature contained within an image into the form at least one polyphonic musical sequences;

playing the at least one polyphonic musical sequence to the person; and

,

wherein a subset of a full image is encoded into the at least one polyphonic musical, the step of encoding a subset of a full image comprising recognizing and extracting predetermined features from the image, and encoding said predetermined features into at least one polyphonic musical sequence.

2. A method according to claim 1 in which spatial information is encoded by selecting a note or chord dependent on the a distribution of the at least one feature along an axis.

3. A method according to claim 1 in which the image comprises a letter or a number.

4. A method according to claim 1 in which the image comprises the person'"'"'s environment.

5. A method according to claim 1 in which spatial information is encoded by:

representing the image as a 2D image;

forming one or more musical sequences, each comprising a series of notes or chords, in which i) each note or chord is selected dependent upon a distribution of the at least one feature along a portion of the 2D image, and ii) different notes or chords in a sequence correspond to different portions of the 2D image.

6. A method according to claim 5 in which the 2D image, or a portion of the 2D image, is divided into a matrix of pixels, and i) each note or chord is selected dependent upon the distribution of the at least one feature along a column of pixels and ii) one of either different notes and chords in a sequence correspond to the distribution of the at least one feature along different columns of pixels.

7. A method according to claim 6 in which a different note is associated with each pixel along a column and, if a feature recognised as being present in a pixel, the note corresponding to that pixel comprises part of the musical sequence.

8. A method according to claim 5 in which the 2D image, or a portion of the 2D image, is divided into a matrix of pixels, and i) each note or chord is selected dependent upon the distribution of the at least one feature along rows of pixels and ii) one of either different notes and chords in a sequence correspond to the distribution of the at least one feature along different rows of pixels.

9. A method according to claim 1 enabling a person to visualise moving features comprising the step of playing a plurality of musical sequences corresponding to different positions of the moving feature.

10. A method according to claim 1 in which a feature is simplified by encoding a portion of the feature as a musical sequence.

11. A method according to claim 10 spin which the feature is encoded by encoding different portions of the feature as different musical sequences.

12. A method according to claim 1 in which the image is encoded into the form of a plurality of musical sequences which are played to the person as a melody.

13. A method according to claim 12 in which the image is encoded as a plurality of musical sequences, each corresponding to different spatial resolutions.

14. A method according to claim 13 in which the image has a centre, and the image is divided into two or more concentric zones, the zone at the centre of the image being optionally encoded at a higher spatial resolution than the spatial resolution of the zone furthest from the centre of the image.

15. A method according to claim 14 in which a feature or features are visualised by obtaining a plurality of images in a saccharic-like series of movements, functionally equivalent to a scan path that an eye follows when it examines an object.

16. A method according to claim 13 in which the notes and chords in the sequence have a duration, and the spatial resolution corresponding to a musical sequence is indicated by the duration of the notes and chords in the sequence.

17. A method according to claim 1 in which a colour of the at least one feature is encoded by producing at least one musical sequence which comprises a plurality of different sets of waveforms mixed in variable ratios, wherein the different sets of waveforms correspond to different sounding instruments having different characteristic harmonic waveforms.

18. A method according to claim 17 in which three waveforms are mixed in variable ratios.

19. A method according to claim 18 in which the three waveforms are produced by filtering a master waveform between different frequency ranges.

20. A method according to claim 1 in which the at least one feature has a brightness and the at least one musical sequence has an intensity associated therewith, and wherein the brightness of the at least one feature is encoded by varying the intensity of the at least one musical sequence or sequences.

21. A method according to claim 1 in which image motion cues are translated into equivalent sound patterns which are utilised to segregate foreground features from background in 3D and produce information on relative depth between features.

22. A method according to claim 1 enabling a person to visualise moving features comprising the step of playing a plurality of musical sequences corresponding to different orientations of the moving feature.

23. A device enabling a person to visualise images comprising:

imaging means for obtaining images of at least one feature;

encoding means for encoding spatial information relating to the at least one into the form of at least one polyphonic musical sequences;

playing means for playing the at least one musical sequence to a person; and

,

wherein the encoding means comprises means for encoding a subset of a full image into the at least one polyphonic musical sequence by recognizing and extracting predetermined features from the image, and encoding said predetermined features into at least one polyphonic musical sequence.

24. A device according to claim 23 in which the imaging means comprises at least one video camera.

25. A device according to claim 23 in which the imaging means comprises at least one charge coupled detector.

26. A device according to claim 23 in which the encoding means comprises a microprocessor.

27. A device according to claim 23 in which the playing means comprises an ear-piece.

28. A device according to claim 23 which is portable in which the imaging means is hand-held.