Device, method and system for instant real time neuro-compatible imaging of a signal

• US 9,466,316 B2
• Filed: 02/05/2015
• Issued: 10/11/2016
• Est. Priority Date: 02/06/2014
• Status: Active Grant

• Current Assignee: Analog Devices, Inc.
• Original Assignee: OtoSense, Inc. (Analog Devices, Inc.)
• Inventors: Christian, Sebastien J. V.
• Primary Examiner(s): Tweel, Jr., John A
• Application Number:
  US14/615,290
• Publication Number:
  US 20150221190A1
• Time in Patent Office: 614 Days
• Field of Search: 340/540, 846/26, 844/64.R, 846/62, 345/440.1, 367/7, 381/56
• US Class Current: 1/1
• CPC Class Codes: G08B 1/08 using electric transmission ; transformation of alarm signals to electrical signals from a different medium, e.g. transmission of an electric alarm signal upon detection of an audible alarm signal
• CPC Class Codes: G08B 17/10 Actuation by presence of smoke or gases , e.g. automatic alarm devices for analysing flowing fluid materials by the use of optical means
• CPC Class Codes: G08B 21/18 Status alarms G08B21/02 takes precedence
• CPC Class Codes: G10H 2210/301 Soundscape or sound field simulation, reproduction or control for musical purposes, e.g. surround or 3D sound; Granular synthesis
• CPC Class Codes: G10L 21/14 by displaying frequency domain information
• CPC Class Codes: G10L 25/27 characterised by the analysis technique
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

A method, apparatus and system for transforming a progressing sound signal into a progressing visual pattern, the progressing visual pattern being perceptible and recognizable as the progressing sound signal to a user in real time. The progressing visual pattern displays in real time a set of optical attributes, the set of optical attributes being transformations from a set of sound features that define the sound signal in real time. The sound features and optical attributes, along with changes in the sound features and optical attributes over time, are preselected to be isomorphic to sound, perceptible to human vision, efficiently processed by human cognition, and therefore to be recognizable to a human who has been exposed and actively or passively trained to it.

Claims

1. A method comprising:

transforming a progressing sound signal into a progressing visual pattern displayed on a surface over a time duration by executing, on a computing hardware component, an application implementing a plurality of steps, the steps comprising;

calculating from the progressing sound signal a set of preselected values, the set of preselected values being represented formulaically and/or numerically and characterizing a set of preselected sound features from which the progressing sound signal can be reconstructed to be recognizable; and

transforming the set of preselected sound features into a set of preselected optical attributes, wherein the set of preselected optical attributes defines the progressing visual pattern displayed on the surface, wherein transforming comprises mapping the set of preselected values into a coordinate system of the progressing visual pattern thereby defining numerical values for the set of optical attributes defining the progressing visual pattern; and

the computing hardware component causing display of the progressing visual pattern on the surface;

wherein the progressing visual pattern is displayed in a form that is isomorphic to sound, perceptible to human vision, can be processed by human cognition, and which is therefore recognizable to a human during the time duration.

2. The method of claim 1, wherein the set of preselected values transforms one of the set of preselected sound features into one of the set of preselected optical attributes, and wherein an equivalency between the one preselected sound feature and the one preselected optical attribute is displayed in a recognizable form to a user viewing the display of the progressing visual pattern on the surface.

3. The method of claim 1, wherein each preselected value in the set of preselected values transforms one of the set of preselected sound features into one of the set of preselected optical attributes, and wherein an equivalency between the one preselected sound feature and the one preselected optical attribute is recognizable to a user viewing the display of the progressing visual pattern on the surface.

4. The method of claim 1, wherein transforming the progressing sound signal into the progressing visual pattern for display occurs in real time.

5. The method of claim 1, wherein the progressing sound signal comprises an aperiodic portion.

6. The method of claim 1, wherein one feature of the set of preselected sound features is a volume quantity and a preselected value associated with the volume quantity is a user-perceptible loudness, calculated as a log of the averaged power spectrum of the sound signal over an interval of time.

7. The method of claim 1, wherein one feature of the set of preselected sound features is a pitch indication, and a preselected value associated with the pitch indication is a frequency level, calculated as the log of the average frequency spectrum of the sound signal over an interval of time.

8. The method of claim 1, wherein one feature of the set of preselected sound features is a rhythm indication, and a preselected value associated with the rhythm indication is an intensity variation quantity, calculated as a first derivative of the log of the averaged intensity of the sound signal over an interval of time.

9. The method of claim 1, wherein one attribute of the set of preselected optical attributes is a shape indication, and a preselected value associated with the shape indication defines a linear dimension.

10. The method of claim 1, wherein one attribute of the set of preselected optical attributes is a transparency indication, and a preselected value associated with the transparency indication defines a level of transparency.

11. The method of claim 1, wherein the surface comprises one or more surface of a plurality of surfaces selected from the group consisting of a flat panel display, a liquid crystal display, a diode array display, a display screen, a mobile phone surface, a personal computer surface, a surface of a helmet, a visor, a pair of glasses, an apparatus for engaging an infant, or any other type of display surface that can be oriented in view of a user.

12. The method of claim 1, wherein the computing hardware component comprises one or more display device of a plurality of display devices selected from the group consisting of a mobile phone, a personal computer, a helmet, a visor, a pair of glasses, a display screen, a display surface, and an apparatus for engaging an infant.

13. The method of claim 1, wherein the progressing visual pattern is displayed on the surface in real time.

14. The method of claim 1, wherein the progressing visual pattern is recognizable by a human who has been exposed and actively or passively trained to the progressing visual pattern.

15. A device comprising:

a surface displaying a progressing visual pattern transformed from a progressing sound signal over a time duration;

a computing hardware component configured in communication with the surface displaying the progressing visual pattern;

an application executing on the computing hardware component, transforming the progressing sound signal into the progressing visual pattern displaying on the surface;

wherein a set of preselected values is calculated by the executing application, the set of preselected values being represented formulaically and/or numerically and characterizing a set of preselected sound features from which the progressing sound signal can be reconstructed to be recognizable;

wherein the set of preselected sound features is transformed into a set of preselected optical attributes, wherein the transforming comprises mapping the set of preselected values into a coordinate system of the progressing visual pattern thereby defining numerical values for the set of optical attributes defining the progressing visual pattern;

wherein the set of preselected optical attributes defines the progressing visual pattern, and wherein the progressing visual pattern is displayed in a form that is isomorphic to sound, perceptible to human vision, can be processed by human cognition, and which is therefore recognizable to a human during the time duration.

16. The device of claim 15, wherein the set of preselected values transforms one of the set of preselected sound features into one of the set of preselected optical attributes, and wherein an equivalency between the one preselected sound feature and the one preselected optical attribute is recognizable to a user viewing the display of the progressing visual pattern on the surface.

17. The device of claim 15, wherein each preselected value in the set of preselected values transforms one of the set of preselected sound features into one of the set of preselected optical attributes, and wherein an equivalency between the one preselected sound feature and the one preselected optical attribute is recognizable to a user viewing the display of the progressing visual pattern on the surface.

18. The device of claim 15, wherein the transformation of the progressing sound signal into the progressing visual pattern occurs in real time.

19. The device of claim 15, wherein the progressing sound signal comprises an aperiodic portion.

20. The device of claim 15, wherein one feature of the set of preselected sound features is a volume quantity and a preselected value associated with the volume quantity is a user-perceptible loudness, calculated as a log of the averaged power spectrum of the sound signal over an interval of time.

21. The device of claim 15, wherein one feature of the set of preselected sound features is a pitch indication, and a preselected value associated with the pitch indication is a frequency level, calculated as the log of the average frequency spectrum of the sound signal over an interval of time.

22. The device of claim 15, wherein one feature of the set of preselected sound features is a rhythm indication, and a preselected value associated with the rhythm indication is an intensity variation quantity, calculated as a first derivative of the log of the averaged intensity of the sound signal over an interval of time.

23. The device of claim 15, wherein one attribute of the set of preselected optical attributes is a shape indication, and a preselected value associated with the shape indication defines a linear dimension.

24. The device of claim 15, wherein one attribute of the set of preselected optical attributes is a transparency indication, and a preselected value associated with the transparency indication defines a level of transparency.

25. The device of claim 15, wherein the surface comprises one or more surface of a plurality of surfaces selected from the group consisting of a flat panel display, a liquid crystal display, a diode array display, a display screen, a mobile phone surface, a personal computer surface, a surface of a helmet, a visor, a pair of glasses, an apparatus for engaging an infant, or any other type of display surface that can be oriented in view of a user.

26. The device of claim 15, wherein the computing hardware component comprises one or more device of a plurality of devices selected from the group consisting of a mobile phone, a personal computer, a helmet, a visor, a pair of glasses, a display screen, and an apparatus for engaging an infant.

27. The device of claim 15, wherein a visual representation of each one of the set of preselected optical attributes displayed on the surface is recognizable to a user as representing a preselected sound feature in a set of preselected sound features.

28. The device of claim 15, wherein the progressing visual pattern is displayed on the surface in real time.

29. The device of claim 15, wherein the device is a distributed device, the distributed device having components that may be separated by physical distances from each other.

30. The device of claim 15, wherein the progressing visual pattern is recognizable by a human who has been exposed and actively or passively trained to the progressing visual pattern.

31. A device comprising:

a display surface displaying a progressing visual pattern transformed from a progressing sound signal;

a computing hardware component;

an application executing on the computing hardware component;

wherein the computing hardware component is configured in communication with the surface;

wherein a set of pre-determined values are extracted by the application from the progressing sound signal, the set of pre-determined values being resented formulaically and/or numerically and characterizing a set of preselected sound features from which the progressing sound signal can be reconstructed to be recognizable;

wherein each one of the set of pre-determined values extracted from the progressing sound signal transforms one preselected sound feature of the set of preselected sound features into one pre-selected optical attribute of a set of pre-selected optical attributes, wherein the transforming comprises mapping the set of pre-determined values into a coordinate system of the progressing visual pattern thereby defining numerical values for the set of pre-selected optical attributes defining the progressing visual pattern;

wherein the set of pre-selected optical attributes define the progressing visual pattern displayed on the display surface; and

wherein the progressing visual pattern is displayed in a form that is isomorphic to sound, perceptible to human vision, can be processed by human cognition, and which is therefore recognizable to a human during the time duration.

32. The device of claim 31, wherein the progressing visual pattern is recognizable by a human who has been exposed and actively or passively trained to the progressing visual pattern.