Enhancement and display methods for intuitively read three-dimensional visual representations of digital audio files

• US 9,704,276 B1
• Filed: 08/05/2014
• Issued: 07/11/2017
• Est. Priority Date: 08/05/2014
• Status: Active Grant

• Current Assignee: Randall C. Wilson
• Original Assignee: Randall C. Wilson
• Inventors: Wilson, Randall Charles
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US14/451,811
• Time in Patent Office: 1,071 Days
• Field of Search: 345419
• CPC Class Codes: G06F 2203/04802 3D-info-object: information is displayed on the internal or external surface of a three dimensional manipulable object, e.g. on the faces of a cube that can be rotated by the user
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/20108 Interactive selection of 2D slice in a 3D data set
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2220/005 Non-interactive screen display of musical or status data
• CPC Class Codes: G10H 2220/131 for abstract geometric visualisation of music, e.g. for interactive editing of musical parameters linked to abstract geometric figures
• CPC Class Codes: G10H 2250/235 Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT]
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

The proposed method creates a detailed, accurate, frequency and amplitude-based three-dimensional moving display of digital audio files. It adds timestamps, used later in the process to achieve accurate synchronization between a moving display and playback of the analyzed audio file. It details how the analyzed data is processed and enhanced to prominently show the most fundamental elements in the audio file. The method proposes different layouts for displays and ways of showing separate elements of the audio simultaneously. Upcoming audio is displayed in locations that allow viewers to anticipate and react to events about to happen. It introduces the temporal plane of playback for clearly showing the part of the moving display that corresponds to the exact part of the analyzed audio that is playing. The temporal plane of playback demonstrates the direct correlation with the audio and accurate synchronization between sound and picture.

Claims

1. A method for generating a three-dimensional visual representation of one or more digital audio files on a display including a time domain temporal aspect and a frequency domain with a temporal plane of playback that during audio video playback highlights the point in the image that corresponds to the audio that is currently playing as the image of the audio dynamically moves across said plane, three-dimensional images resulting from frequency analysis and enhancement processes are committed and stored for reproduction in a non-transitory medium:

a) receiving a digital audio file comprising tones;

b) decomposing said digital audio file in the temporal domain to produce and sequentially store temporal audio slices for frequency analysis;

c) generating timestamps for marking locations of said temporal audio slices in said digital audio file along a time axis;

d) analyzing said temporal audio slices in the frequency domain to produce audio frequency spectra of said temporal audio slices with frequency being shown along a frequency axis and amplitude being shown along an amplitude axis;

e) applying at least one visual enhancement function to said audio frequency spectra to produce visually enhanced audio frequency spectra;

f) organizing said temporal audio slices with said visually enhanced audio frequency spectra along said time axis based on said timestamps and stored sequentially to generate said three-dimensional visual representation of said digital audio file for later playback synchronized with said audio file;

g) displaying said three-dimensional visual representation from a predetermined viewpoint to a user while said digital audio file is being played back;

h) scrolling a visual playback indicator at a temporal plane of playback of said digital audio file along said time axis and over said three-dimensional visual representation;

wherein said three-dimensional visual representation is initially displayed in an orientation where said visual representation moves from the background forward across the temporal plane of playback.

2. The method of claim 1, wherein said step of applying said at least one visual enhancement function comprises applying said at least one visual enhancement function to at least one component of said spectra.

3. The method of claim 2, wherein said at least one visual enhancement function is related to at least one signal processing function.

4. The method of claim 3, wherein said at least one signal processing function is selected from the group of functions consisting of visual distortion and skewing as well as audio signal processes including filtering, equalization, Fourier transformation, Hartley transformation, spectral decomposition, and attenuation.

5. The method of claim 2, further comprising identifying said at least one component of said spectra with a source of at least one of said tones.

6. The method of claim 5, wherein said at least one visual enhancement function is applied to at least one component of said spectra identified with said source.

7. The method of claim 1 uses a computer to generate a three-dimensional image of said audio with process instructions and the resulting image stored in memory.

8. The method of claim 1, further comprising the step of changing said predetermined viewpoint.

9. The method of claim 8, wherein said step of changing said predetermined viewpoint is controlled by said user.

10. The method of claim 8, wherein said step of changing of said predetermined viewpoint comprises any change selected from three-dimensional translations and three-dimensional rotations of said predetermined viewpoint.

11. The method of claim 1, further comprising the step of altering at least one imaging characteristic of said three-dimensional visual representation from said predetermined viewpoint.

12. The method of claim 11, wherein said at least one imaging characteristic comprises zooming.

13. The method of claim 1, wherein said at least one visual enhancement function comprises the use of color.

14. The method of claim 1, wherein said at least one visual enhancement function comprises the use of texture.

15. The method of claim 1, wherein said at least one visual enhancement function comprises the use of lighting or shading.

16. The method of claim 1, wherein upcoming audio which is about to play can be displayed, allowing viewers;

including musicians, garners, artists, engineers, and others, to anticipate and react to events about to occur.