Song transition effects for browsing

• US 9,326,082 B2
• Filed: 12/15/2011
• Issued: 04/26/2016
• Est. Priority Date: 12/30/2010
• Status: Active Grant

• Current Assignee: Dolby International AB (Dolby Laboratories Incorporated)
• Original Assignee: Dolby International AB (Dolby Laboratories Incorporated)
• Inventors: Engdegard, Jonas
• Primary Examiner(s): Shah, Paras D
• Assistant Examiner(s): Tzeng, Forrest F
• Application Number:
  US13/995,780
• Publication Number:
  US 20130282388A1
• Time in Patent Office: 1,594 Days
• Field of Search: 381/119
• US Class Current: 1/1
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2210/125 Medley, i.e. linking parts of different musical pieces in one single piece, e.g. sound collage, DJ mix
• CPC Class Codes: G10H 2210/131 Morphing, i.e. transformation of a musical piece into a new different one, e.g. remix
• CPC Class Codes: G10H 2210/136 Morphing interpolation, i.e. interpolating in pitch, harmony or time, tempo or rhythm, between two different musical pieces, e.g. to produce a new musical work
• CPC Class Codes: G11B 27/038 Cross-faders therefor
• CPC Class Codes: H04H 60/04 Studio equipment; Interconnection of studios
• CPC Class Codes: H04S 1/00 Two-channel systems H04S5/00, H04S7/00 take precedence
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field

Abstract

In one aspect, a method of providing directive transitions between audio signals comprises associating a first/second browsing direction (A1, A2) with a first/second transition effect template. In response to a browsing action in one of said browsing directions, a transition is played in which an exit segment (S0-out1, S0-out2) and an entry segment (S1-in, S2-in) are mixed in accordance with the associated transition effect template. A further aspect proposes a method of transitioning between audio signals decoded from audio data including time markers encoded as audio metadata and indicating at least one section of the respective audio signal. The method includes retrieving time marking information in the audio data; extracting an exit segment (S0-out1, S0-out2) and an entry segment (S1-in, S2-in), wherein an endpoint of at least one of the segments is synchronized with a time marker; and playing a transition in which the exit and entry segments are mixed in accordance with a transition effect template.

Claims

1. A method, performed by a device, for providing audio transitions between audio signals during audio browsing, comprising the steps of:

the device associating a first browsing direction (A1), selected by a user via a control means, for transitioning from a current audio signal (S0) to a first alternative audio signal (S1) with a first transition effect template;

the device associating a second browsing direction (A2), selected by a user via a control means, which is different from the first browsing direction, for transitioning from the current audio signal to a second alternative audio signal (S2), with a second transition effect template, which is perceptually different from the first transition effect template, wherein the effect is selected from the group comprising simulated motion, simulated Dopper effect, stereo regeneration, spectral band replication, and reverberation; and

the device playing, in response to a browsing action in one of said browsing directions, a transition in which an exit segment (S0-out1, S0-out2), extracted from the current audio signal, and an entry segment (S1-in, S2-in), extracted from the alternative audio signal, are mixed in accordance with the associated transition effect template; and

the device subsequently playing the alternative audio signal from the end of the entry segment.

2. The method of claim 1, wherein:

said steps of playing includes reproducing the entry and exit segments stereophonically using at least two channels;

the first transition effect template includes using a first exit channel power distribution for the exit segment (S0-out1) and a first entry channel power distribution for the entry segment (S1-in);

the second transition effect template includes using a second exit channel power distribution for the exit segment (S0-out2) and a second exit channel distribution for the entry segment (S2-in);

and at least one of the following holds;

(a) the exit channel power distributions differ by a permutation of at least two channels;

(b) the entry channel power distributions differ by a permutation of at least two channels.

3. The method of claim 1, wherein:

said steps of playing include reproducing the entry and exit segments stereophonically using at least two channels;

the first transition effect template is obtainable by moving an audio source reproducing one of the segments in a first spatial direction relative to a listener;

the second transition effect template is obtainable by moving an audio source reproducing one of the segments in a second spatial direction relative to a listener; and

the first and second directions are distinct.

4. The method of claim 1, wherein at least one transition effect template is obtainable by spatially moving an audio source reproducing said entry segment and/or an audio source reproducing said exit segment relative to a listener.

5. The method of claim 1, wherein:

said step of playing includes reproducing the entry and exit segments stereophonically using at least two channels; and

said transition effect template includes using an exit channel power distribution for the exit segment and using an entry channel power distribution, different from the exit channel power distribution, for the entry segment.

6. The method of claim 1, wherein said transition effect template includes mixing the entry and exit segments with a previously obtained audio segment.

7. The method of claim 1, further comprising

obtaining characteristics of a playback configuration, such as a distance between audio sources reproducing different channels, and/or obtaining characteristics of at least one of the audio signals, such as tempo, beatiness and beat strength,

and adapting said transition effect templates accordingly.

8. A non-transitory computer-readable medium storing computer-executable instructions for performing the method set forth in claim 1.

9. A decoder for outputting audio signals by decoding audio entries in an ordered database permitting browsing in at least a first and a second browsing direction (A1, A2), wherein:

the first and second browsing directions are different, and are respectively associated with perceptually distinct first and second transition effect templates, wherein the effect is selected from the group comprising simulated motion, simulated Doppler effect, stereo regeneration, spectral band replication, and reverberation; and

the decoder comprises one or more processors configured to react to a browsing action, initiated by a user via a control means, in one of said browsing directions by;

initially outputting, from one or more of the processors, a transition signal segment comprising an exit segment (S0-out1, S0-out2), decoded from a current audio entry, and an entry segment (S1-in, S2-in), decoded from an alternative audio entry located in the concerned browsing direction in relation to the current audio entry, mixed in accordance with the associated transition effect template; and

subsequently outputting, from one or more of the processors, an alternative signal decoded from the alternative audio entry.

10. An audio player comprising:

an ordered database presenting audio entries and permitting browsing in at least a first and a second browsing direction (A1, A2) wherein the first and second browsing directions are different and are respectively associated with perceptually distinct first and second transition effect templates;

an audio source for reproducing an audio signal; and

the decoder of claim 9, configured to decode entries from the database and to output resulting audio signals to the audio source.

11. A method, performed by a device, for providing a transition during audio browsing, between a current and an alternative audio signal decoded from audio data, wherein perceptually distinct transition effect templates are used for different browsing directions, and

wherein the audio data include time markers pre-determined via an encoding means, and encoded as audio metadata and indicating at least one section of the respective audio signal, wherein the at least one section is representative of the respective audio signal and may correspond to any portion of the respective audio signal;

the method comprising the steps of;

the device retrieving time marking information in the audio data;

the device extracting an exit segment (S0-out1) from the current audio signal and an entry segment (S1-in) from the alternative audio signal, wherein an endpoint of at least one of the segments is synchronized with a time marker;

the device playing, in response to a browsing direction selected by a user via a control means, a transition in which the exit segment and the entry segment are mixed in accordance with a transition effect template, wherein the effect is selected from the group comprising simulated motion, simulated Doppler effect, stereo regeneration, spectral band replication, and reverberation; and

the device subsequently playing the alternative audio signal from the end of the entry segment.

12. The method of claim 11, wherein the endpoint is related to a time marker in one of the following ways:

the endpoint coincides with a time marker;

the time marker refers to a beginning of a segment and the endpoint is located at a time interval before the time marker;

the time marker refers to an end of a section and the endpoint is located at a time interval after the time marker.

13. The method of claim 11, wherein the time markers are endpoints of representative segments extracted by:

assigning a plurality of ranking scores to a plurality of candidate representative segments, each individual candidate representative segment comprising at least one scene in one or more statistical patterns in media features of the audio data based on one or more types of features extractable from the audio data, each individual ranking score in the plurality of ranking scores being assigned to an individual candidate representative segment; and

selecting from the candidate representative segments, based on said plurality of ranking scores, a representative segment.

14. The method of claim 13, wherein each individual ranking score in said plurality of ranking scores comprises at least one component score based on one or more of:

duration, a measure for overlapping between different candidate representative segments, time-wise positions of candidate representative segments in the media data, chroma distance, MFCC, spectral contrast, spectral centroid, spectral bandwidth, spectral roll-off, spectral flatness, presence of singing voice, absence of singing voice, one or more rhythm patterns, energy, one or more stereo parameters, perceptual entropy, co-modulation, dynamics.

15. The method of claim 11, wherein the time markers are encoded in a header section of the audio data.

16. The method of claim 11, wherein at least one transition effect template is obtainable by spatially moving an audio source reproducing said entry segment and/or an audio source reproducing said exit segment relative to a listener.

17. The method of claim 11, wherein:

said step of playing includes reproducing the entry and exit segments stereophonically using at least two channels; and

said transition effect template includes using an exit channel power distribution for the exit segment and using an entry channel power distribution, different from the exit channel power distribution, for the entry segment.

18. The method of claim 11, wherein said transition effect template includes mixing the entry and exit segments with a previously obtained audio segment.

19. The method of claim 11, further comprising

obtaining characteristics of a playback configuration, such as a distance between audio sources reproducing different channels, obtaining characteristics of at least one of the audio signals, such as tempo, beatiness and beat strength,

and adapting said transition effect templates accordingly.

20. A non-transitory computer-readable medium storing computer-executable instructions for performing the method set forth in claim 11.

21. A decoder for outputting audio signals by decoding audio data, comprising:

a time marker extractor for retrieving time markers indicating at least one section of an audio signal, wherein the at least one section is representative of the respective audio signal and may correspond to any portion of the respective audio signal;

a segment extractor for extracting an exit segment (S0-out1) from a current audio signal and an entry segment (S1-in) from an alternative audio signal, wherein segment extractor is configured to synchronize an endpoint of at least one of the segments with a time marker;

a decoding unit comprising one or more processors configured

i) to play, in response to a browsing direction selected by a user via a control means, a transition, in which the exit segment and the entry segment are mixed in accordance with a transition effect template wherein perceptually distinct transition effect templates are used for different browsing directions, wherein the effect is selected from the group comprising simulated motion, simulated Doppler effect, stereo regeneration, spectral band replication, and reverberation, and subsequently

ii) to play the alternative audio signal from the end of the entry segment.

22. An audio player comprising:

an ordered database for storing audio data including time markers encoded as audio metadata and indicating at least one section of the respective audio signal, wherein the at least one section is representative of the respective audio signal and may correspond to any portion of the respective audio signal;

an audio source for reproducing an audio signal; and

the decoder of claim 21, configured to decode entries from the database and to output resulting audio signals to the audio source.