PERSPECTIVE-BASED DYNAMIC AUDIO VOLUME ADJUSTMENT

• US 20180210697A1
• Filed: 01/24/2017
• Published: 07/26/2018
• Est. Priority Date: 01/24/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: RAKSHIT, Sarbajit K.
• Granted Patent: US 10,592,199 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/038 Cross-faders therefor
• CPC Class Codes: G11B 27/102 Programmed access in sequence to addressed parts of tracks of operating record carriers access by moving the head G11B3/08, G11B5/54, G11B7/085, G11B21/022; by moving the record carrier G11B15/005, G11B17/005, by driving of both record carrier and head G11B15/1816

Abstract

Perspective-based dynamic audio volume adjustment. A method obtains multimedia content, the multimedia content including video frames of a scene and audio content for playing in conjunction with display of the video frames. The audio content includes audio portions corresponding to objects in the scene. Based on a selection of a viewing perspective from which to view the scene, the method determines an audio mix for the audio portions given the selected viewing perspective. Determining the audio mix includes determining, for each object of the objects in the scene, a respective audio volume level for an audio portion, of the audio portions, corresponding to the object. The respective audio volume level is based at least in part on a position of the object relative to the selected viewing perspective.

Claims

1. A computer-implemented method comprising:

obtaining multimedia content, the multimedia content comprising a plurality of video frames of a scene and audio content for playing in conjunction with display of the plurality of video frames, the audio content comprising a plurality of audio portions corresponding to a plurality of objects in the scene;

based on a selection of a viewing perspective from which to view the scene, the selection made from a plurality of available viewing perspectives available from which to view the scene, determining an audio mix for the plurality of audio portions given the selected viewing perspective, the determining the audio mix comprising determining, for each object of the plurality of objects in the scene, a respective audio volume level for an audio portion, of the plurality of audio portions, corresponding to the object, the respective audio volume level based at least in part on a position of the object relative to the selected viewing perspective.

2. The method of claim 1, wherein the determining the audio mix further comprises determining positions of the plurality of objects relative to each other, and wherein the determining the audio volume level for the audio portion corresponding to an object of the plurality of objects is further based at least in part on the position of the object relative to one or more other objects of the plurality of objects of the scene.

3. The method of claim 1, wherein the multimedia content further comprises position information indicating positions of the plurality of objects, and wherein the determining the respective audio volume level for each object of the plurality of objects uses the position information.

4. The method of claim 1, wherein the determining the audio volume level for the audio portion corresponding to an object of the plurality of objects is further based at least in part on a number of sub-objects, of the object, generating the sound.

5. The method of claim 1, wherein the determining the audio volume level for the audio portion corresponding to an object of the plurality of objects is further based at least in part on a baseline loudness of the object.

6. The method of claim 1, wherein the selected viewing perspective comprises a selected zoomed-in perspective of the plurality of video frames, the zoomed-in perspective simulating a closer distance to one or more objects of the plurality of objects than a full-frame perspective of the plurality of video frames, wherein the audio volume level for the audio portions corresponding to the one or more objects is increased based on selecting the zoomed-in perspective.

7. The method of claim 6, wherein based on selecting the zoomed-in perspective, at least one object of the plurality of objects is not within view on a display on which the multimedia content is to be played, and wherein the determining the audio mix filters-out at least one audio portion, corresponding to the at least one object, of the plurality of audio portions, in which the audio volume level of the at least one audio portion is silenced or the at least one audio portion is omitted from the audio mix.

8. The method of claim 1, wherein the multimedia content comprises multiple-view video content providing multiple selectable viewing angles from which to view the scene, the multiple selectable viewing angles each being from a unique position relative to the plurality of objects of the scene and having a corresponding unique set of distances between the unique position and the plurality of objects of the scene, wherein the selected viewing perspective is a selected viewing angle of the multiple selectable viewing angles and the determined audio mix is based on the set of distances between the unique position of the selected viewing angle and the plurality of objects of the scene.

9. The method of claim 1, wherein positions of the plurality of objects relative to the selected viewing perspective vary across the plurality of video frames, and wherein determining the respective audio volume level for the audio portion corresponding to each object of the plurality of objects comprises determining varying volume levels corresponding to the varying positions of the object across the plurality of video frames.

10. The method of claim 1, further comprising playing the multimedia content with the determined audio mix, wherein each audio portion of the plurality of audio portions is played with the determined audio volume level for that portion.

11. A computer program product comprising:

a computer readable storage medium readable by a processor and storing instructions for execution by the processor for performing a method comprising;

obtaining multimedia content, the multimedia content comprising a plurality of video frames of a scene and audio content for playing in conjunction with display of the plurality of video frames, the audio content comprising a plurality of audio portions corresponding to a plurality of objects in the scene;

based on a selection of a viewing perspective from which to view the scene, the selection made from a plurality of available viewing perspectives available from which to view the scene, determining an audio mix for the plurality of audio portions given the selected viewing perspective, the determining the audio mix comprising determining, for each object of the plurality of objects in the scene, a respective audio volume level for an audio portion, of the plurality of audio portions, corresponding to the object, the respective audio volume level based at least in part on a position of the object relative to the selected viewing perspective.

12. The computer program product of claim 11, wherein the determining the audio mix further comprises determining positions of the plurality of objects relative to each other, and wherein the determining the audio volume level for the audio portion corresponding to an object of the plurality of objects is further based at least in part on the position of the object relative to one or more other objects of the plurality of objects of the scene.

13. The computer program product of claim 11, wherein the multimedia content further comprises position information indicating positions of the plurality of objects, and wherein the determining the respective audio volume level for each object of the plurality of objects uses the position information.

14. The computer program product of claim 11, wherein the selected viewing perspective comprises a selected zoomed-in perspective of the plurality of video frames, the zoomed-in perspective simulating a closer distance to one or more objects of the plurality of objects than a full-frame perspective of the plurality of video frames, wherein the audio volume level for the audio portions corresponding to the one or more objects is increased based on selecting the zoomed-in perspective.

15. The computer program product of claim 14, wherein based on selecting the zoomed-in perspective, at least one object of the plurality of objects is not within view on a display on which the multimedia content is to be played, and wherein the determining the audio mix filters-out at least one audio portion, corresponding to the at least one object, of the plurality of audio portions, in which the audio volume level of the at least one audio portion is silenced or the at least one audio portion is omitted from the audio mix.

16. The computer program product of claim 11, wherein the multimedia content comprises multiple-view video content providing multiple selectable viewing angles from which to view the scene, the multiple selectable viewing angles each being from a unique position relative to the plurality of objects of the scene and having a corresponding unique set of distances between the unique position and the plurality of objects of the scene, wherein the selected viewing perspective is a selected viewing angle of the multiple selectable viewing angles and the determined audio mix is based on the set of distances between the unique position of the selected viewing angle and the plurality of objects of the scene.

17. A computer system comprising:

a memory; and

a processor in communications with the memory, wherein the computer system is configured to perform a method, the method comprising;

obtaining multimedia content, the multimedia content comprising a plurality of video frames of a scene and audio content for playing in conjunction with display of the plurality of video frames, the audio content comprising a plurality of audio portions corresponding to a plurality of objects in the scene;

based on a selection of a viewing perspective from which to view the scene, the selection made from a plurality of available viewing perspectives available from which to view the scene, determining an audio mix for the plurality of audio portions given the selected viewing perspective, the determining the audio mix comprising determining, for each object of the plurality of objects in the scene, a respective audio volume level for an audio portion, of the plurality of audio portions, corresponding to the object, the respective audio volume level based at least in part on a position of the object relative to the selected viewing perspective.

18. The computer system of claim 17, wherein the determining the audio mix further comprises determining positions of the plurality of objects relative to each other, and wherein the determining the audio volume level for the audio portion corresponding to an object of the plurality of objects is further based at least in part on the position of the object relative to one or more other objects of the plurality of objects of the scene.

19. The computer system of claim 17, wherein the selected viewing perspective comprises a selected zoomed-in perspective of the plurality of video frames, the zoomed-in perspective simulating a closer distance to one or more objects of the plurality of objects than a full-frame perspective of the plurality of video frames, wherein the audio volume level for the audio portions corresponding to the one or more objects is increased based on selecting the zoomed-in perspective.

20. The computer system of claim 17, wherein the multimedia content comprises multiple-view video content providing multiple selectable viewing angles from which to view the scene, the multiple selectable viewing angles each being from a unique position relative to the plurality of objects of the scene and having a corresponding unique set of distances between the unique position and the plurality of objects of the scene, wherein the selected viewing perspective is a selected viewing angle of the multiple selectable viewing angles and the determined audio mix is based on the set of distances between the unique position of the selected viewing angle and the plurality of objects of the scene.