Audio information transforming method, video/audio format, encoder, audio information transforming program, and audio information transforming device

• US 20040119889A1
• Filed: 10/22/2003
• Published: 06/24/2004
• Est. Priority Date: 10/29/2002
• Status: Active Grant

• Current Assignee: Socionext Inc.
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Ogata, Satoshi
• Granted Patent: US 7,480,386 B2
• Time in Patent Office: Days
• US Class Current: 348/515
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence

Abstract

The present invention provides an audio information transforming method, a program product, a device, an encoder, and an video/audio format utilized therein, which are capable of providing an audio information by adjusting the Doppler effect caused by movement of the object, in response to change of the listening point. In the invention, a virtual listening point is set at a position different from a basic listening point where a listener listens to a sound of an object, then a velocity of the object observed from the virtual listening point is calculated based on position information of the virtual listening point and position information of the object. Then, an audio frequency of an audio heard at the virtual listening point is changed based on the calculated velocity. For example, the frequency of the sound is increased if the object approaches the virtual listening point, and the frequency is decreased if opposite.

Claims

1. An audio information transforming method applied to a video/audio format in which a screen includes a plurality of objects and each object has video information, position information, and audio information, said method comprising the steps of:

virtual listening point setting of setting a virtual listening point at a position different from a basic listening point that is set as a position at which a listener listens to an audio;

relative velocity calculating of calculating a relative velocity between the virtual listening point and the object; and

audio frequency transforming of executing an audio frequency transformation based on the relative velocity to add a Doppler effect to the audio information at the virtual listening point.

2. The audio information transforming method according to claim 1,

wherein the relative velocity calculating step calculates the relative velocity between the virtual listening point and the object by calculating velocity information of the object based on position information of the object before and after a predetermined time has lapsed.

3. The audio information transforming method according to claim 1,

wherein the relative velocity calculating step calculates the relative velocity by extracting velocity information of the object and then comparing the position information and the velocity information of the object and position information of the virtual listening point.

4. The audio information transforming method according to claim 1,

wherein the relative velocity calculating step calculates the relative velocity between the virtual listening point and the object by calculating velocity information of the virtual listening point based on position information of the virtual listening point before and after a predetermined time has lapsed.

5. The audio information transforming method according to claim 1,

wherein the relative velocity calculating step calculates the relative velocity by extracting velocity information of the virtual listening point and then comparing position information and the velocity information of the virtual listening point and the position information of the object.

7. The audio information transforming method according to claim 1,

wherein, when the audio information including the Doppler effect previously is included in the object, the audio frequency transforming step executes an audio frequency transformation to cancel the Doppler effect included in the audio information of the object, and executes the audio frequency transformation based on the relative velocity to add the Doppler effect to the audio information of the virtual listening point.

8. The audio information transforming method according to claim 1,

wherein, in respect to a final image unit, the audio frequency transforming step is executed by adding the Doppler effect to the audio information at the virtual listening point by using a formula by which the audio frequency transformation of the audio information at the virtual listening point prior to the final image by one image unit is executed.

9. The audio information transforming method according to claim 1 or 6,

wherein the video/audio format includes reduced scale information of the screen every scene.

10. A video/audio format utilized in the method set forth in any one of claims 1 to 9, said format comprising at least one of:

velocity information of an object, said object is one of objects included on a screen;

velocity information and direction information of a scene which is replayed on the screen; and

reduced scale information of the screen every scene.

11. An encoder utilized in the method set forth in any one of claims 1 to 9, said encoder for encoding at leas one of:

velocity information of an object, which is one of objects included in a screen;

velocity information and direction information of a scene; and

reduced scale information of the screen every scene.

6. An audio information transforming method applied to a video/audio format in which each scene that is replayed on a screen has video information and audio information, and the scene has velocity information and direction information based on which a background is moved, said method comprising the steps of:

virtual listening point setting step of setting a virtual listening point at a position different from -a basic listening point that is set as a position at which a listener listens to an audio;

relative velocity calculating step of calculating a relative velocity between the virtual listening point and a background based on the velocity information and the direction information of the background; and

audio frequency transforming step of transforming an audio frequency based on the relative velocity to add a Doppler effect to the audio information at the virtual listening point.

12. A program product for transforming audio information and for causing a computer to execute the procedures of;

setting a virtual listening point at a position different from a basic listening point that is set as a position at which a listener listens to an audio;

calculating a relative velocity between the virtual listening point and the object; and

executing an audio frequency transformation based on the relative velocity to add a Doppler effect to the audio information at the virtual listening point.

13. The program product according to claim 12,

wherein the procedure of calculating the relative velocity includes a procedure of calculating velocity information of the object based on position information of the object before and after a predetermined time has lapsed.

14. The program product according to claim 12,

wherein the procedure of calculating the relative velocity includes the procedures of:

extracting velocity information of the object; and

comparing the position information and the velocity information of the object and position information of the virtual listening point.

15. The program product according to claim 12,

wherein the procedure of calculating the relative velocity includes a procedure of calculating velocity information of the virtual listening point based on position information of the virtual listening point before and after a predetermined time has lapsed.

16. The program product according to claim 12,

wherein the procedure of calculating the relative velocity includes the procedures of:

calculating the relative velocity by extracting velocity information of the virtual listening point; and

comparing position information and the velocity information of the virtual listening point and the position information of the object.

18. The program product according to any one of claims 12 or 17,

wherein, when the object previously includes audio information having Doppler effect, the procedure of executing an audio frequency transformation includes the procedures of:

executing an audio frequency transformation to cancel the Doppler effect included in the audio information of the object; and

executing the audio frequency transformation based on the relative velocity to add the Doppler effect to the audio information of the virtual listening point.

19. The audio information transforming program according to any one of claims 12 or 17,

wherein, when audio information transformation at a time of final image unit is executed, said program product further comprising a procedure of:

adding the Doppler effect to the audio information at the virtual listening point by using a formula, said formula for executing the audio frequency transformation of the audio information at the virtual listening point prior to the final image by one image unit.

20. The audio information transforming program according to claim 12 or 17,

wherein the video/audio format includes reduced scale information of the screen every scene.

17. A program product for transforming audio information and for causing a computer to execute the procedures of;

setting a virtual listening point at a position different from a basic listening point that is set as a position at which a listener listens to an audio;

calculating a relative velocity between the virtual listening point and a background according to a velocity and a direction based on which the background of a scene is moved; and

executing an audio frequency transformation based on the relative velocity to add a Doppler effect to the audio information at the virtual listening point.

21. An audio information transforming device for a video/audio format in which a screen includes a plurality of objects and each object has video information, position information, and audio information, said device comprising:

virtual listening point setting section for setting a virtual listening point at a position different from a basic listening point that is set as a position at which a listener listens to an audio;

relative velocity calculating section for calculating a relative velocity between the virtual listening point and the object; and

an audio frequency transforming section for executing an audio frequency transformation based on the relative velocity to add a Doppler effect to the audio information at the virtual listening point.

22. The audio information transforming device according to claim 21,

wherein the relative velocity calculating section calculates the relative velocity by comparing position information of the virtual listening point and the position information of the object and the position information of the virtual listening point and the position information of the object after a predetermined time has lapsed.

23. The audio information transforming device according to claim 21,

wherein the relative velocity calculating section calculates the relative velocity by comparing the position information and velocity information of the object-and the position information of the virtual listening point.

24. The audio information transforming device according to claim 21,

wherein the relative velocity calculating section calculates the relative velocity by comparing the position information of the object and the position information and velocity information of the virtual listening point.

25. An audio information transforming device for a video/audio format in which each scene that is replayed on a screen has video information and audio information, and the scene has velocity information and direction information based on which a background is moved, said device comprising:

a virtual listening point setting section for setting a virtual listening point at a position different from a basic listening point that is set as a position at which a listener listens to an audio;

a relative velocity calculating section for calculating a relative velocity between the virtual listening point and the background based on the velocity information and the direction information of the background; and

an audio frequency transforming section for executing an audio frequency transformation based on the relative velocity to add a Doppler effect to the audio information at the virtual listening point.