Camera driven audio spatialization

• US 9,596,555 B2
• Filed: 09/27/2012
• Issued: 03/14/2017
• Est. Priority Date: 09/27/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Kaburlasos, Nikos, Cheng, Scott W., Worrell, Devon
• Primary Examiner(s): Elahee, Md S
• Application Number:
  US13/628,464
• Publication Number:
  US 20130064376A1
• Time in Patent Office: 1,629 Days
• Field of Search: 358/1.15, 345/419, 345/213, 705/39, 365/189.05, 381/17, 381/74, 381/362, 381370-374, 348/14.02
• US Class Current: 1/1
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 7/304 For headphones

Abstract

A camera input can be used by the computer to support audio spatialization or to improve audio spatialization of an application that already supports it. A computer system may to support audio spatialization, for example, by modifying the relative latency or relative amplitude of the rendered audio packets. If a sound is intended, for example, to be located on the left side of the user, then the audio channel that is rendered on the headset speaker located on the user'"'"'s left ear may have a somewhat decreased latency and increased amplitude compared to the other audio channel.

Claims

1. A method comprising:

using video analytics to determine the position and orientation of a user'"'"'s head relative to a processor based system;

based on said position and orientation, adjusting sound provided by said processor based system to headphones to produce three-dimensional sound effects which depends on distance of the user from a display of the processor based system and the orientation of the user'"'"'s head relative to the display, wherein,

adjusting the sound includes adjusting latency of audio streams produced by said processor based system, based on the distance of the user from the display and the orientation of the user'"'"'s head relative to the display.

2. The method of claim 1 including adjusting the amplitude of audio streams produced by said processor based system based on said video analytics.

3. The method of claim 1 including using a digital signal processor to produce said three-dimensional effects.

4. The method of claim 1 including introducing an appropriate delay or amplitude manipulation on each of two audio streams to spatialize them based on the position and orientation of the user'"'"'s head.

5. The method claim 1 including periodically determining the position of the user'"'"'s head.

6. The method claim 1 including periodically determine the orientation of the user'"'"'s head.

7. One or more non-transitory computer readable media storing instructions executed by a computer to perform a sequence comprising:

using video analytics to determine the position and orientation, of a user'"'"'s head relative to a processor based system;

based on said position and orientation, adjusting sound provided by said processor based system to headphones to produce three-dimensional sound effects which depends on distance of the user from a display of the processor based system and the orientation of the user'"'"'s head relative to the display, wherein,

adjusting the sound includes adjusting latency of audio streams produced by said processor based system, based on the distance of the user from the display and the orientation of the user'"'"'s head relative to the display.

8. The media of claim 7, the sequence including adjusting the amplitude of audio streams produced by said processor based system based on said video analytics.

9. The media of claim 7, the sequence including using a digital signal processor to produce said three-dimensional effects.

10. The media of claim 7, the sequence including introducing an appropriate delay or amplitude manipulation on each of two audio streams to spatialize them based on the position and orientation of the user'"'"'s head.

11. The media of claim 7, the sequence including periodically determining the position of the user'"'"'s head.

12. The media of claim 7, the sequence including periodically determine the orientation of the user'"'"'s head.

13. An apparatus comprising:

a processor to adjust sound based on a position and orientation, of a user by adjusting latency of audio streams produced by said processor based system to headphones and the orientation of the user'"'"'s head relative to a display, wherein; and

to adjust to the sound includes adjusting the latency of audio streams produced by said processor based system, based on a distance of the user from the display and the orientation of the user'"'"'s head relative to the display; and

an audio codec coupled to said processor.

14. The apparatus of claim 13 wherein said processor is a digital signal processor.

15. The apparatus of claim 13 including a device to use video analytics to determine the user'"'"'s position.

16. The apparatus of claim 15 wherein said device includes said processor.

17. The apparatus of claim 16 wherein said device includes a camera.

18. The apparatus of claim 13 said processor to determine an orientation of the user'"'"'s head.

19. The apparatus of claim 13 said processor to adjust for the distance of the user from the computer for the processor based system.

20. The apparatus of claim 19 said processor to adjust for the orientation of the user'"'"'s head relative to the computer of the processor based system.

21. The apparatus of claim 13 including an operating system.

22. The apparatus of claim 13 including a battery.

23. The apparatus of claim 13 including firmware and a module to update said firmware.