Tagging a sound in a virtual environment

• US 10,585,641 B2
• Filed: 04/30/2018
• Issued: 03/10/2020
• Est. Priority Date: 04/30/2018
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Anantharaman, Chetan, Tirunagaru, Bharath Kumar
• Primary Examiner(s): Nguyen, Tuan S
• Application Number:
  US15/967,305
• Publication Number:
  US 20190332352A1
• Time in Patent Office: 680 Days
• Field of Search: None
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 13/822 Strategy games; Role-playing games A63F13/825, A63F13/828 take precedence
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 2499/15 Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers
• CPC Class Codes: H04R 5/027 Spatial or constructional arrangements of microphones, e.g. in dummy heads
• CPC Class Codes: H04S 1/005 For headphones
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 7/308 Electronic adaptation dependent on speaker or headphone connection

Abstract

A method includes generating, at a processor, a first virtual scene that includes a virtual object. The method also includes generating a user option to insert a virtual microphone into the first virtual scene. The user option enables user selection of a location of the virtual microphone. The method further includes generating a second virtual scene. The method also includes outputting a tagged sound associated with the virtual object while a display device displays the first virtual scene. The method further includes outputting the tagged sound while the display device displays the second virtual scene in response to a determination that the virtual microphone is inserted into the first virtual scene. Properties of the tagged sound are based on the location of the virtual microphone while the display device displays the second virtual scene.

Claims

1. An apparatus comprising:

a display device;

a processor coupled to the display device, the processor configured to;

generate a first virtual scene comprising a virtual object;

generate a user option to insert a virtual microphone into the first virtual scene, the user option enabling user selection of a location of the virtual microphone; and

generate a second virtual scene; and

a speaker coupled to the processor, the speaker configured to;

output a tagged sound associated with the virtual object while the display device displays the first virtual scene; and

output the tagged sound while the display device displays the second virtual scene in response to a determination that the virtual microphone is inserted into the first virtual scene, wherein properties of the tagged sound, while the display device displays the second virtual scene, are based on the location of the virtual microphone.

2. The apparatus of claim 1, wherein the user option further enables user selection of an orientation of the virtual microphone, and wherein the properties of the tagged sound, while the display device displays the second virtual scene, are further based on the orientation of the virtual microphone.

3. The apparatus of claim 1, further comprising a virtual reality headset, wherein the display device, the speaker, and the processor are integrated within the virtual reality headset.

4. The apparatus of claim 1, further comprising:

a memory storing;

a first subroutine corresponding to first audio of the first virtual scene; and

a second subroutine corresponding to second audio of the second virtual scene; and

a sound engine integrated into the processor, the sound engine configured to;

execute the first subroutine to render the first audio while the display device displays the first virtual scene, the rendered first audio output by the speaker as first virtual scene sounds associated with the first virtual scene; and

execute the second subroutine to render the second audio while the display device displays the second virtual scene, the rendered second audio output by the speaker as second virtual scene sounds associated with the second virtual scene.

5. The apparatus of claim 4, wherein in response to the determination that the virtual microphone is inserted into the first virtual scene, the sound engine is further configured to:

continuously execute a subset of instructions in the first subroutine to render tagged audio of the first virtual scene, the tagged audio based on the location of the virtual microphone, and the rendered tagged audio output by the speaker as the tagged sound.

6. The apparatus of claim 5, wherein the processor is further configured to identify the location of the virtual microphone, and wherein the subset of instructions is associated with the identified location.

7. The apparatus of claim 1, wherein the display device, the processor, and the speaker are integrated into a mobile device.

8. The apparatus of claim 1, further comprising a base station and a user device, wherein the user device includes the display device, the processor, and the speaker, and wherein the base station is configured to send data to the processor.

9. A method comprising:

generating, at a processor, a first virtual scene comprising a virtual object;

generating a user option to insert a virtual microphone into the first virtual scene, the user option enabling user selection of a location of the virtual microphone;

generating a second virtual scene;

outputting a tagged sound associated with the virtual object while a display device displays the first virtual scene; and

outputting the tagged sound while the display device displays the second virtual scene in response to a determination that the virtual microphone is inserted into the first virtual scene, wherein properties of the tagged sound, while the display device displays the second virtual scene, are based on the location of the virtual microphone.

10. The method of claim 9, wherein the user option further enables user selection of an orientation of the virtual microphone, and wherein the properties of the tagged sound, while the display device displays the second virtual scene, are further based on the orientation of the virtual microphone.

11. The method of claim 9, wherein the display device is integrated into a virtual reality headset.

12. The method of claim 9, further comprising:

executing, at the processor, a first subroutine to render the first audio while the display device displays the first virtual scene, the rendered first audio output by the speaker as first virtual scene sounds associated with the first virtual scene; and

executing a second subroutine to render the second audio while the display device displays the second virtual scene, the rendered second audio output by the speaker as second virtual scene sounds associated with the second virtual scene.

13. The method of claim 12, further comprising, in response to the determination that the virtual microphone is inserted into the first virtual scene:

continuously executing a subset of instructions of the first subroutine to render tagged audio of the first virtual scene, the tagged audio based on the location of the virtual microphone, and the rendered tagged audio output by the speaker as the tagged sound.

14. The method of claim 13, further comprising identifying the location of the virtual microphone, and wherein the subset of instructions is associated with the identified location.

15. A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform operations comprising:

generating a first virtual scene comprising a virtual object;

generating a user option to insert a virtual microphone into the first virtual scene, the user option enabling user selection of a location of the virtual microphone;

generating a second virtual scene;

outputting a tagged sound associated with the virtual object while a display device displays the first virtual scene; and

outputting the tagged sound while the display device displays the second virtual scene in response to a determination that the virtual microphone is inserted into the first virtual scene, wherein properties of the tagged sound, while the display device displays the second virtual scene, are based on the location of the virtual microphone.

16. The non-transitory computer-readable medium of claim 15, wherein the user option further enables user selection of an orientation of the virtual microphone, and wherein the properties of the tagged sound, while the display device displays the second virtual scene, are further based on the orientation of the virtual microphone.

17. The non-transitory computer-readable medium of claim 15, wherein the operations further comprise:

executing a first subroutine to render the first audio while the display device displays the first virtual scene, the rendered first audio output by the speaker as first virtual scene sounds associated with the first virtual scene; and

executing a second subroutine to render the second audio while the display device displays the second virtual scene, the rendered second audio output by the speaker as second virtual scene sounds associated with the second virtual scene.

18. The non-transitory computer-readable medium of claim 17, wherein in response to the determination that the virtual microphone is inserted into the first virtual scene, the operations further comprise:

continuously executing a subset of instructions of the first subroutine to render tagged audio of the first virtual scene, the tagged audio based on the location of the virtual microphone, and the rendered tagged audio output by the speaker as the tagged sound.

19. The non-transitory computer-readable medium of claim 18, wherein the operations further comprise identifying the location of the virtual microphone, the subset of instructions associated with the identified location.

20. An apparatus comprising:

a processor configured to;

generate a first virtual scene and a second virtual scene, the first virtual scene comprising a virtual object;

generate a user option to insert a virtual microphone into the first virtual scene, the user option enabling user selection of a location of the virtual microphone; and

output a tagged sound associated with the virtual object, the tagged sound outputted while a display device displays the first virtual scene, and the tagged sound outputted while the display device displays the second virtual scene in response to a determination that the virtual microphone is inserted into the first virtual scene, wherein properties of the tagged sound, while the display device displays the second virtual scene, are based on the location of the virtual microphone.

21. The apparatus of claim 20, wherein the user option further enables user selection of an orientation of the virtual microphone, and wherein the properties of the tagged sound, while the display device displays the second virtual scene, are further based on the orientation of the virtual microphone.