Selective sound source listening in conjunction with computer interactive processing

• US 20060204012A1
• Filed: 05/04/2006
• Published: 09/14/2006
• Est. Priority Date: 07/27/2002
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Marks, Richard L., Mao, Xiadong
• Granted Patent: US 7,760,248 B2
• Time in Patent Office: Days
• US Class Current: 381/26
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field

Abstract

A method and apparatus for capturing image and sound during interactivity with a computer program is provided. The apparatus includes an image capture unit that is configured to capture one or more image frames. Also provided is a sound capture unit. The sound capture unit is configured to identify one or more sound sources. The sound capture unit generates data capable of being analyzed to determine a zone of focus, at which to process sound to the substantial exclusion of sounds outside of the zone of focus. In this manner, sound that is captured and processed for the zone of focus is used for interactivity with the computer program.

Claims

1. An apparatus for capturing image and sound during interactivity with a computer program, comprising:

an image capture unit configured to capture one or more image frames; and

a sound capture unit, the sound capture unit being configured to identify one or more sound sources, the sound capture unit generating data capable of being analyzed to determine a zone of focus at which to process sound to the substantial exclusion of sounds outside of the zone of focus, sound that is captured and processed for the zone of focus is used for interactivity with the computer program.

2. An apparatus for capturing image and sound during interactivity with a computer program as recited in claim 1, wherein the sound capture unit includes an array of microphones, the array of microphones being configured for receiving sound from the one or more sound sources, the sounds of the one or more sound sources defining sound paths to each of the microphones.

3. An apparatus for capturing image and sound during interactivity with a computer program as recited in claim 2, wherein the sound paths include particular delays that enable calculation of direction of each of the one or more sound sources relative to the apparatus for capturing image and sound.

4. An apparatus for capturing image and sound during interactivity with a computer program as recited in claim 1, further comprising:

a computing system for interfacing with the apparatus for capturing image and sound, the computing system including,

a processor, and

memory, the memory being configured to store at least part of the computer program and selective sound source listening code, the selective sound source listening code enabling the identification of which of the one or more sound sources identify as the zone of focus.

5. An apparatus for capturing image and sound during interactivity with a computer program as recited in claim 1, wherein the sound capture unit includes at least four microphones, and one of the four microphones is not in a same plane as the others.

6. An apparatus for capturing image and sound during interactivity with a computer program as recited in claim 5, wherein the four microphones define a spatial volume.

7. An apparatus for capturing image and sound during interactivity with a computer program as recited in claim 6, wherein the spatial volume is defined as a volume of focus for listening during interactivity with the computer program.

8. An apparatus for capturing image and sound during interactivity with a computer program as recited in claim 7, wherein the computer program is a game program.

9. An apparatus for capturing image and sound during interactivity with a computer program as recited in claim 1, wherein the computer program is a game program.

10. An apparatus for capturing image and sound during interactivity with a computer program as recited in claim 9, wherein the image capture unit is camera and the sound capture unit is defined by an array of two or more microphones.

11. A method for selective sound source listening during interactivity with a computer program, comprising:

receiving input from one or more sound sources at two or more sound source capture microphones;

determining delay paths from each of the sound sources;

identifying a direction for each of the received inputs of each of the one or more sound sources; and

filtering out sound sources that are not in an identified direction of a zone of focus, the zone of focus supplying the sound source for the interactivity with the computer program.

12. A method for selective sound source listening during interactivity with a computer program as recited in claim 11, wherein filtering receives processed input data after analysis by an image capture unit, the image capture unit being directionally positioned to receive image input for the computer program.

13. A method for selective sound source listening during interactivity with a computer program as recited in claim 11, wherein the computer program is a game, and the game receives interactive input from both image data and sound data, the sound data being from the sound source of the zone of focus.

14. A method for selective sound source listening during interactivity with a computer program as recited in claim 11, wherein the two or more sound capture microphones include at least four microphones, and at least one of the four microphones is on a different plane than the others.

15. A method for selective sound source listening during interactivity with a computer program as recited in claim 14, wherein identifying the direction for each of the received inputs of each of the one or more sound sources includes processing a triangulation algorithm, the triangulation algorithm defining the direction that is relative to a location at which input is received from the one or more sound sources at the two or more sound source capture microphones.

16. A method for selective sound source listening during interactivity with a computer program as recited in claim 15, further comprising:

buffering the received input from the one or more sound sources associated with the two or more sound source capture microphones; and

delay processing the received buffered inputs;

the filtering further, including,

selecting the one of the sound sources, the selected sound source output being a summation of sound from each of the sound source capture microphones.

17. A game system, comprising:

an image-sound capture device, the image-sound capture device being configured to interface with a computing system that enables execution of an interactive computer game, the image-capture device including,

video capture hardware capable of being positioned to capture video from a zone of focus, and

an array of microphones for capturing sound from one or more sound sources, each sound source being identified and associated with a direction relative to the image-sound capture device, the zone of focus associated with the video capture hardware is configured to be used to identify one of the sound sources at the direction that is in the proximity of the zone of focus.

18. A game system as recited in claim 17, wherein the video capture hardware receives video data to enable interactivity with features of the computer game.

19. A game system as recited in claim 17, wherein the sound source in the proximity of the zone of focus is enabled interactivity with the computer game or voice communication with other game users.

20. A game system as recited in claim 19, wherein sound sources outside of the zone of focus are filtered out of interactivity with the computer game.

21. An apparatus for capturing sound during interactivity with a computer program, comprising:

a sound capture unit for capturing sound from one or more sound sources;

a processor and memory for processing and receiving the sound, the processor being configured to execute instructions to identify one of the sound sources as associated with a zone of focus, the sound from the identified sound source being processed to enable interactive input with the computer program.

22. An apparatus for capturing sound during interactivity with a computer program as recited in claim 21, wherein the instructions to identify one of the sound sources uses triangulation to identify a direction of each of the sound sources.

23. An apparatus for capturing sound during interactivity with a computer program as recited in claim 21, wherein the instructions to identify one of the sound sources uses sound frequencies to identify each of the sound sources.

24. An apparatus for capturing sound during interactivity with a computer program as recited in claim 21, wherein the interactive input is one of communication with a program or communication with a third party.

25. An apparatus for capturing sound during interactivity with a computer program as recited in claim 21, wherein the input is used for interactive input interfaces with features of a computer game.

26. An apparatus for capturing sound during interactivity with a computer program as recited in claim 21, wherein the interactive input interfaces with an electronic apparatus.