Selective sound source listening in conjunction with computer interactive processing

• US 8,723,984 B2
• Filed: 06/22/2010
• Issued: 05/13/2014
• Est. Priority Date: 07/27/2002
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Marks, Richard L., Mao, Xiadong
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): DAGNEW, MEKONNEN D
• Application Number:
  US12/820,618
• Publication Number:
  US 20100303298A1
• Time in Patent Office: 1,421 Days
• Field of Search: 348/231, 348/515, 348/736, 348/738, 348/14.09, 348/231.4, 381/26, 381/92, 381/356, 381/357, 381111-115, 381/95, 381/122.91, 463/36
• US Class Current: 348/231.4
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field

Abstract

A method and apparatus for capturing image and sound during interactivity with a computer program is provided. The apparatus includes an image capture unit that is configured to capture one or more image frames. Also provided is a sound capture unit. The sound capture unit is configured to identify one or more sound sources. The sound capture unit generates data capable of being analyzed to determine a zone of focus at which to process sound to the substantial exclusion of sounds outside of the zone of focus. In this manner, sound that is captured and processed for the zone of focus is used for interactivity with the computer program.

Claims

1. An apparatus for capturing image and sound during interactivity with a computer game in a game environment, comprising:

an image capture unit for capturing one or more image frames of the game environment, the game environment defined for interactive game play in a vicinity of a display upon which the computer game is rendered, the captured one or more image frames being processed to identify at least one active image area that is to include a user, the active image area identifying a zone of focus from which to process interactive sounds of the user for the interactive game play; and

a sound capture unit for capturing sound from the game environment, the sound capture unit identifying one or more sound sources, the sound capture unit configured to process sound to the substantial exclusion of sounds outside of the zone of focus, the excluded sounds being sounds which are not central to gameplay of the computer game, wherein the interactive sounds of the user are captured and processed for the zone of focus and used for interactivity with the computer game.

2. An apparatus for capturing image and sound during interactivity with a computer game as recited in claim 1,

wherein the sound capture unit includes an array of microphones, the array of microphones for receiving sound from the one or more sound sources, the sounds of the one or more sound sources defining sound paths to each of the microphones;

and

wherein the image capture unit includes a camera.

3. An apparatus for capturing image and sound during interactivity with a computer game as recited in claim 1, further comprising:

a computing system for interfacing with the apparatus for capturing image and sound, the computing system including,

a processor, and

memory, the memory for storing at least part of the computer game and selective sound source listening code, the selective sound source listening code enabling the identification of which of the one or more sound sources to identify as the zone of focus.

4. An apparatus for capturing image and sound during interactivity with a computer game as recited in claim 3,

wherein the computer system is connected to a network;

wherein the computer game is a networked computer game.

5. An apparatus for capturing image and sound during interactivity with a computer game as recited in claim 4, further comprising,

one or more speakers connected to the computer system;

wherein the computer system receives audio from the network and outputs the audio via the speakers;

wherein the processing of the sound prevents feedback of the audio received from the network.

6. An apparatus for capturing image and sound during interactivity with a computer game as recited in claim 1,

wherein the one or more sound sources include multiple users;

wherein the at least one active image area includes multiple active image areas each defined by one of the multiple users; and

wherein the at least one zone of focus includes multiple zones of focus in the game environment each substantially determined by one of the multiple active image areas.

7. A method for selective sound source listening during interactivity with a computer game in a game environment, comprising:

receiving input from one or more sound sources in the game environment at two or more sound source capture microphones, the game environment defined for interactive game play by a user in a vicinity of a display upon which the computer game is rendered;

identifying a direction for each of the received inputs of each of the one or more sound sources; and

filtering out sound sources that are not in an identified direction of at least one zone of focus in the game environment, the filtered out sound sources being sound sources which are not central to gameplay of the computer game, the zone of focus supplying interactive sounds of the user for the interactivity with the computer game;

wherein filtering receives processed input data after analysis by an image capture unit, the image capture unit producing image data to at least partially identify the zone of focus, and the image capture device being directionally positioned to receive image input for the computer game.

8. A method for selective sound source listening during interactivity with a computer game as recited in claim 7, wherein the computer game receives interactive input from both image data and sound data, the sound data being from the sound source of the zone of focus.

9. A method for selective sound source listening during interactivity with a computer game as recited in claim 7, wherein the sound sources include one or more speakers.

10. A method for selective sound source listening during interactivity with a computer game as recited in claim 9,

wherein the computer game interfaces with a network;

wherein the speakers output audio received over the network; and

wherein the filtering prevents feedback of the audio received over the network.

11. A method for selective sound source listening during interactivity with a computer game as recited in claim 7,

wherein the image data includes multiple users;

and

wherein the at least one zone of focus includes multiple zones of focus each being at least partially identified by one of the multiple users in the image data.

12. A game system, comprising:

an image-sound capture device, the image-sound capture device being configured to interface with a computing system that enables execution of an interactive computer game, the interactive computer game facilitating interactivity in a game environment, the image-capture device including,

video capture hardware positioned to capture video from at least one zone of focus in the game environment, and

an array of microphones for capturing sound from one or more sound sources in the game environment, the game environment defined for interactive game play in a vicinity of a display upon which the computer game is rendered, each sound source being identified and associated with a direction relative to the image-sound capture device, the zone of focus associated with the video capture hardware is used to at least partially identify one of the sound sources at the direction that is in proximity of the zone of focus, wherein interactive sounds of the identified one of the sound sources are captured and used for interactivity with the computer game;

wherein sound sources outside of the zone of focus are filtered out of interactivity with the computer game, the filtered out sound sources being sound sources which are not central to gameplay of the computer game.

13. A game system as recited in claim 12, wherein the sound source in the proximity of the zone of focus enables interactivity with the computer game or voice communication with other game users.

14. A game system as recited in claim 12,

wherein the computer system is connected to a network;

wherein the computer game is a networked computer game.

15. A game system as recited in claim 14, further comprising,

one or more speakers connected to the computing system;

wherein the computer system receives audio from the network and outputs the audio via the speakers;

wherein the processing of the sound prevents feedback of the audio received from the second computer system.

16. A game system as recited in claim 12,

wherein the at least one zone of focus includes multiple zones of focus;

and

wherein the one or more sound sources include one or more users.