Control of data processing

• US 8,135,066 B2
• Filed: 06/29/2005
• Issued: 03/13/2012
• Est. Priority Date: 06/29/2004
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Harrison, Philip Robert, Marshall, Peter John
• Primary Examiner(s): Sam, Phirin
• Application Number:
  US11/631,143
• Publication Number:
  US 20090010335A1
• Time in Patent Office: 2,449 Days
• Field of Search: None
• US Class Current: 375/240.16
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6072 of an input signal, e.g. pitch and rhythm extraction, voice recognition
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player

Abstract

Data processing apparatus arranged to receive successive images from a video camera and to receive audio data from a microphone comprises means for detecting inter-image motion in the video images; means for detecting audio signals in the associated audio data satisfying one or more predetermined audio criterion; and activation means, for activating a control function of the data processing apparatus in response to a predetermined sequence of a detection of inter-image motion satisfying one or more predetermined motion criterion and a detection of an audio signal in the associated audio data which satisfies the predetermined audio criteria.

Claims

1. Data processing apparatus arranged to process successive video images and an associated audio signal;

the apparatus comprising;

means for detecting inter-image motion in the successive video images;

means for detecting whether the inter-image motion satisfies one or more predetermined motion criteria;

means for detecting an audio signal pattern in the associated audio signal satisfying one or more predetermined audio criteria;

activation means for activating a control function of the data processing apparatus in response to a substantially simultaneous occurrence of;

(a) a detection of inter-image motion satisfying the one or more predetermined motion criteria; and

(b) a detection of an audio signal pattern in the associated audio signal which satisfies the one or more predetermined audio criteria; and

means for detecting, for each of a series of successive images, one or more active image regions in each image where inter-image motion has been detected;

in which a predetermined motion criterion is satisfied if at least one of the one or more active image regions is subject to a predetermined displacement between successive images.

2. The apparatus according to claim 1, further comprising:

means for detecting whether a displacement in location of at least one of the active image regions between successive images is greater than a predetermined threshold; and

in which a predetermined motion criterion is satisfied if the displacement in location of at least one active image region remains above the predetermined threshold for N successive images and subsequently falls below the predetermined threshold, N being a predetermined integer.

3. Data processing apparatus arranged to process successive video images and an associated audio signal, the apparatus comprising:

means for detecting inter-image motion in the successive video images;

means for detecting whether the inter-image motion satisfies one or more predetermined motion criteria;

means for detecting an audio signal pattern in the associated audio signal satisfying one or more predetermined audio criteria;

activation means for activating a control function of the data processing apparatus in response to a substantially simultaneous occurrence of;

(a) a detection of inter-image motion satisfying the one or more predetermined motion criteria; and

(b) a detection of an audio signal pattern in the associated audio signal which satisfies the one or more predetermined audio criteria; and

means for detecting whether inter-image motion occurs between successive images at a predetermined proportion of a test area defined with respect to the video images; and

in which a predetermined motion criterion is satisfied if inter-image motion occurs between successive images at the predetermined proportion of the test area for N successive images and subsequently falls below the predetermined threshold, N being a predetermined integer.

4. Data processing apparatus arranged to process successive video images and an associated audio signal, the apparatus comprising:

means for detecting inter-image motion in the successive video images;

means for detecting whether the inter-image motion satisfies one or more predetermined motion criteria;

means for detecting an audio signal pattern in the associated audio signal satisfying one or more predetermined audio criteria;

activation means for activating a control function of the data processing apparatus in response to a substantially simultaneous occurrence of;

(a) a detection of inter-image motion satisfying the one or more predetermined motion criteria; and

(b) a detection of an audio signal pattern in the associated audio signal which satisfies the one or more predetermined audio criteria; and

means for detecting, for each of a series of the video images, a plurality of active image regions where inter-image motion has been detected;

in which a predetermined motion criterion are satisfied if the displacement of at least two of the active image regions with respect to each other drops below a predetermined value.

5. Data processing apparatus arranged to process successive video images and an associated audio signal, the apparatus comprising:

means for detecting inter-image motion in the successive video images;

means for detecting whether the inter-image motion satisfies one or more predetermined motion criteria;

means for detecting an audio signal pattern in the associated audio signal satisfying one or more predetermined audio criteria; and

activation means for activating a control function of the data processing apparatus in response to a substantially simultaneous occurrence of;

(a) a detection of inter-image motion satisfying the one or more predetermined motion criteria; and

(b) a detection of an audio signal pattern in the associated audio signal which satisfies the one or more predetermined audio criteria,

in which a grouping is satisfied if there is a time displacement of less than a threshold amount between time-correlated positions in the audio signal and the sequence of video images at which the audio signal pattern is detected and the motion criteria are met.

6. Data processing apparatus arranged to process successive video images and an associated audio signal, the apparatus comprising:

means for detecting inter-image motion in the successive video images;

means for detecting whether the inter-image motion satisfies one or more predetermined motion criteria;

means for detecting an audio signal pattern in the associated audio signal satisfying one or more predetermined audio criteria;

activation means for activating a control function of the data processing apparatus in response to a substantially simultaneous occurrence of;

(a) a detection of inter-image motion satisfying the one or more predetermined motion criteria; and

(b) a detection of an audio signal pattern in the associated audio signal which satisfies the one or more predetermined audio criteria; and

means for detecting if the amplitude of the associated audio signal exceeds a predetermined threshold; and

in which a predetermined audio criterion is satisfied if the amplitude of the associated audio signal remains substantially above the predetermined threshold for at least a predetermined duration.

7. Data processing apparatus arranged to process successive video images and an associated audio signal, the apparatus comprising:

means for detecting inter-image motion in the successive video images;

means for detecting whether the inter-image motion satisfies one or more predetermined motion criteria;

means for detecting an audio signal pattern in the associated audio signal satisfying one or more predetermined audio criteria;

activation means for activating a control function of the data processing apparatus in response to a substantially simultaneous occurrence of;

(a) a detection of inter-image motion satisfying the one or more predetermined motion criteria; and

(b) a detection of an audio signal pattern in the associated audio signal which satisfies the one or more predetermined audio criteria; and

means for detecting the spectral signature of the associated audio signal;

in which a predetermined audio criterion is satisfied if the determined spectral signature substantially matches predetermined parameters.