Control of data processing

• US 20090010335A1
• Filed: 06/29/2005
• Published: 01/08/2009
• Est. Priority Date: 06/29/2004
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Harrison, Philip Robert, Marshall, Peter John
• Granted Patent: US 8,135,066 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6072 of an input signal, e.g. pitch and rhythm extraction, voice recognition
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player

Abstract

Data processing apparatus arranged to receive successive images from a video camera and to receive audio data from a microphone comprises means for detecting inter-image motion in the video images; means for detecting audio signals in the associated audio data satisfying one or more predetermined audio criterion; and activation means, for activating a control function of the data processing apparatus in response to a predetermined sequence of a detection of inter-image motion satisfying one or more predetermined motion criterion and a detection of an audio signal in the associated audio data which satisfies the predetermined audio criteria.

Claims

1. Data processing apparatus arranged to process successive video images and an associated audio signal;

the apparatus comprising;

means for detecting inter-image motion in the successive video images;

means for detecting whether the inter-image motion satisfies one or more predetermined motion criteria;

means for detecting an audio signal pattern in the associated audio signal satisfying one or more predetermined audio criteria; and

activation means for activating a control function of the data processing apparatus in response to a substantially simultaneous occurrence of;

(a) a detection of inter-image motion satisfying the one or more predetermined motion criteria; and

(b) a detection of an audio signal pattern in the associated audio signal which satisfies the one or more predetermined audio criteria.

2. Apparatus according to claim 1, comprising:

means for detecting, for each of a series of successive images, one or more active image regions in each image where inter-image motion has been detected; and

in which a predetermined motion criterion is satisfied if at least one of the one or more active image regions is subject to a predetermined displacement between successive images.

3. Apparatus according to claim 2, comprising:

means for detecting whether a displacement in location of at least one of the active image regions between successive images is greater than a predetermined threshold; and

in which a predetermined motion criterion is satisfied if the displacement in location of at least one active image region remains above the predetermined threshold for N successive images and subsequently falls below the predetermined threshold, N being a predetermined integer.

4. Apparatus according to claim 1, comprising:

means for detecting whether inter-image motion occurs between successive images at a predetermined proportion of a test area defined with respect to the video images; and

in which a predetermined motion criterion is satisfied if inter-image motion occurs between successive images at the predetermined proportion of the test area for N successive images and subsequently falls below the predetermined threshold, N being a predetermined integer.

5. Apparatus according to claim 1, comprising:

means for detecting, for each of a series of the video images, a plurality of active image regions where inter-image motion has been detected;

in which a predetermined motion criterion are satisfied if the displacement of at least two of the active image regions with respect to each other drops below a predetermined value.

6. Apparatus according to claim 1, in which a predetermined motion criterion is that the inter-image motion occurs within a predetermined image region.

7. Apparatus according to claim 1, in which the grouping is satisfied if there is a time displacement of less than a threshold amount between time-correlated positions in the audio signal and the sequence of video images at which the audio signal pattern is detected and the motion criteria are met.

8. Apparatus according to claim 1, comprising:

means for detecting if the amplitude of the associated audio signal exceeds a predetermined threshold; and

in which a predetermined audio criterion is satisfied if the amplitude of the associated audio signal remains substantially above the predetermined threshold for at least a predetermined duration.

9. Apparatus according to claim 1, comprising:

means for detecting the spectral signature of the associated audio signal;

in which a predetermined audio criterion is satisfied if the determined spectral signature substantially matches predetermined parameters

10. Apparatus according to claim 1, in which the successive video images are received substantially in real time from a video camera and the associated audio signal is received substantially in real time from a microphone.

11. Apparatus according to claim 10, comprising a video camera for supplying the successive images.

12. Apparatus according to claim 11, comprising a microphone integral to the video camera.

13. Apparatus according to claim 10, comprising a microphone for supplying the audio data.

14. Apparatus according to claim 1, comprising a display screen.

15. Apparatus according to claim 14, comprising:

means for displaying on the display screen a representation of the successive video images.

16. Apparatus according to claim 14, comprising:

means for displaying on the display screen a representation of the inter-image motion.

17. Apparatus according to claim 1, the apparatus being a games machine;

and the control function being a game control function.

18. A data processing method comprising the steps of:

detecting inter-image motion in a series of successive video images;

detecting whether the inter-image motion satisfies one or more predetermined motion criteria;

detecting an audio signal pattern in an associated audio signal satisfying one or more predetermined audio criteria;

activating a control function of a data processing apparatus in response to a substantially simultaneous occurrence of;

(a) a detection of inter-image motion satisfying the one or more predetermined motion criteria; and

(b) a detection of an audio signal pattern in the associated audio signal which satisfies the one or more predetermined audio criteria.

19. A method according to claim 18 comprising the steps of:

receiving successive images from a video camera;

receiving audio data from a microphone.

20. Computer software having program code for carrying out a method according to claim 18.

21. A providing medium for providing software according to claim 20.

22. A medium according to claim 21, the medium being a transmission medium.

23. A medium according to claim 21, the medium being a storage medium.

24. Data processing apparatus arranged to process successive video images and an associated audio signal;

said apparatus comprising;

a first detector to detect inter-image motion in said successive video images;

a second detector to detect whether said inter-image motion satisfies one or more predetermined motion criteria;

a third detector to detect an audio signal pattern in said associated audio signal satisfying one or more predetermined audio criteria;

logic to activate a control function of said data processing apparatus in response to a substantially simultaneous occurrence of;

(a) a detection of inter-image motion satisfying said one or more predetermined motion criteria; and

(b) a detection of an audio signal pattern in said associated audio signal which satisfies the one or more predetermined audio criteria; and

a fourth detector to detect, for each of a series of said video images, a plurality of active image regions where inter-image motion has been detected;

in which a predetermined motion criterion are satisfied if the displacement of at least two of said active image regions with respect to each other drops below a predetermined value.