CONTROL OF DATA PROCESSING

• US 20090251553A1
• Filed: 05/17/2007
• Published: 10/08/2009
• Est. Priority Date: 05/24/2006
• Status: Abandoned Application

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Seeger, Mauritius, Cambell, Diarmid Archibald
• Application Number:
  US12/302,136
• Publication Number:
  US 20090251553A1
• Time in Patent Office: Days
• US Class Current: 348/211.990
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Data processing apparatus arranged to receive successive images from a video camera comprises means for detecting inter-image motion for selected points in an image region associated with a control function of the data processing apparatus; and means for executing the control function if inter-image motion is detected in the image region; in which the detecting means is arranged to detect inter-image motion by detecting the cumulative inter-image motion for the points in the image region over a group of m consecutive images, where m is at least three.

Claims

1. A data processing apparatus arranged to receive successive images from a video camera;

the apparatus comprising;

means for detecting inter-image motion for selected points in an image region associated with a control function of the data processing apparatus; and

means for executing the control function if inter-image motion is detected in the image region;

wherein the detecting means is arranged to detect inter-image motion by detecting the cumulative inter-image motion for the points in the image region over a group of m consecutive images, where m is at least three.

2. Apparatus according to claim 1, further comprising a video camera for supplying the successive images.

3. Apparatus according to claim 1, further comprising:

a video display;

means for displaying, on the video display, a movable user control; and

wherein the control function comprises at least moving the moveable user control.

4. Apparatus according to claim 3, wherein the moveable user control is moved by an amount dependent upon the degree of inter-image motion detected by the detecting means.

5. Apparatus according to claim 1 comprising difference measuring means operable:

to generate difference data, the difference data comprising a set of values proportional to the difference in pixel values between two successive images; and

to subsequently smooth the difference data.

6. Apparatus according to claim 5, wherein the difference measuring means is further operable to apply a threshold to the difference data at a difference threshold level, and output resulting binary difference data.

7. Apparatus according to claim 6, further comprising difference data storage means operable to combine the binary difference data for a sequence of images to generate a cumulative binary difference image indicating all points over a sequence of images that have shown a difference between successive images that has exceeded a difference threshold.

8. Apparatus according to claim 6, comprising means for processing image data received from the video camera by applying one or both of the following processes:

i. spatial averaging, and

ii. temporal averaging.

9. Apparatus according to claim 6, wherein the means for detecting inter-image motion is operable to select points in a preceding image based upon correspondence with selected points in a current image, and analyse the resulting pairs of points to detect motion.

10. Apparatus according to claim 9, wherein the means for detecting inter-image motion is operable to select test points in the current image based upon correspondence with the points in the preceding image that themselves correspond to the points originally selected in the current image, and discard pairs of points whose test points do not to substantially coincide with the originally selected points in the current image.

11. Apparatus according to claim 9, wherein the means for detecting inter-image motion is operable to discard, prior to any selection of points in a preceding image, points in the current image for which the equivalent position in the difference image shows a difference below the difference threshold level.

12. Apparatus according to claim 9, wherein the means for detecting inter-image motion is operable to discard, prior to any selection of test points in the current image based upon correspondence with the points in the preceding image, points in the preceding image for which the equivalent position in the difference image shows a difference below the difference threshold level.

13. Apparatus according to claim 1, wherein the difference measuring means is operable to calculate a difference threshold between successive images, based upon the variance in the difference values of selected points that are found to be stationary.

14. Apparatus according to claim 1, wherein the difference measuring means is operable to calculate a difference threshold between successive images, the calculation being based upon a histogram of difference values for pixels in two or more successive images, and identifying the first histogram peak encountered from the low difference value end of the histogram as the mean camera noise value above which the difference threshold is set.

15. Apparatus according to claim 9, wherein the difference measuring means is operable recursively to obtain pairs of points for detecting motion for n previous images stored in an image history store, and to compare the cumulative net motion of these pairs of points to a distance threshold, to distinguish intentional motion from image noise.

16. Apparatus according to claim 15, wherein n is three or more.

17. Apparatus according to claim 15, wherein the means for detecting inter-image motion is operable to use a cumulative difference image to discard those points where no difference between successive images sufficient to exceed a difference threshold level has occurred, prior to obtaining pairs of points recursively from previous images in the image history.

18. Apparatus according to claim 15, comprising a points history storage means for storing points detected to have shown intentional motion.

19. Apparatus according to claim 18 comprising means for graphically displaying data derived from the points history, to provide information on detected motion to a user.

20. Apparatus according to claim 15, comprising motion modelling means operable to determine changes to any or all of

i. translation;

ii. rotation; and

iii. scale;

in pairs of points for the current and preceding image that have been determined to show intentional motion.

21. Apparatus according to claim 20, wherein point pairs for some or all stored images older than the current and preceding image are used by the motion modelling means.

22. Apparatus according to claim 20 comprising motion constrain means operable to constrain the modelled motion to conform to a desired behaviour.

23. Apparatus according to claim 20 comprising display means to update at least a first display item according to the modelled motion.

24. Apparatus according to claim 23, wherein the or each display item represents a user interface control.

25. Apparatus according to claim 24, wherein the user interface control is any one of

i. a wheel control;

ii. a dial control;

iii. a scrolling control; and

iv. a scale control.

26. Apparatus according to claim 25, wherein the wheel control and the dial control are scalable.

27. Apparatus according to claim 22, wherein deviation by the user from the constrained modelled motion is evaluated, and the user interface control is locked if the deviation exceeds a threshold.

28. Apparatus according to claim 20, wherein the modelled motion is interpreted as a user input to the apparatus.

29. Apparatus according to claim 1, wherein the data processing apparatus is a games machine.

30. A data processing method comprising the steps of

receiving successive captured images from a video camera;

detecting inter-image motion for selected points in an image region associated with a control function of the data processing apparatus; and

executing the control function if inter-image motion is detected in the image region;

wherein the detecting step comprises detecting inter-image motion by detecting the cumulative inter-image motion for the points in the image region over a group of m consecutive images, where m is at least three.

31. A computer-operable software executing on a computer, the software comprising:

a program code, which when loaded into a computer, causes the computer to operate as a data processing apparatus,

wherein the apparatus comprises

means for detecting inter-image motion for selected points in an image region associated with a control function of the data processing apparatus; and

means for executing the control function if inter-image motion notion is detected in the image region;

wherein the detecting means is arranged to detect inter-image motion by detecting the cumulative inter-image motion for the points in the image region over a group of m consecutive images, where m is at least three.

32. A computer-operable software executing on a computer, the software having comprising:

a program code for carrying out of a data processing method, the method comprising the steps of

receiving successive captured images from a video camera;

detecting inter-image motion for selected points in an image region associated with a control function of the data processing apparatus; and

executing the control function if inter-image motion is detected in the image region;

wherein the detecting step comprises detecting inter-image motion by detecting the cumulative inter-image motion for the points in the image region over a group of m consecutive images, where m is at least three.

33. A providing medium for a computer-operable software executing on a computer, the software comprising:

a program code, which when loaded into a computer, causes the computer to operate as a data processing apparatus,

wherein the apparatus comprises

means for detecting inter-image motion for selected points in an image region associated with a control function of the data processing apparatus; and

means for executing the control function if inter-image motion notion is detected in the image region;

wherein the detecting means is arranged to detect inter-image motion by detecting the cumulative inter-image motion for the points in the image region over a group of m consecutive images, where m is at least three.

34. The medium of claim 33, wherein the medium comprises a transmission medium.

35. The medium of claim 33, wherein the medium being comprises a storage medium.