CONTROL OF DATA PROCESSING

• US 20090268040A1
• Filed: 05/17/2007
• Published: 10/29/2009
• Est. Priority Date: 05/24/2006
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Campbell, Diarmid Achibald, Seeger, Mauritius
• Granted Patent: US 8,654,099 B2
• Time in Patent Office: Days
• US Class Current: 348/207.100
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Data processing apparatus arranged to receive successive images from a video camera comprises means for detecting inter-image motion for selected points in an image region associated with a control function of the data processing apparatus; and means for executing the control function if inter-image motion is detected in the image region; in which the detecting means is arranged to select points in an image region of a current image that are validated by selecting test points in a currently received image based upon their correspondence with points in a preceding received image, the points in the preceding received image themselves corresponding with the points originally selected in the currently received image, and deselect points in the current image whose corresponding test points do not to substantially coincide.

Claims

1. Data processing apparatus arranged to receive successive images from a video camera;

the apparatus comprising;

means for detecting inter-image motion for selected points in an image region associated with a control function of the data processing apparatus; and

means for executing the control function if inter-image motion is detected in the image region;

in which the detecting means is arranged to select points in an image region of a current image that are validated by selecting test points in a currently received image based upon their correspondence with points in a preceding received image, the points in the preceding received image themselves corresponding with the points originally selected in the currently received image, and deselect points in the current image whose corresponding test points do not to substantially coincide.

2. Apparatus according to claim 1, comprising a video camera for supplying the successive images.

3. Apparatus according to claim 1, comprising:

a video display;

means for displaying, on the video display, a movable user control;

in which the control function comprises at least moving the moveable user control.

4. Apparatus according to claim 3, in which the moveable user control is moved by an amount dependent upon the degree of inter-image motion detected by the detecting means.

5. Apparatus according to claim 1 comprising difference measuring means operable:

to generate difference data, the difference data comprising a set of values proportional to the difference in pixel values between two successive images; and

to subsequently smooth the difference data.

6. Apparatus according to claim 5, in which the difference measuring means is further operable to apply a threshold to the difference data at a difference threshold level, and output resulting binary difference data.

7. Apparatus according to claim 6, comprising difference data storage means operable to combine the binary difference data for a sequence of images to generate a cumulative binary difference image indicating all points over a sequence of images that have shown a difference between successive images that has exceeded a difference threshold.

8. Apparatus according to claim 6, comprising means for processing image data received from the video camera by applying one or both of the following processes:

i. spatial averaging, and ii. temporal averaging.

9. Apparatus according to claim 6, in which the means for detecting inter-image motion is operable to discard, prior to any selection of points in a preceding image, points in the current image for which the equivalent position in the difference image shows a difference below the difference threshold level.

10. Apparatus according to claim 6, in which the means for detecting inter-image motion is operable to discard, prior to any selection of test points in the current image based upon correspondence with the points in the preceding image, points in the preceding image for which the equivalent position in the difference image shows a difference below the difference threshold level.

11. Apparatus according to claim 5, in which the difference measuring means is operable to calculate a difference threshold between successive images, based upon the variance in the difference values of selected points that are found to be stationary.

12. Apparatus according to claim 5, in which the difference measuring means is operable to calculate a difference threshold between successive images, the calculation being based upon a histogram of difference values for pixels in two or more successive images, and identifying the first histogram peak encountered from the low difference value end of the histogram as the mean camera noise value above which the difference threshold is set.

13. Apparatus according to claim 9, in which the difference measuring means is operable recursively to obtain pairs of points for detecting motion for n previous images stored in an image history store, and to compare the cumulative net motion of these pairs of points to a distance threshold, to distinguish intentional motion from image noise.

14. Apparatus according to claim 13 in which n is three or more.

15. Apparatus according to claim 13, in which the means for detecting inter-image motion is operable to use a cumulative difference image to discard those points where no difference between successive images sufficient to exceed a difference threshold level has occurred, prior to obtaining pairs of points recursively from previous images in the image history.

16. Apparatus according to claim 13, comprising a points history storage means for storing points detected to have shown intentional motion.

17. Apparatus according to claim 16 comprising means for graphically displaying data derived from the points history, to provide information on detected motion to a user.

18. Apparatus according to claim 13, comprising motion modelling means operable to determine changes to any or all of:

i. translation;

ii. rotation; and

iii. scale;

in pairs of points for the current and preceding image that have been determined to show intentional motion.

19. Apparatus according to claim 18 in which point pairs for some or all stored images older than the current and preceding image are used by the motion modelling means.

20. Apparatus according to claim 18 comprising motion constraint means operable to constrain the modelled motion to conform to a desired behaviour.

21. Apparatus according to claim 18 comprising display means to update at least a first display item according to the modelled motion.

22. Apparatus according to claim 21 in which the or each display item represents a user interface control.

23. Apparatus according to claim 22 in which the user interface control is any one of:

i. a wheel control;

ii. a dial control;

iii. a scrolling control; and

iv. a scale control.

24. Apparatus according to claim 23 in which the wheel control and the dial control are scalable.

25. Apparatus according to claim 20 in which deviation by the user from the constrained modelled motion is evaluated, and the user interface control is locked if the deviation exceeds a threshold.

26. Apparatus according to claim 18 in which the modelled motion is interpreted as a user input to the apparatus.

27. Apparatus according to claim 1 in which the data processing apparatus is a games machine.

29. Computer software having program code which when loaded into a computer, causes the computer to operate as a data processing apparatus according to claim 1.

31. A providing medium for providing software according to claim 29.

32. A medium according to claim 31, the medium being a transmission medium.

33. A medium according to claim 31, the medium being a storage medium.

28. A data processing method comprising the steps of:

receiving successive captured images from a video camera;

detecting inter-image motion for selected points in an image region associated with a control function of the data processing apparatus; and

executing the control function if inter-image motion is detected in the image region;

in which the detecting step comprises;

selecting points in an image region of a current image that are validated by selecting test points in a currently received image based upon their correspondence with points in a preceding received image, the points in the preceding received image themselves corresponding with the points originally selected in the currently received image, and deselecting points in the current image whose corresponding test points do not to substantially coincide.

30. Computer software having program code for carrying out a method according to claim 28.