APPARATUS AND METHOD OF INTERACTION WITH A DATA PROCESSOR

• US 20090318228A1
• Filed: 07/17/2007
• Published: 12/24/2009
• Est. Priority Date: 07/25/2006
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Hughes, Colin Jonathan
• Granted Patent: US 8,241,125 B2
• Time in Patent Office: Days
• US Class Current: 463/36
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A method of interaction with a data processor such as a games machine comprises the steps of capturing a wide angle image of an environment, obtaining image features from the wide angle image, and then interpreting these image features as inputs to software running on the games machine. The step of obtaining image features from the wide angle image further comprises the step of mapping one or more regions of the wide-angle image to corresponding one or more flat image planes, and obtaining image features from one or more of said planes of the mapped wide-angle image. Advantageously, the planes can be arranged to reflect the topology of a room or the positioning of players, and allow a consistent representation of image features and consequently of input motion to a process such as a game, without position dependent distortions of the features inherent in the wide angle image.

Claims

1. A method of interaction with a games machine, comprising the steps of:

capturing a wide angle image of an environment;

obtaining image features from the wide angle image, and

interpreting those image features as inputs to software running on the games machine,

wherein the step of obtaining image features from the wide angle image further comprises the steps of;

mapping one or more regions of the wide-angle image to corresponding one or more flat image planes, hereafter referred to as facets, and

obtaining image features from one or more of said facets of the mapped wide angle image.

2. A method according to claim 1 in which the wide-angle image is substantially hemispherical.

3. A method according to claim 1 in which there are five facets that together form five planar surfaces of a cube.

4. A method according to claim 1, further comprising a step in which the image features are obtained by performing edge detection on one of more facets.

5. A method according to claim 4 in which edge detection is based upon changes in any or all of:

i. luminance, and;

ii. chrominance.

6. A method according to claim 4 in which edge detection employs any or all of:

i. the Sobel operator;

ii. the Canny operator;

iii. the pyramid operator, and;

iv. the Laplace operator.

7. A method according to claim 4 further comprising a step of selecting any or all of the following edge types as image features:

i. substantially linear edges;

ii. points where two or more linear edges meet, and;

iii. predetermined shapes formed by sets of edges.

8. A method according to claim 7 in which substantially linear edges are determined by identifying three or more substantially co-linear edge points.

9. A method according to claim 7 in which predetermined shapes are identified by any one of:

i. template matching;

ii. Hough'"'"'s transform, and;

iii. Bresenham'"'"'s transform.

10. A method according to claim 1 comprising a step of estimating, from image features obtained from one or more of the facets, any or all of:

i. an initial orientation, and;

ii. an initial position,

with respect to said features.

11. A method according to claim 10 comprising a further step of re-estimating, from image features obtained from one or more of the facets, any or all of:

i. a current orientation, and;

ii. a current position,

12. A method according to claim 11 in which relative movement of selected image features is tracked over successive images to determine changes in any or all of:

i. orientation, and;

ii. position,

relative to a preceding orientation and position, as applicable.

13. A method according to claim 12 in which the relative movement of selected image features is determined by generating an image transform embodying any or all of:

i. orientation parameters, and;

ii. translation parameters,

and parametrically minimising the error between the current image features and a prediction of the current image features based upon previously obtained image features processed by said image transform.

14. A method according to claim 12 in which successive estimates of movement are combined as inputs to a game.

15. A method according to claim 14 in which an input may represent the movement of any one of:

i. a torch or gun;

ii. a computer mouse;

iii. a wand or gestural device;

iv. a golf club;

v. a fishing rod;

vi. a melee weapon, and;

vii. a point in 3D space.

16. A method according to claim 1 comprising a step of estimating, from image features obtained from one or more of the facets, the positioning of at least part of the body of at least a first user.

17. A method according to claim 16 further comprising a step of tracking movements of at least a first user by comparing image features from one or more of the facets of successive images.

18. A method according to claim 16 in which the wide angle image is mapped to facets such that of each of a plurality of users in the wide angle image is mapped to a separate respective facet.

19. A method according to claim 18 in which one or more of a plurality of users whose image is mapped to a separate respective facet are displayed within a game.

20. A method according to claim 1 in which a processing sequence comprising:

i. mapping a wide-angle image to one or more facets;

ii. obtaining image features from one or more facets, and;

iii. estimating current orientation and/or position with respect to said image features,

is carried out by an image processing means of a host data processor upon receipt of a wide-angle image transmitted from a peripheral control means.

21. A method according to claim 1 in which a processing sequence comprising:

1. mapping a wide-angle image to one or more facets;

ii. obtaining image features from one or more facets, and;

iii. estimating current orientation and/or position with respect to said image features,

is carried out by an image processing means of a peripheral control means.

22. A method according to claim 21 in which orientation and/or position information is transmitted from the peripheral control means to a host data processor.

23. A method according to claim 1 in which a processing sequence comprising:

i. mapping a wide-angle image to one or more facets;

ii. obtaining image features from one or more facets, and;

iii. estimating current orientation and/or position with respect to said image features,

is shared at any point in the processing sequence between firstly a peripheral control means and secondly a host data processor.

24. A method according to claim 23 in which information relevant to the next step in the processing sequence is transmitted from the peripheral control means to a host data processor.

25. A method according to claim 1 in which the data processor is a games machine.

36. A games machine operable to carry out the method of claim 1, and comprising;

an image processor.

37. A data carrier comprising computer readable instructions that, when loaded into a computer, cause the computer to carry out the method of claim 1.

40. A data signal comprising computer readable instructions that, when loaded into a computer, cause the computer to carry out the method of claim 1.

43. Computer readable instructions that, when loaded into a computer, cause the computer to carry out the method of claim 1.

26. A control means for a games machine, comprising an imaging means coupled with a wide-angle lens means and operable to capture a wide-angle image, wherein the wide-angle lens means provides a substantially hemispherical field of view, and wherein image features obtained from the wide angle image are interpretable as inputs to software running on said games machine.

27. A control means according to claim 26 in which the imaging means comprises a charge coupled device operable to generate at wide-angle image comprising at least

1.5 megapixels.

28. A games system, comprising;

a control means according to claim 26, and;

a games machine operably coupled to the control means,

the games system comprising image processing means operable to map one or more regions of said wide-angle image to corresponding one or more flat image planes, hereafter referred to as facets.

29. A games system according to claim 28 in which the control means comprises an image processing means.

30. A games system according to claim 28 in which the games machine comprises an image processing means.

31. A games system according to claim 28 in which the image processing means is further operable to obtain image features from one or more of said facets of the mapped wide-angle image.

32. A games system according to claim 28 in which the image processing means is further operable to obtain any or all of the following image features from one or more of said facets of the mapped wide-angle image:

i. substantially linear edges;

ii. points where two or more linear edges meet, and;

iii. predetermined shapes formed by a sets of edges.

33. A games system according to claim 31 in which the image processing means is further operable to estimate a current orientation and/or position with respect to said image features.

34. A games system according to claim 31 in which the image processing means is further operable to estimate the positioning of at least part of the body of at least a first user based on said image features.

35. A games system according to claim 28 in which the control means is operably coupled to the games machine by wireless communications means.

38. A data carrier comprising computer readable instructions that, when loaded into a computer, cause the computer to operate as a control means according claim 26.

39. A data carrier comprising computer readable instructions that, when loaded into a computer, cause the computer to operate as a games machine of a games system according to claim 28.

41. A data signal comprising computer readable instructions that, when loaded into a computer, cause the computer to operate as a control means according to claim 26.

42. A data carrier comprising computer readable instructions that, when loaded into a computer, cause the computer to operate as a games machine of a games system according to claim 28.

44. Computer readable instructions that, when loaded into a computer, cause the computer to operate as a control means according to claim 26.