ENTERTAINMENT DEVICE, SYSTEM, AND METHOD

• US 20120086729A1
• Filed: 04/30/2010
• Published: 04/12/2012
• Est. Priority Date: 05/08/2009
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Baseley, Nathan James, Doucet, Nicolas
• Granted Patent: US 8,933,968 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/8058 Virtual breeding, e.g. tamagotchi
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

An entertainment device generates a composite image with a combiner that combines camera-captured images with a computer-generated image of an object resting on a virtual surface. The device also includes a detector that detects image movement in the captured images in one or more contact point regions corresponding to image positions at which the object contacts the virtual surface. The device further comprises an initiator for initiating movement of the object to a new position with respect to the virtual surface in response to detected motion in the contact point regions. The detector detects whether a first image area corresponding to a captured image feature is greater than a predetermined proportion of a second image area corresponding to a full field of view of the camera. If the first image area is greater than the predetermined proportion, the initiator initiates movement of the object to an avoidance position.

Claims

1. An entertainment device for receiving images from a video camera, the device comprising:

a combiner operable to combine images captured by the camera with a computer-generated image of an object resting on a virtual surface so as to generate a composite image;

a detector operable to detect image movement in the captured images in one or more contact point regions corresponding to image positions at which the object contacts the virtual surface; and

an initiator operable to initiate movement of the object to a new position with respect to the virtual surface in response to detected motion in the contact point regions of the captured images,

in which;

the detector is operable to detect whether a first image area corresponding to a captured image feature is greater than a predetermined proportion of a second image area corresponding to a full field of view of the camera; and

in response to a detection that the first image area is greater than the predetermined proportion of the second image area, the initiator is operable to initiate movement of the object to an avoidance position within the composite image.

2. An entertainment device according to claim 1, in which the detector is operable to detect a direction of motion of the image movement in the captured images at the one or more contact points;

and

the new position lies on a line representing the detected direction of motion of the detected image movement, and which passes through the contact point region closest to an image region corresponding to the detected image movement.

3. An entertainment device according to claim 1, in which the new position is a predetermined distance from one of the contact point regions.

4. An entertainment device according to claim 1, in which:

the object comprises at least part of an interaction region associated with the object;

the detector is operable to detect image movement in the captured images at an image position corresponding to the interaction region; and

the initiator is operable to cause the object to perform a game action in response to detected motion in the interaction region.

5. An entertainment device according to claim 1, in which:

the detector is operable to detect a game state of the object; and

in response to detected motion at one or more of the contact point regions of the captured images, the initiator is operable to cause the object to perform a game response action which is dependent on the detected game state of the object.

6. An entertainment device according to claim 1, in which:

the detector is operable to detect static regions of the virtual image plane, the static regions corresponding to image positions which correspond to positions on the virtual surface at which no image movement is detected in the captured images; and

the initiator is operable to set the new position so that it is located at an image position corresponding to a static region.

7. An entertainment device according to claim 1, in which the image feature comprises inter-image movement.

8. An entertainment device according to claim 1, in which:

the detector is operable to store a background image representing an image captured by the camera in a calibration operation, and to subtract image pixels of the background image from image pixels of a current captured image so as to generate a foreground image; and

the image feature comprises the foreground image.

9. An entertainment device according to claim 1, in which the initiator is operable to apply a zoom operation to the object when the object moves to the avoidance position.

10. An entertainment device according to claim 1, in which the avoidance position is a forward image position corresponding to a lower edge of the captured images.

11. An entertainment device according to claim 1, in which one or more of the contact point regions include a region of the virtual surface surrounding the image positions at which the object contacts the virtual surface.

12. An entertainment device according to claim 1, in which the object is a virtual creature.

13. An entertainment device according to claim 1, in which the initiator is operable to generate the new position so that the new position is at a random position with respect to the virtual surface.

14. An entertainment system comprising:

an entertainment device including;

a combiner operable to combine images captured by the camera with a computer-generated image of an object resting on a virtual surface so as to generate a composite image;

a detector operable to detect image movement in the captured images in one or more contact point regions corresponding to image positions at which the object contacts the virtual surface; and

an initiator operable to initiate movement of the object to a new position with respect to the virtual surface in response to detected motion in the contact point regions of the captured images,

in which;

the detector is operable to detect whether a first image area corresponding to a captured image feature is greater than a predetermined proportion of a second image area corresponding to a full field of view of the camera; and

in response to a detection that the first image area is greater than the predetermined proportion of the second image area, the initiator is operable to initiate movement of the object to an avoidance position within the composite image;

a video camera operable to transmit images captured by the camera to the entertainment device via a communications link; and

a display operable to display the composite image generated by the entertainment device.

15. An entertainment method comprising:

receiving images from a video camera;

combining a computer-generated image of an object resting on a virtual surface with images captured by the camera so as to generate a composite image;

detecting image movement in the captured images in one or more contact point regions corresponding to image positions at which the object contacts the virtual surface;

initiating movement of the object to a new position with respect to the virtual surface in response to detected motion in the contact point regions of the captured images;

detecting whether a first image area corresponding to a captured image feature is greater than a predetermined proportion of a second image area corresponding to a full field of view of the camera; and

initiating movement of the object to an avoidance position within the composite image in response to a detection that the first image area is greater than the predetermined proportion of the second image area.

16. A tangible, non-transitory computer readable storage medium storing a computer program, the computer program, when executed by a processor, causing the processor to implement an entertainment, the method comprising:

receiving images from a video camera;

combining a computer-generated image of an object resting on a virtual surface with images captured by the camera so as to generate a composite image;

detecting image movement in the captured images in one or more contact point regions corresponding to image positions at which the object contacts the virtual surface;

initiating movement of the object to a new position with respect to the virtual surface in response to detected motion in the contact point regions of the captured images;

detecting whether a first image area corresponding to a captured image feature is greater than a predetermined proportion of a second image area corresponding to a full field of view of the camera; and

initiating movement of the object to an avoidance position within the composite image in response to a detection that the first image area is greater than the predetermined proportion of the second image area.