Entertainment device and method

• US 8,340,504 B2
• Filed: 04/26/2011
• Issued: 12/25/2012
• Est. Priority Date: 04/26/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Southworth, James Thomas
• Primary Examiner(s): Zhao, Daquan
• Application Number:
  US13/094,002
• Publication Number:
  US 20120275755A1
• Time in Patent Office: 609 Days
• Field of Search: 382/151, 382/216, 382/291, 382/293, 382/295, 386/239, 386/241, 386/248, 386/278, 386/280
• US Class Current: 386/280
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/4781 Games

Abstract

An entertainment device for generating augmented reality environments comprises an input for a captured video signal, an image analyser to detect a predetermined marker object location within the captured video signal, a graphics generator to generate a graphics object for integration with the video signal at a selected position to generate an augmented reality image, a controller to positionally couple the graphics object to the detected marker object location within the captured video signal, and a detection failure logic to signal when the image analyser cannot adequately detect the marker object'"'"'s location within the captured signal. If the logic signals the image analyser cannot detect the location of the marker object within the captured video signal, then the controller positionally decouples the graphics object from the last detected location of the marker object within the video signal, and instructs the graphics generator to modify a characteristic of the graphics object.

Claims

1. An entertainment device for generating augmented reality environments, comprising:

an input for a captured video signal;

an image analyser arranged to detect a location of a predetermined marker object within said captured video signal;

a graphics generator arranged to generate a graphics object for integration with the captured video signal at a selected position, to generate an augmented reality image;

a controller arranged to positionally couple the graphics object to the detected location of said predetermined marker object within said captured video signal; and

a detection failure logic arranged to signal when the image analyser cannot detect the location of said predetermined marker object within said captured video signal;

and in which

if the detection failure logic signals that the image analyser cannot adequately detect the location of said predetermined marker object within said captured video signal, then

the controller is arranged to positionally decouple said graphics object from a last detected location of said predetermined marker object within said captured video signal; and

the controller is arranged to instruct the graphics generator to modify a characteristic of the of the graphics object.

2. An entertainment device according to claim 1, in which the controller is arranged to positionally decouple said graphics object from the last detected location of said predetermined marker object after a predetermined threshold period.

3. An entertainment device according to claim 1, comprising

a motion detector;

and in which the controller is arranged to only positionally decouple said graphics object and instruct the graphics generator to modify a characteristic of the graphics object if in conjunction with a failure signal from the detection failure logic, the motion detector detects motion in the plane of the captured video signal exceeding a predetermined threshold.

4. An entertainment device according to claim 3, in which the motion detector comprises an accelerometer.

5. An entertainment device according to claim 3, in which the motion detector comprises an optical flow detector.

6. An entertainment device according to claim 3, in which said predetermined threshold represents a motion sufficient to move said predetermined marker object from its last detected location to a location with respect to said captured video signal where said image analyser is incapable of detecting the location of said predetermined marker object.

7. An entertainment device according to claim 3, in which the motion detector is operable to predict the location of said predetermined marker object in said captured video signal.

8. An entertainment device according to claim 7, in which the image analyser is operable to enhance detection sensitivity in a region responsive to the location of the predetermined marker object predicted by the motion detector.

9. An entertainment device according to claim 7, in which the image analyser is operable to limit detection to a region responsive to the location of the predetermined marker object predicted by the motion detector.

10. A failsafe method for an entertainment device, comprising the steps of:

receiving a captured video signal;

detecting a location of a predetermined marker object within said captured video signal;

generating a graphics object for integration with the captured video signal at a selected position, to generate an augmented reality image;

positionally coupling the graphics object to the detected location of said predetermined marker object within said captured video signal;

signalling a detection failure if the location of said predetermined marker object within said captured video signal cannot be adequately detected;

and if so signalled,

positionally decoupling said graphics object from a last detected location of said predetermined marker object within said captured video signal; and

modifying a characteristic of the of the graphics object.

11. A method according to claim 10, in which the steps of positionally decoupling said graphics object and modifying a characteristic of the of the graphics object are conditional upon a detection failure being signalled for a predetermined threshold period.

12. A method according to claim 10, in which the steps of positionally decoupling said graphics object and modifying a characteristic of the of the graphics object are conditional upon detecting motion of the entertainment device in the plane of the captured video signal that exceeds a predetermined threshold.

13. A method according to claim 10, in which said predetermined threshold represents a motion sufficient to move said predetermined marker object from its last detected location to a location with respect to said captured video signal where detection of the location of said predetermined marker object is no longer possible.

14. A method according to claim 10, comprising the step of predicting the location of said predetermined marker object in said captured video signal based upon detected motion of the entertainment device.

15. A method according to claim 14, comprising the step of enhancing detection sensitivity in a region responsive to the location of said predetermined marker object predicted by said detected motion of the entertainment device.

16. A method according to claim 14, in which the step of detecting a location of a predetermined marker object is limited to a region of the video signal image responsive to the location of the predetermined marker object predicted by said detected motion of the entertainment device.

17. A non-transitory storage medium comprising a computer program product on which computer readable instructions of a computer program are stored, the instructions, when executed by a processor, cause the processor to perform a failsafe method for an entertainment device, the method comprising:

receiving a captured video signal;

detecting a location of a predetermined marker object within said captured video signal;

generating a graphics object for integration with the captured video signal at a selected position, to generate an augmented reality image;

positionally coupling the graphics object to the detected location of said predetermined marker object within said captured video signal;

signalling a detection failure if the location of said predetermined marker object within said captured video signal cannot be adequately detected;

and if so signalled,

positionally decoupling said graphics object from a last detected location of said predetermined marker object within said captured video signal; and

modifying a characteristic of the of the graphics object.