IMAGE PROCESSOR, IMAGE PROCESSING METHOD, RECORDING MEDIUM, COMPUTER PROGRAM AND SEMICONDUCTOR DEVICE

• US 20020097247A1
• Filed: 10/05/2001
• Published: 07/25/2002
• Est. Priority Date: 10/06/2000
• Status: Active Grant

• Current Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Ohba, Akio
• Granted Patent: US 6,771,277 B2
• Time in Patent Office: Days
• US Class Current: 345/501
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Abstract of Disclosure

The present invention provides an image processor to use images taken by an image pickup device, etc. as an input interface to enter commands, etc. The present invention provides an image processor that includes image capturing means for capturing a mirrored moving image partially including a moving target, image generating means for generating an object image expressing a predetermined object according to the movement of the target included in the mirrored moving image captured by the image capturing means and controlling means for combining the object image generated by this image generating means with the captured mirrored moving image and displaying the combined image on a predetermined display device.

Claims

1. An image processor comprising:

image capturing means for capturing a mirrored moving image at least partially including a moving target;

image generating means for generating an object image representing a predetermined object according to the movement of the target included in the mirrored moving image; and

controlling means for combining the object image generated by said image generating means with said captured mirrored moving image and for displaying said combined image on a predetermined display device.

2. An image processor comprising:

image capturing means for capturing a mirrored moving image in a time series, said mirrored moving image at least partially including a moving target;

detecting means for detecting said target and a movement component thereof by detecting image features between a mirrored moving image at an actual time point and an immediately preceding mirrored moving image;

image generating means for generating an object image representing a predetermined object in such a way as to change according to said movement component of the target detected by said detecting means; and

controlling means for combining said object image generated by the image generating means with said captured mirrored moving image and for displaying said combined image on a predetermined display device.

3. The image processor according to claim 2, wherein said image generating means is operable to generate said object image in such a way as to follow said movement of said detected target.

4. The image processor according to claim 2, further comprising means for making preparations for executing required processing based on said generated object image according to said movement component of said target.

5. The image processor according to claim 2, further comprising means for comparing a combined image obtained by combining said object image generated by said image generating means and said mirrored moving image at the actual time point with a template image, which is an image of part of said target included in said immediately preceding mirrored moving image, for detecting a part of said combined image whose image features most resemble said template image, and for making preparations for executing required processing based on said object image when the image of said part of said combined image includes said object image.

6. The image processor according to claim 2, further comprising means for executing predetermined processing which is linked to said object image when said movement component of said target detected by said detecting means satisfies predetermined conditions.

7. The image processor according to claim 2, wherein:

said mirrored moving image includes a plurality of said targets,said detecting means is operable to detect said movement components of said plurality of targets and to detect one target based on respective movement components of said detected plurality of targets, and said image generating means is operable to change said object image according to said movement component of said one target detected by said detecting means.

8. An image processing method, comprising:

capturing a mirrored moving image at least partially including a moving target in an image processor;

generating an object image representing a predetermined object using said image processor according to a movement of said target included in said captured mirrored moving image; and

combining said generated object image with said captured mirrored moving image and displaying said combined image on a predetermined display device.

9. A recording medium containing a computer program for causing a computer with a display device connected thereto to execute steps comprising:

capturing a mirrored moving image at least partially including a moving target;

generating an object image representing a predetermined object according to movement of said target included in said captured mirrored moving image; and

combining said object image with said captured mirrored moving image and displaying said combined image on said display device.

10. (Cancelled)

11. A semiconductor device for causing a computer with a display device connected thereto to perform functions comprising:

capturing a mirrored moving image at least partially including a moving target;

generating an object image representing a predetermined object according to movement of the target included in said captured mirrored moving image; and

combining said object image with said captured mirrored moving image and displaying the combined image on said display device.

12. A graphics processor, comprising:

an image input device operable to capture image frames from a camera in a time series such that said image frames include at least some features of a user of the graphics processor;

an image inverter operable to produce mirrored image frames from said captured image frames, one or more of said features of said mirrored image frames being at least part of a target;

an object image generator operable to produce object frames, said object frames representing one or more predetermined objects for display;

a superimposing image generator operable to combine said mirrored image frames and said object image frames to produce combined image frames;

a detector operable to determine movement of said target from frame to frame and to quantify said movement; and

a controller operable to cause adjustments to said object frames in response to said quantification of said movement of said target and for displaying said combined image frames on a display device.

13. The graphics processor of claim 12, wherein said objects include menu indicia;

and said controller is operable to determine whether said movement of said target indicates a desire by said user to select said menu indicia based on said quantification of said movement of said target by said detector, and to execute one or more further graphics processes based on said menu selection made by said user.

14. The graphics processor of claim 12, wherein said objects include a cursor;

and said controller is operable to determine whether said movement of said target indicates a desire by said user to at least one of acquire and manipulate said cursor based on said quantification of said movement of said target by said detector, and to execute one or more further graphics processes based on manipulation of said cursor by said user.

15. The graphics processor of claim 14, wherein said target includes said cursor when said user has acquired said cursor.

16. A method, comprising:

capturing image frames from a camera in a time series such that said image frames include at least some features of a user of a graphics processor;

producing mirrored image frames from said captured image frames, one or more of said features of said mirrored image frames being at least part of a target;

producing object frames, said object frames representing one or more predetermined objects for display;

superimposing said mirrored image frames with said object image frames to produce combined image frames;

determining movement of said target from frame to frame and quantifying said movement;

adjusting said object frames in response to said quantification of said movement of said target; and

displaying said combined image frames on a display device.

17. The method of claim 16, wherein said objects include menu indicia;

and said method further includes determining whether said movement of said target indicates a desire by said user to select said menu indicia based on said quantification of said movement of said target, and executing one or more further graphics processes based on said menu selection made by said user.

18. The method of claim 16, wherein said objects include a cursor;

and said method further includes determining whether said movement of said target indicates a desire by said user to at least one of acquire and manipulate said cursor based on said quantification of said movement of said target, and executing one or more further graphics processes based on manipulation of said cursor by said user.

19. The method of claim 18, wherein said target includes said cursor when said user has acquired said cursor.