Controller for electronic appliance

• US 20060061548A1
• Filed: 09/20/2005
• Published: 03/23/2006
• Est. Priority Date: 09/21/2004
• Status: Active Grant

• Current Assignee: JVC Kenwood Corporation
• Original Assignee: Victor Company of Japan Limited (JVC Kenwood Corporation)
• Inventors: Kitaura, Masahiro
• Granted Patent: US 7,629,959 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/47 End-user applications

Abstract

A controller controls an electronic appliance. The electronic appliance has a display. The controller employs a video camera to photograph an image of a user and converts the image into a mirror image. The controller generates an operational image including operation buttons, overlays the operational image on the user'"'"'s image, and displays the images on the display. In the displayed images, the user moves his or her hand onto one of the displayed operation buttons and bends the fingers of the hand to select the operation button. This selecting operation is detected by a detector of the controller, and information about the selected operation button is supplied to a processing unit, which outputs control signals corresponding to the selected operation button.

Claims

1. A controller for controlling an electronic appliance having a display, comprising:

a video camera configured to photograph an operator who is in front of the display;

a mirror image converter configured to convert an image of the operator photographed by the video camera into a mirror image and output a mirror image signal representative of the mirror image;

an operational image generator configured to generate an operational image signal representative of an operational image that is displayed on the display and is used by the operator to make the electronic appliance perform a desired operation;

a mixer configured to mix the mirror image signal with the operational image signal;

a detector configured to detect that the operator displayed on the display has pointed to the operational image displayed on the display, the operator and operational image being displayed on the display according to the mixed signals from the mixer; and

a controlling unit configured to detect whether or not the operator has carried out an operation with respect to the displayed operational image while the detector is indicating that the operator has pointed to the displayed operational image, and if detecting that the operator has carried out an operation with respect to the displayed operational image, control the electronic appliance according to an operation that is determined from a position of the operational image where the operator has pointed to,

the detector comprising;

a color filter configured to extract a specific-color area from the mirror image signal, the specific-color area corresponding to a hand of the operator pointing to the displayed operational image.

2. The controller of claim 1, wherein the detector further comprises:

a gradation limiter configured to extract a range of predetermined gradation levels from the mirror image signal; and

a synthesizer configured to extract the area corresponding to the hand of the operator according to the extracted specific-color area and the extracted range of predetermined gradation levels.

3. The controller of claim 2, wherein the detector further comprises:

a gate configured to replace the mirror image signal excluding the hand corresponding area with a given brightness or color.

4. A controller for controlling an electronic appliance having a display, comprising:

a video camera configured to photograph an operator who is in front of the display;

a mirror image converter configured to convert an image of the operator photographed by the video camera into a mirror image and output a mirror image signal representative of the mirror image;

an operational image generator configured to generate an operational image signal representative of an operational image that is displayed on the display and is used by the operator to make the electronic appliance perform a desired operation;

a mixer configured to mix the mirror image signal with the operational image signal;

a detector configured to detect that the operator displayed on the display has pointed to the operational image displayed on the display, the operator and operational image being displayed on the display according to the mixed signals from the mixer; and

a controlling unit configured to detect whether or not the operator has carried out an operation with respect to the displayed operational image while the detector is indicating that the operator has pointed to the displayed operational image, and if detecting that the operator has carried out an operation with respect to the displayed operational image, control the electronic appliance according to an operation that is determined from a position of the operational image where the operator has pointed to,

the detector comprising;

an extractor configured to extract, from the mirror image signal, an area corresponding to a hand of the operator pointing to the displayed operational image; and

a histogram detector configured to detect, in the extracted hand corresponding area, a histogram showing the frequencies of occurrence of gradation levels,

the controlling unit detecting in the detected histogram whether or not the operator has carried out an operation with respect to the displayed operational image.

5. The controller of claim 4, wherein:

the controlling unit detects whether or not the operator has carried out an operation with respect to the displayed operational image according to temporal changes in the detected histogram.

6. The controller of claim 4, wherein the extractor comprises:

a color filter configured to extract a specific-color area from the mirror image signal, the specific-color area corresponding to the hand of the operator pointing to the displayed operational image.

7. The controller of claim 6, wherein the extractor further comprises:

a gradation limiter configured to extract a range of predetermined gradation levels from the mirror image signal; and

a synthesizer configured to extract the hand corresponding area according to the extracted specific-color area and the extracted range of predetermined gradation levels.

8. The controller of claim 7, wherein the extractor further comprises:

a gate configured to replace the mirror image signal excluding the hand corresponding area with a given brightness or color.

9. A method of controlling an electronic appliance having a display, comprising:

photographing an operator who is in front of the display with a video camera;

converting an image of the operator photographed by the video camera into a mirror image and outputting a mirror image signal representative of the mirror image;

generating an operational image signal representative of an operational image that is displayed on the display and is used by the operator to make the electronic appliance perform a desired operation;

mixing the mirror image signal with the operational image signal;

detecting that the operator displayed on the display has pointed to the operational image displayed on the display by extracting a specific-color area from the mirror image signal, the specific-color area corresponding to a hand of the operator pointing to the displayed operational image, the operator and operational image being displayed on the display according to the mixed signals;

detecting whether or not the operator has carried out an operation with respect to the displayed operational image while it is being detected that the operator has pointed to the displayed operational image; and

controlling, if it is detected that the operator has carried out an operation with respect to the displayed operational image, the electronic appliance according to an operation that is determined from a position of the operational image where the operator has pointed to.

10. The method of claim 9, wherein the detecting that the operator displayed on the display has pointed to the operational image displayed on the display comprises:

extracting a range of predetermined gradation levels from the mirror image signal; and

extracting an area corresponding to the hand of the operator according to the extracted specific-color area and the extracted range of predetermined gradation levels.

11. The method of claim 10, wherein the detecting that the operator displayed on the display has pointed to the operational image displayed on the display further comprises:

replacing the mirror image signal excluding the hand corresponding area with a given brightness or color.

12. A method of controlling an electronic appliance having a display, comprising:

photographing an operator who is in front of the display with a video camera;

converting an image of the operator photographed by the video camera into a mirror image and outputting a mirror image signal representative of the mirror image;

generating an operational image signal representative of an operational image that is displayed on the display and is used by the operator to make the electronic appliance perform a desired operation;

mixing the mirror image signal with the operational image signal;

detecting that the operator displayed on the display has pointed to the operational image displayed on the display, the operator and operational image being displayed on the display according to the mixed signals;

detecting whether or not the operator has carried out an operation with respect to the displayed operational image while it is being detected that the operator has pointed to the displayed operational image; and

controlling, if it is detected that the operator has carried out an operation with respect to the displayed operational image, the electronic appliance according to an operation that is determined from a position of the operational image where the operator has pointed to,

the detecting that the operator displayed on the display has pointed to the operational image displayed on the display comprising;

extracting, from the mirror image signal, an area corresponding to a hand of the operator pointing to the displayed operational image; and

detecting, in the extracted hand corresponding area, a histogram showing the frequencies of occurrence of gradation levels,

the detecting whether or not the operator has carried out an operation with respect to the displayed operational image comprising;

detecting in the detected histogram whether or not the operator has carried out an operation with respect to the displayed operational image.

13. The method of claim 12, wherein:

the detecting whether or not the operator has carried out an operation with respect to the displayed operational image detects whether or not the operator has carried out an operation with respect to the displayed operational image according to temporal changes in the detected histogram.

14. The method of claim 12, wherein the extracting, from the mirror image signal, the area corresponding to the hand of the operator comprises:

extracting a specific-color area from the mirror image signal, the specific-color area corresponding to the hand of the operator pointing to the displayed operational image.

15. The method of claim 14, wherein the extracting, from the mirror image signal, the area corresponding to the hand of the operator further comprises:

extracting a range of predetermined gradation levels from the mirror image signal; and

extracting the hand corresponding area according to the extracted specific-color area and the extracted range of predetermined gradation levels.

16. The method of claim 15, wherein the extracting, from the mirror image signal, the area corresponding to the hand of the operator further comprises:

replacing the mirror image signal excluding the hand corresponding area with a given brightness or color.