System and method for control based on face or hand gesture detection

• US 10,582,144 B2
• Filed: 05/28/2013
• Issued: 03/03/2020
• Est. Priority Date: 05/21/2009
• Status: Active Grant

• Current Assignee: May Patents Ltd.
• Original Assignee: May Patents Ltd.
• Inventors: Binder, Yehuda
• Primary Examiner(s): Bukowski, Kenneth
• Application Number:
  US13/903,569
• Publication Number:
  US 20130258113A1
• Time in Patent Office: 2,471 Days
• Field of Search: 345156-158, 345175, 34820814, 382103
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G09G 2330/027 Arrangements or methods related to powering off a display
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/10 Intensity circuits
• CPC Class Codes: H04B 1/38 Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving
• CPC Class Codes: H04M 3/02 Calling substations, e.g. by ringing selective calling H04Q
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/43632 : involving a wired protocol, e.g. IEEE 1394 high-speed IEEE 1394 serial bus H04L12/40052
• CPC Class Codes: H04N 21/43635 : HDMI
• CPC Class Codes: H04N 21/44008 : involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 23/51 : Housings
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 25/75 : Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/63 : Generation or supply of power specially adapted for television receivers
• CPC Class Codes: H04N 5/645 : Mounting of picture tube on chassis or in housing
• CPC Class Codes: H04N 7/005 : using at least one opto-electrical conversion device
• CPC Class Codes: H04N 7/015 : High-definition television systems
• CPC Class Codes: H04N 7/0255 : Display systems therefor
• CPC Class Codes: H04N 7/104 : Switchers or splitters
• CPC Class Codes: H04N 7/163 : by receiver means only
• CPC Class Codes: H04N 9/3141 : Constructional details thereof details not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04W 88/02 : Terminal devices

Abstract

System and method for control using face detection or hand gesture detection algorithms in a captured image. Based on the existence of a detected human face or a hand gesture in an image captured by a digital camera, a control signal is generated and provided to a device. The control may provide power or disconnect power supply to the device (or part of the device circuits). The location of the detected face in the image may be used to rotate a display screen to achieve a better line of sight with a viewing person. The difference between the location of the detected face and an optimum is the error to be corrected by rotating the display to the required angular position. A hand gesture detection can be used as a replacement to a remote control for the controlled unit, such as a television set.

Claims

1. A television set responsive to an element in an image, the television set comprising:

a flat screen for displaying television channels;

a digital camera for capturing an image, said digital camera having an output port, and being operative to transmit a digital data form of the captured image via said output port, said digital camera being fixed in position relative to said screen to be movable in unison with said screen and being oriented to capture an image of a scene substantially in front of the screen;

an image processor coupled to receive the image in the digital data from said digital camera and for applying an algorithm to detect the element in the captured image;

an antenna for over-the-air radio-frequency communication;

a wireless transceiver coupled between said digital camera output port and said antenna for transmitting the digital data form of the captured image over the air; and

a single enclosure housing said flat screen, said digital camera, said image processor, said antenna and said wireless transceiver,

wherein information is displayed on said flat screen in response to the detection of the element in the captured image.

2. The television set according to claim 1, wherein said digital camera comprises:

an optical lens for focusing received light, a photosensitive image sensor array disposed approximately at an image focal point plane of the optical lens for capturing the image and producing an analog form of the captured image; and

an analog-to-digital (A/D) converter coupled to said image sensor for generating the digital data form of the captured image.

3. The television set according to claim 2 wherein said image sensor array is based on multiple Charge-Coupled Devices (CCD) or Complementary Metal-Oxide-Semiconductor (CMOS) based components.

4. The television set according to claim 1, wherein the element is an image of a human face, and said image processor applies a face detection algorithm to detect an existence and a position of the human face in the captured image.

5. The television set according to claim 1, wherein the element is an image of a human hand, and said image processor applies a hand gesture algorithm to detect an existence and a position of a hand gesture in the captured image.

6. The television set according to claim 1, wherein said image processor applies a face detection algorithm to detect an existence and a position of a human face in the captured image.

7. The television set according to claim 6, wherein the image processor further applies a hand gesture detection algorithm of a hand gesture that is based on the position of the human face detected in the captured image.

8. The television set according to claim 5, wherein the hand gesture includes extending a single finger.

9. The television set according to claim 5, wherein the hand gesture includes extending multiple fingers.

10. The television set according to claim 5, wherein the hand gesture includes extending all fingers of one hand.

11. The television set according to claim 1, further comprising a timer associated with a pre-set time period, and wherein said screen is blanked in response to not detecting the element in the captured image during the pre-set time period.

12. The television set according to claim 1, wherein the information is displayed on said screen in response to a detected position of the detected element in the captured image.

13. The television set according to claim 1, further responsive to a deviation between the position of the detected element in the captured image and a center of the image.

14. The television set according to claim 13, wherein said transceiver is further coupled to said digital camera for transmitting the captured image over the network medium.

15. The television set according to claim 1, further connected for displaying multiple television channels, and wherein a television channel is selected to be displayed on said screen in response to the detection of the element in the captured image.

16. The television set according to claim 1, wherein said image processor comprises firmware or software and a processor to execute said firmware or software.

17. The television set according to claim 1, wherein a power is supplied to a load in response to the detection of the element in the captured image.

18. The television set according to claim 17, further comprising a switch connected to be actuated in response to the detection of the element in the captured image.

19. The television set according to claim 1, wherein the over-the-air radio frequency communication uses a license-free frequency band.

20. The television set according to claim 19, wherein the license-free frequency band is an ISM band.

21. The television set according to claim 20, wherein the ISM band is 5 GHz or 2.4 GHz.

22. The television set according to claim 1, wherein:

the over-the-air radio frequency communication is over a Wireless Personal Area Network (WPAN);

said antenna is a WPAN antenna; and

said wireless transceiver is a WPAN transceiver.

23. The television set according to claim 22, wherein the WPAN is according to, or based on, Bluetooth™

, IEEE 802.15, Ultra-Wide-Band (UWB), or ZigBee™

.

24. The television set according to claim 1, wherein:

the over-the-air radio frequency communication is over a Wireless Local Area Network (WLAN);

said antenna is a WLAN antenna; and

said wireless transceiver is a WLAN transceiver.

25. The television set according to claim 24, wherein the WLAN is according to, or based on, IEEE 802.11a, IEEE 802.11b, or IEEE 802.11g.

26. The television set according to claim 1, wherein:

the over-the-air radio frequency communication is over a Wireless Wide Area Network (WWAN);

said antenna is a WWAN antenna; and

said wireless transceiver is a WWAN transceiver.

27. The television set according to claim 1 wherein:

the over-the-air radio frequency communication is a cellular communication;

said antenna is a cellular antenna; and

said wireless transceiver is a cellular transceiver.

28. The television set according to claim 27, wherein the cellular communication is according to, or based on, GSM (Global Television set for Mobile Communications), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), EDGE (Enhanced Data Rates for GSM Evolution), 3GSM, DECT (Digital Enhanced Cordless Telecommunications), Digital AMPS or iDEN (Integrated Digital Enhanced Network).

29. The television set according to claim 1, wherein:

the over-the-air radio frequency communication uses a 60 GHz license-free frequency band and is used for in-room communication.

30. The television set according to claim 29, wherein the communication is according to, or based on, WirelessHD™

.

31. The television set according to claim 1, further comprising an image or video compressor coupled between said digital camera output port and said wireless transceiver, for compressing the captured image digital data according to a compression scheme.

32. The television set according to claim 31, wherein the compression scheme is a lossy or lossless type.

33. The television set according to claim 31, wherein the compression scheme is according to, or based on, JPEG (Joint Photographic Experts Group) and MPEG (Moving Picture Experts Group) standard.

34. The television set according to claim 1, wherein said flat screen is silicon-based.

35. The television set according to claim 34 wherein said flat screen is LCD (Liquid Crystal Display) or TFT (Thin-Film Transistor) based.

36. The television set according to claim 1, further operative for displaying High Definition (HD), and wherein said television set further comprises an HDMI (High-Definition Multimedia Interface) for receiving and displaying HD video.

37. The television set according to claim 1, wherein only the information displayed on said flat screen is responsive to the detection of the element in the captured image.

38. A television set operative to receive and display television channels on a screen, the television set further operative for detecting distinct first and second elements and responding to the first and second elements location in an image, the television set comprising a single enclosure comprising:

a digital camera for capturing the image in a digital data form;

an image processor coupled to receive the image from said digital camera and for executing an element detection algorithm to detect the first and second elements in the captured image and to determine the elements location in the image; and

a second processor and firmware or software executable by said second processor, said second processor being coupled to control said image processor and said digital camera and being operative for generating a control signal in response to the locations of the first and second elements.

39. The television set according to claim 38, wherein said image processor is further operative to calculate the average location of the first and second elements, and wherein the television set is operative for generating the control signal in response to the center location of the locations of the first and second elements.

40. The television set according to claim 38, wherein the image is a non-visible image.

41. The television set according to claim 40, wherein the non-visible image is in the infrared or ultraviolet spectrum.

42. The television set according to claim 38, wherein the elements are a body parts.

43. The television set according to claim 42, wherein the body parts are human hands or human faces.

44. The television set according to claim 38, wherein the elements are hand gestures.

45. The television set according to claim 44, wherein the hand gestures include extending a single finger, extending multiple fingers, or extending all fingers of one hand.

46. The television set according to claim 38, further comprising a switch coupled to be actuated by the control signal.

47. The television set according to claim 46, wherein said switch is connected between a power source and an apparatus, for powering the apparatus in response to the control signal.

48. The television set according to claim 47, wherein the apparatus is external to said single enclosure.

49. The television set according to claim 38, wherein said image processor consists of, or comprises, a non-transitory computer readable medium storing an image processing algorithm for detecting the elements in the captured image.

50. The television set according to claim 38, further comprising a communication port coupled to said second processor for coupling to a network medium;

and a transceiver coupled to said communication port for transmitting the control signal to the network medium.

51. The television set according to claim 38, wherein said digital camera comprises:

an optical lens for focusing received light, said lens being mechanically oriented to guide the image;

a photosensitive image sensor array disposed at least approximately at an image focal point plane of the optical lens for capturing the image and producing an analog signal representing the image; and

an analog-to-digital (A/D) converter coupled to said image sensor array for converting the analog signal to a digital data representation of the image.

52. The television set according to claim 51, wherein said image sensor array is based on Charge-Coupled Devices (CCD) or Complementary Metal-Oxide-Semiconductor (CMOS) components.