SYSTEM AND METHOD FOR CONTROL BASED ON FACE ORE HAND GESTURE DETECTION

• US 20100295782A1
• Filed: 03/16/2010
• Published: 11/25/2010
• Est. Priority Date: 05/21/2009
• Status: Abandoned Application

• Current Assignee: May Patents Ltd.
• Original Assignee: May Patents Ltd.
• Inventors: Binder, Yehuda
• Application Number:
  US12/724,896
• Publication Number:
  US 20100295782A1
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G09G 2330/027 Arrangements or methods related to powering off a display
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/10 Intensity circuits
• CPC Class Codes: H04B 1/38 Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving
• CPC Class Codes: H04M 3/02 Calling substations, e.g. by ringing selective calling H04Q
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/43632 : involving a wired protocol, e.g. IEEE 1394 high-speed IEEE 1394 serial bus H04L12/40052
• CPC Class Codes: H04N 21/43635 : HDMI
• CPC Class Codes: H04N 21/44008 : involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4788 : communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 23/51 : Housings
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 25/75 : Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/63 : Generation or supply of power specially adapted for television receivers
• CPC Class Codes: H04N 5/645 : Mounting of picture tube on chassis or in housing
• CPC Class Codes: H04N 7/005 : using at least one opto-electrical conversion device
• CPC Class Codes: H04N 7/015 : High-definition television systems
• CPC Class Codes: H04N 7/0255 : Display systems therefor
• CPC Class Codes: H04N 7/104 : Switchers or splitters
• CPC Class Codes: H04N 7/163 : by receiver means only
• CPC Class Codes: H04N 9/3141 : Constructional details thereof details not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04W 88/02 : Terminal devices

Abstract

System and method for control using face detection or hand gesture detection algorithms in a captured image. Based on the existence of a detected human face or a hand gesture in an image captured by a digital camera, a control signal is generated and provided to a device. The control may provide power or disconnect power supply to the device (or part of the device circuits). The location of the detected face in the image may be used to rotate a display screen to achieve a better line of sight with a viewing person. The difference between the location of the detected face and an optimum is the error to be corrected by rotating the display to the required angular position. A hand gesture detection can be used as a replacement to a remote control for the controlled unit, such as a television set.

Claims

1. A computer implemented method for improving the angular field of view for a person watching an image on a screen, the method comprising:

(a) capturing an image of a scene in front of the screen;

(b) converting the image into a digital data form;

(c) detecting the position of an image of a human face in the captured image;

(d) calculating at least one deviation between the detected position of the human face image in the captured image and an imaginary line that passes through the center of the image on the screen and extends substantially perpendicular to the screen; and

(e) rotating the screen in a direction to reduce the calculated deviation.

2. The method according to claim 1, further comprising repeatedly performing steps (a) to (e) until the calculated deviation is smaller than a predefined value.

3. The method according to claim 1, wherein steps (a) to (e) are executed repeatedly to form a linear feedback control loop, wherein the calculated deviation serves as a control loop error, the control loop has a set point equal to zero, and the rotation of the display is controlled by the loop.

4. The method according to claim 3, wherein said method defines a linear proportional control loop, wherein the amount of rotation of the screen is proportional to the calculated deviation.

5. The method according to claim 3, wherein said method defines a PID (Proportional, Integral and Derivative) control loop, the amount of rotation of the screen is calculated based on proportional, integral and derivative computations of the calculated deviation.

6. The method according to claim 1, wherein:

the deviation is horizontal, the deviation is calculated in the captured image, and the rotation of the display is in the horizontal direction.

7. The method according to claim 1, wherein the deviation is vertical, the deviation is calculated in the captured image, and the rotation of the display is in the vertical direction.

8. The method according to claim 1, wherein the at least one deviation includes a horizontal deviation and a vertical deviation, the horizontal and vertical deviations are calculated in the captured image, and said rotation of the display includes rotating in the horizontal direction in response to the horizontal deviation and rotating in the vertical direction in response to the vertical deviation.

9. The method according to claim. 1, wherein if no human face image is detected is step (c), no rotation is performed.

10. The method according to claim 1, wherein if two or more human faces are detected in the captured image, then the average of the positions of the detected faces is calculated, and in step (d) the deviation is calculated between the average and the image center.

11. An apparatus for improving the angular field of view for a person watching a display on a screen, said apparatus comprising:

a digital camera for capturing an image in digital data form, said camera being fixed in position relative to the screen and oriented to capture an image of the scene substantially in front of the screen;

an image processor coupled to receive the image in digital data form from said digital camera, for applying face detection algorithm to detect and locate an image of a human face in the captured image; and

a motor mechanically attached to the screen for angularly rotating the screen;

wherein said apparatus is operative to rotate said motor in response to the location of the detected image of a face in the captured image.

12. The apparatus according to claim 11, further comprising firmware or software and a controller to execute said firmware or software coupled between said digital camera and said motor for controlling rotation of the screen in response to the location of the detected image of a face in the captured image.

13. The apparatus according to claim 11, wherein said motor is a stepper motor.

14. The apparatus according to claim 11, further operative to calculate the deviation between the detected face image location and the image center, and wherein the angular rotation of the screen is based on the calculated deviation.

15. The apparatus according to claim 14, wherein no motor rotation is executed when the calculated deviation is smaller than a predefined value.

16. The apparatus according to claim 14, wherein said apparatus is operative to continuously rotate said motor in response to the location of the detected face image in the captured image in a linear feedback control loop, wherein the error used by the control loop is the calculated deviation, the set point of the control loop is zero and the angular rotation of the screen is the actuator controlled by the loop.

17. The apparatus according to claim 16, wherein the control loop is a linear proportional control loop, and the amount of angular rotation of the screen is proportional to the calculated deviation.

18. The apparatus according to claim 16, wherein the control loop is a PID (Proportional, Integral and Derivative) control loop, and the amount of angular rotation is calculated based on proportional, integral and derivative computations of the calculated deviation.

19. The apparatus according to claim 16, wherein the deviation is horizontal and is calculated in the captured image, and said motor is attached to effect rotation of the screen in a horizontal plane.

20. The apparatus according to claim 19, further comprising a second motor attached to effect rotation of the screen in a vertical plane, wherein a vertical deviation is calculated in the captured image, and wherein said second motor is attached to effect screen rotation in the vertical plane in response to the calculated vertical deviation.

21. The apparatus according to claim 16, wherein the deviation is vertical and is calculated in the captured image, and said motor is attached to effect rotation of the screen in the vertical plane.

22. The apparatus according to claim 11, wherein if no human face is detected by the image processor, no motor rotation is executed.

23. The apparatus according to claim 14, wherein if two or more human faces are detected in the captured image, then said image processor calculates the average of the positions of the detected face images and the deviation between that average and the image center.

24. A computer implemented method for controlling a device based on face detection, the method comprising the steps of

(a) capturing an image of a scene;

(b) converting the image into a digital data form;

(c) detecting an image of a human face in the captured image; and

(d) providing a first control signal associated with the detection of the human face image in the captured image.

25. The method according to claim 24, further comprising performing steps (a) to (d) repeatedly.

26. The method according to claim 25, further comprising waiting a pre-set period before repeating the steps (a) to (d).

27. The method according to claim 24, wherein step (d) comprises supplying power to the device is response to the detection of a human face image in the captured image.

28. The method according to claim 24, wherein step (d) comprises disconnecting power to the device is response to not detecting a human face image in the captured image.

29. The method according to claim 24, wherein said device comprises a display screen.

30. The method according to claim 29, wherein the image captured is of a scene substantially in front of the display screen.

31. The method according to claim 29, wherein in step (d) the display screen is blanked in response to not detecting a human face in said captured image.

32. The method according to claim 24, wherein in step (d) the control signal is generated in response to detecting a human face image in the captured image over a pre-defined period.

33. The method according to claim 24, wherein in step (d) the control signal is generated in response to not detecting a human face in said captured image over a pre-defined period.

34. The method according to claim 24, wherein the first control signal is generated in response to detecting a human face image in the captured image over a first pre-defined period, and said method further comprises generating a second control signal in response to not detecting a human face image in the captured image over a second pre-defined period.

35. The method according to claim 34, wherein the first control signal is operative to supply power to at least part of the device, and the second control signal is operative to disconnect power to at least part of the device.

36. The method according to claim 35, wherein the first control signal is operative to supply power to a selected part of the device, and the second control signal is operative to disconnect power to the selected part of the device.

37. An apparatus for face detection based control of a device, said apparatus comprising:

a digital camera for capturing an image in a digital data form;

an image processor coupled to receive the image in a digital data form from said digital camera, for applying a face detection algorithm to detect the position of a human face image in the captured image; and

a controller coupled to said image processor for generating a control signal in response to the detection or non-detection of a human face image in the captured image.

38. The apparatus according to claim 37, further comprising firmware or software executed by said controller.

39. The apparatus according to claim 37, wherein said camera is mechanically fixed to the device.

40. The apparatus according to claim 37, wherein said camera, said image processor and said controller are housed within a single enclosure.

41. The apparatus according to claim 37, further comprising a switch connected to be actuated by the control signal.

42. The apparatus according to claim 41, wherein said switch is connected between a power source and the device, for powering the device in response to the control signal.

43. The apparatus according to claim 42, operative to actuate said switch and to supply power to the device is response to the detection of a human face image in the captured image.

44. The apparatus according to claim 42, operative to actuate said switch and for supplying power to the device is response to not detecting a human face in said captured image.

45. The apparatus according to claim 41, wherein said camera, said image processor, said controller and said switch are housed within a single enclosure.

46. The apparatus according to claim 37, further comprising a first timer for signaling a pre-set first time period, said first timer being operatively associated with said controller to cause the control signal to be generated after elapse of the first time period following detecting a human face in said captured image for a first pre-defined period.

47. The apparatus according to claim 46, further comprising a second timer for signaling a pre-set second time period, said second timer being operatively associated with said controller to cause said controller to generate a second control signal in response to not detecting a human face image in the captured image for the second time period.

48. The apparatus according to claim 47, wherein the first-recited control signal is effective to initiate supplying of power to the device, and the second control signal is effective to disconnect power from the device.

49. The method according to claim 48 wherein the first-recited control signal is effective to initiate supplying of power to part of the device, and the second control signal is effective to disconnect power from the part of the device.

50. The apparatus according to claim 37, further comprising a first timer for signaling a pre-set first period coupled or within said controller, and the control signal is generated in response to not detecting a human face in said captured image for the first pre-defined period.

51. The apparatus according to claim 37, wherein said device is a display or a television set.

52. The apparatus according to claim 37, wherein the device has a display screen and said camera is positioned such that the image captured is substantially in front of the display screen.

53. The apparatus according to claim 37, wherein the device has a display, and said apparatus further comprises a timer associated with said controller for signaling a pre-set time period, and said controller is operative to blank the display in response to not detecting a human face image in the captured image.

54. A computer implemented method for controlling a device based on hand gesture detection, said method comprising:

(a) capturing an image with a camera;

(b) electronically converting the image into a digital data form;

(c) detecting a hand gesture image in said captured image by use of a computer running a detecting algorithm; and

(d) producing a control signal in response to the detection of a hand gesture image in the captured image.

55. The method according to claim 54, further comprising repeating steps (a) to (d).

56. The method according to claim 55, further comprising waiting a pre-set time period before repeating steps (a) to (d).

57. The method according to claim 54, wherein step (d) comprises supplying power to the device is response to the detection of a hand gesture image in the captured image.

58. The method according to claim 54, wherein step (d) comprises disconnecting power to the device in response to detecting a hand gesture face in the captured image.

59. The method according to claim 54, wherein the device is a display or a television set.

60. The method according to claim 54, wherein the device has a display screen, and the image captured is substantially of a scene facing the display screen.

61. The method according to claim 60, wherein in step (d) the display screen is blanked in response to not detecting a hand gesture image in the captured image.

62. The method according to claim 54, wherein in step (d) the control signal is generated in response to detecting a hand gesture image in the captured image for a pre-defined time period.

63. The method according to claim 62, further comprising producing a second control signal in response to not detecting a hand gesture image in the captured image for a second pre-defined time period.

64. The method according to claim 63 wherein the first-recited control signal is effective to initiate supplying of power to the device, and the second control signal is effective to disconnect power from the device.

65. The method according to claim 64 wherein the first-recited control signal is effective to initiate supplying of power to part of the device, and the second control signal is effective to disconnect power from the part of the device.

66. The method according to claim 54, wherein the detected hand gesture includes extending a single finger.

67. The method according to claim 54, wherein the detected hand gesture includes extending multiple fingers.

68. The method according to claim 54, wherein the detected hand gesture includes extending all fingers of a hand.

69. The method according to claim 54, wherein only a single pre-defined hand gesture can be detected.

70. The method according to claim 54, the detected hand gesture is any one of a plurality of pre-defined hand gestures.

71. The method according to claim 70, further comprising associating a respective dedicated control with each detected hand gesture.

72. The method according to claim 54, further comprising using an image processing algorithm to detect a human face image in the captured image.

73. The method according to claim 72, wherein the control signal is produced only in response to the detection of the hand gesture image and detection of a human face image in the captured image.

74. An apparatus for hand gesture detection based control of a device, said apparatus comprising:

a digital camera for capturing an image in a digital data form;

an image processor coupled to receive said image in a digital data form from said digital camera, for running a hand gesture detection algorithm to detect a hand gesture image in the captured image; and

a controller coupled to said image processor for generating a control signal in response to the detection or non-detection of a hand gesture image in the captured image.

75. The apparatus according to claim 74, further comprising firmware or software to be executed by said controller.

76. The apparatus according to claim 74, wherein said camera is mechanically fixed to the device.

77. The apparatus according to claim 74, wherein said camera, said image processor and said controller are housed within a single enclosure.

78. The apparatus according to claim 74, further comprising a switch coupled to be actuated by the control signal.

79. The apparatus according to claim 78, wherein said switch is connected between a power source and the device, for powering the device in response to the control signal.

80. The apparatus according to claim 79, operative to actuate said switch and to supply power to the device in response to the detection of a hand gesture image in the captured image.

81. The apparatus according to claim 78, wherein said camera, said image processor, said controller and said switch are housed within a single enclosure.

82. The apparatus according to claim 74, further comprising a first timer associated with said controller for signaling a pre-set first time period, and wherein the control signal is generated in response to detecting a hand gesture image in the captured image for the first pre-defined time period.

83. The apparatus according to claim 74, further comprising a first timer for signaling a pre-set first period coupled or within said controller, and the control signal is generated in response to not detecting a hand gesture in said captured image for a first pre-defined period.

84. The apparatus according to claim 82, further comprising a second timer associated with said controller for signaling a pre-set second time period, wherein the first-recited control signal is generated in response to not detecting a hand gesture image in the captured image for a first pre-defined time period, and a second control signal is generated in response to detecting a hand gesture image in the captured image for a second pre-defined time period.

85. The apparatus according to claim 84, wherein the second control signal is effective to supply power to at least part of the device, and the first control signal involves disconnecting power to at least part of the device.

86. The method according to claim 84 wherein said second control signal is effective to supply power to part of the device, and the first control signal is effective to disconnect power from the part of the device.

87. The apparatus according to claim 74, wherein the device is a display or a television set.

88. The apparatus according to claim 74, wherein the device has a display screen, and said camera is positioned such that the image captured by said camera is of a scene substantially facing the display screen.

89. The apparatus according to claim 88, further comprising a timer associated with said controller for signaling a pre-set time period, and the display screen is blanked in response to not detecting a hand gesture image in the captured image.

90. The apparatus according to claim 74, wherein the hand gesture includes extending a single finger.

91. The apparatus according to claim 74, wherein the hand gesture includes extending of multiple fingers.

92. The apparatus according to claim 74, wherein the hand gesture includes extending all fingers of one hand.

93. The apparatus according to claim 74, wherein only a single pre-defined hand gesture can be detected.

94. The apparatus according to claim 74, wherein multiple pre-defined hand gestures can be detected.

95. The apparatus according to claim 94, wherein a respective, dedicated control is associated with each detected hand gesture.

96. The apparatus according to claim 74, further comprising a non-transitory computer readable medium storing an image processing algorithm for detecting a human face image in the captured image.

97. The apparatus according to claim 96, wherein the control signal is provided only in response to the detection of the hand gesture image and the human face image in the captured image.