Method of controlling virtual object by user's figure or finger motion for electronic device

• US 8,203,530 B2
• Filed: 08/01/2007
• Issued: 06/19/2012
• Est. Priority Date: 04/24/2007
• Status: Active Grant

• Current Assignee: Taiwan Semiconductor Manufacturing Company Limited, Vircontech LLC
• Original Assignee: Kuo-Ching Chiang
• Inventors: Chiang, Kuo-Ching
• Primary Examiner(s): Afshar, Kamran
• Assistant Examiner(s): Behnamian, Shahriar
• Application Number:
  US11/882,436
• Publication Number:
  US 20090295738A1
• Time in Patent Office: 1,784 Days
• Field of Search: 345157-162, 455/566, 455/158.4, 455/154.2
• US Class Current: 345/157
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04B 1/401 for selecting or indicating operating mode
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04M 2250/22 including a touch pad, a touch sensor or a touch detector
• CPC Class Codes: H04M 2250/52 including functional features of a camera
• CPC Class Codes: H04W 48/18 Selecting a network or a communication service

Abstract

The present invention provides a method of controlling a virtual object or instruction for a computing device comprising: detecting a user activity by a detecting device; generating a control signal in responsive to the user activity detection; controlling an object displayed on a display in responsive to the control signal to execute the instruction. The user activity is detected by CMOS or CCD. The user activity includes facial motion, eye motion, or finger motion.

Claims

1. A method of controlling a virtual object by figure motion of a user for an electronic computing device having a display comprising:

providing at least one image sensor for said display of said electronic computing device to capture image of said user in front of said display, wherein said at least one image sensor includes CMOS or CCD sensor, wherein said at least one image sensor provided to detect motion of said user'"'"'s figure motion to output a first control signal to a virtual object without touching said display, said user being located in front of said at least one image sensor and keeping said display with a distance and said user'"'"'s figure being not over a keyboard;

capturing images of said figure of said user without touching said display by said at least one image sensor;

analyzing image change of said figure motion;

generating a control signal in responsive to said image change of said figure motion detection;

providing a vocal control through a microphone and a vocal control software of said electronic computing device to output a second control signal through voice recognition; and

controlling said virtual object displayed on said display in responsive to said first, second or combination control signal without touching said display of said computing device.

9. The method of claim 1, wherein said user is illuminated by a light source.

10. The method of claim 9, wherein said light source includes infrared ray (IR).

2. A method of controlling a displayed image for an electronic device having a display comprising:

providing at least one image sensor for said display of said electronic device to capture image of a user in front of said display and said at least one image sensor, wherein said at least one image sensor includes CMOS or CCD sensor;

detecting and response to user'"'"'s facial expression with said at least one image sensor when said user is in front of said electronic device, analyzing facial expression of said user to output a first control signal without touching said display of said computing device;

detecting motion of said user'"'"'s figure located in front of at least one image sensor to output a second control signal to control a virtual object on said display without touching said display with a distance by said at least one image sensor; and

said user'"'"'s figure being not over a keyboard;

analyzing image change of said figure motion;

generating a second control signal in responsive to said image change of figure motion detection;

controlling said displayed image on said display in responsive to said first, second or combination control signal without touching said display of said computing device.

3. The method of claim 2, wherein said control signal includes cursor movement.

4. The method of claim 2, wherein said control signal includes character input.

5. The method of claim 2, wherein said control signal includes software application instruction.

11. The method of claim 2, wherein said user is illuminated by a light source.

12. The method of claim 11, wherein said light source includes infrared ray (IR).

6. A method for browsing web page, text, email or image on a display of an electronic computing device, comprising:

providing at least one image sensor in front of said display of said electronic computing device to detect a user in front of said display and said at least one image sensor, wherein said at least one image sensor includes CMOS or CCD sensor;

detecting and response to user'"'"'s facial expression with said at least one image sensor when said user is in front of said electronic device to output a first control signal without touching said display of said computing device;

detecting motion of said user'"'"'s figure by said at least one image sensor to output a second control signal to a virtual object without touching said display, said user being located in front of said at least one image sensor and keeping said display with a distance and said user'"'"'s figure being not over a keyboard;

providing a vocal control through a microphone and a vocal control software of said electronic computing device to output a third control signal through voice recognition;

displaying a first part of said web page, text, email or image on said display;

analyzing image change of said user figure motion;

generating said second control signal control signal based on said image change of said figure motion, wherein a second part of said web page, text, email or image on said display is displayed based on said first, second, third or combination control signal, wherein said first, second, third or combination control signal indicates where said user would like to browser.

7. The method of claim 6, wherein said user is illuminated by a light source.

8. The method of claim 7, wherein said light source includes infrared ray (IR).

13. A method of controlling a displayed image for an electronic device having a display comprising:

providing at least one image sensor for said display of said electronic device to capture image of a user in front of said display and said at least one image sensor, wherein said at least one image sensor includes CMOS or CCD sensor;

detecting motion of said user'"'"'s finger motion by said at least one image sensor, analyzing image change of said finger motion, generating a first control signal in responsive to said image change of said finger motion detection without touching said display of said computing device;

detecting motion of said user'"'"'s figure located in front of at least one image sensor, analyzing image change of said user'"'"'s figure motion, generating a second control signal in responsive to said image change of said figure motion detection, generating a second control signal in responsive to said image change of said figure motion detection without touching said display of said computing device; and

said user'"'"'s figure and finger being not over a keyboard;

providing a vocal control through a microphone and a vocal control software of said electronic computing device to output a third control signal through voice recognition; and

controlling image on said display in responsive to said first, second or combination control signal.

14. A method of controlling a displayed image for an electronic device having a display comprising:

providing at least one image sensor for said display of said electronic device to capture image of a user in front of said display and said at least one image sensor, wherein said at least one image sensor includes CMOS or CCD sensor;

detecting said user'"'"'s facial expression at least one image sensor, analyzing said detected user'"'"'s facial expression, generating a first control signal in responsive to said facial expression detection without touching said display of said computing device;

detecting motion of said user'"'"'s finger motion by said at least one image sensor, analyzing image change of said finger motion, generating a second control signal in responsive to said image change of said finger motion detection without touching said display of said computing device;

detecting motion of said user'"'"'s figure located in front of at least one image sensor, analyzing image change of said user'"'"'s figure motion, generating a third control signal in responsive to said image change of said figure motion detection without touching said display of said computing device; and

said user'"'"'s figure and finger being not over a keyboard; and

controlling image on said display in responsive to said first, second, third or combination control signal.