Gesture input systems and methods using 2D sensors

• US 9,335,827 B2
• Filed: 05/22/2013
• Issued: 05/10/2016
• Est. Priority Date: 07/17/2012
• Status: Active Grant

• Current Assignee: Wistron Corporation
• Original Assignee: Wistron Corporation
• Inventors: Huang, Chih-Hao, Tsao, Hsun-Chih, Liao, Chih-Pin, Liou, Pin-Hong
• Primary Examiner(s): Bost, Dwayne
• Assistant Examiner(s): Ritchie, Darlene M
• Application Number:
  US13/900,439
• Publication Number:
  US 20140022172A1
• Time in Patent Office: 1,084 Days
• Field of Search: 345/158, 382/168, 348/14.03, 715/848
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

A gesture input system with a two-dimension (2D) image sensor and a processing module is provided. The 2D image sensor obtains a plurality of images of a user. The processing module determines positions of an object and a face of the user in a first image of the plurality of images, and determines an operation area for the user according to the positions of the object and the face. Also, the processing module generates a control command according to the subsequent images to the first image of the user within the operation area.

Claims

1. A gesture input system, comprising:

a two-dimension (2D) image sensor, obtaining a plurality of images of a user; and

a processing module, determining positions of an object and a face of the user in a first image of the plurality of images, determining an operation area for the user according to the positions of the object and the face in a 2D plane corresponding to the plurality of images, determining a trigger area within the operation area and in the 2D plane, determining a first contact relation between positions of the object in the subsequent images and the trigger area in the 2D plane, and generating a first control command according to the first contact relation,

wherein the first contact relation is one of a plurality of predetermined types of first contact relations comprising;

the object moves from the outside of the trigger area to the inside of the trigger area through a side of the trigger area in the 2D plane, and then moves from the inside of the trigger area to the outside of the trigger area through the side of the trigger area in the 2D plane; and

the object moves from the inside of the trigger area to the outside of the trigger area through a side of the trigger area in the 2D plane, and then moves to the inside of the trigger area through the side of the trigger area in the 2D plane.

2. The gesture input system of claim 1, wherein the processing module further performs mapping of the operation area to a user interface for determining the trigger area.

3. The gesture input system of claim 1, wherein the processing module further determining a second contact relation between positions of the object in the subsequent images and the trigger area in the 2D plane, and generating a second control command according to the second contact relation, wherein the second contact relation is one of a plurality of predetermined types of second contact relations comprising:

the object moves from the outside of the trigger area to the inside of the trigger area in the 2D plane; and

the object moves from the inside of the trigger area to the outside of the trigger area in the 2D plane.

4. The gesture input system of claim 1, wherein the operation area is determined by specifying the positions of the object and the face as boundaries of the operation area, or by a distance between the positions of the object and the face.

5. The gesture input system of claim 1, wherein the processing module further repeats the determination of the positions of the object and the face periodically to update the operation area.

6. The gesture input system of claim 1, wherein the processing module further determines an average moving range in which the object has moved within the operation area according to positions of the object in the subsequent images, and periodically updates the operation area according to the average moving range.

7. The gesture input system of claim 1, wherein the processing module further detects a movement of the object via the 2D image sensor for determining the position of the object, and searching for where the face of the user is around the object for determining the position of the face.

8. The gesture input system of claim 1, wherein each of the plurality of predetermined types of first contact relations corresponds to a single first control command.

9. A gesture input method, comprising:

obtaining a plurality of images of a user via a two-dimension (2D) image sensor;

determining positions of an object and a face of the user in a 2D plane in a first image of the plurality of images;

determining an operation area for the user in the 2D plane according to the positions of the object and the face;

determining a trigger area within the operation area and in the 2D plane;

determining a first contact relation between positions of the object in the subsequent images and the trigger area in the 2D plane; and

generating a first control command according to the contact relation,

wherein the first contact relation is one of a plurality of predetermined types of first contact relations comprising;

the object moves from the outside of the trigger area to the inside of the trigger area through a side of the trigger area in the 2D plane, and then moves from the inside of the trigger area to the outside of the trigger area through the side of the trigger area in the 2D plane; and

the object moves from the inside of the trigger area to the outside of the trigger area through a side of the trigger area in the 2D plane, and then moves to the inside of the trigger area through the side of the trigger area in the 2D plane.

10. The gesture input method of claim 9, wherein the step of determining the trigger area further comprises mapping the operation area to a user interface.

11. The gesture input method of claim 9, further comprising the steps of:

determining a second contact relation between positions of the object in the subsequent images and the trigger area in the 2D plane; and

generating a second control command according to the contact relation, wherein the second contact relation is one of a plurality of predetermined types of second contact relations comprising;

the object moves from the outside of the trigger area to the inside of the trigger area in the 2D plane;

the object moves from the inside of the trigger area to the outside of the trigger area in the 2D plane.

12. The gesture input method of claim 9, wherein the operation area is determined by specifying the positions of the object and the face as boundaries of the operation area, or by a distance between the positions of the object and the face.

13. The gesture input method of claim 9, further comprising repeating the determination of the positions of the object and the face periodically to update the operation area.

14. The gesture input method of claim 9, further comprising:

determining an average moving range in which the object has moved within the operation area according to positions of the object in the subsequent images; and

periodically updating the operation area according to the average moving range.

15. The gesture input method of claim 9, wherein the step of determining the positions of the object and the face further comprises:

detecting a movement of the object via the 2D image sensor for determining the position of the object; and

searching for where the face of the user is around the object for determining the position of the face.

16. The gesture input method of claim 9, wherein each of the plurality of predetermined types of first contact relations corresponds to a single first control command.