Image identification method and image identification device based on fingertip tracking

• US 9,396,387 B2
• Filed: 12/16/2014
• Issued: 07/19/2016
• Est. Priority Date: 10/29/2014
• Status: Active Grant

• Current Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Original Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Inventors: Yu, Yinan, Gu, Jiawei, Wang, Rui, Yu, Kai
• Primary Examiner(s): HEIDEMANN, JASON E
• Application Number:
  US14/571,950
• Publication Number:
  US 20160125236A1
• Time in Patent Office: 581 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An image identification method and an image identification device are provided. The method comprises acquiring a hand feature region within a sight from a first view by skin color detection; acquiring a feature and a position of a tip of a finger from the hand feature region by performing a pattern recognition for a morphological feature of a stretched hand; recording an interframe displacement of a feature point of the tip of the finger when the tip of the finger delimits a periphery of a target object to obtain a delimiting trajectory from the interframe displacement, closing the delimiting trajectory to form a full-perimeter geometry; projecting the full-perimeter geometry on a plane where a direction of the sight is perpendicular to a plane where the target object is located to obtain a projection region, performing an image identification using the projection region as an identification region of the target object.

Claims

1. A computer-implemented image identification method, comprising:

acquiring, at one or more computing devices, a hand feature region within a sight from a first view by a skin color detection, and capturing and tracking the hand feature region in real time;

acquiring, at the one or more computing devices, a feature and a position of a tip of a finger from the hand feature region by performing a pattern recognition for a morphological feature of a stretched hand, and capturing and tracking the feature and the position of the tip of the finger in real time in the first view;

recording, using the one or more computing devices, an interframe displacement of a feature point of the tip of the finger when the tip of the finger delimits a periphery of a target object located in the first view to obtain a delimiting trajectory from the interframe displacement, and closing the delimiting trajectory to form a full-perimeter geometry;

projecting, using the one or more computing devices, the full-perimeter geometry on a plane where a direction of the sight is perpendicular to a plane where the target object is located so as to obtain a projection region, so as to separate the projection region from the first view as a segmented image; and

performing, using the one or more computing devices, an image identification using the projection region as an identification region of the target object.

2. The image identification method according to claim 1, wherein closing the delimiting trajectory to form the full-perimeter geometry comprises:

closing the delimiting trajectory to form the full-perimeter geometry by a closed curve optimization algorithm.

3. The image identification method according to claim 1, further comprising:

receiving, at the one or more computing devices, an image identification request input via speech, a touch screen or a dynamic sensor so as to trigger an operation of acquiring the hand feature region within the sight from the first view by the skin color detection.

4. An image identification device, comprising:

one or more processors configured to execute one or more software modules, the one or more software modules including;

a first acquiring module configured to acquire a hand feature region within a sight from a first view by a skin color detection;

a first tracking module configured to capture and track the hand feature region in real time in the first view;

a second acquiring module configured to acquire a feature and a position of a tip of a finger from the hand feature region by performing a pattern recognition for a morphological feature of a stretched hand;

a second tracking module configured to capture and track the feature and the position of the tip of the finger in real time;

a recording module configured to record an interframe displacement of a feature point of the tip of the finger when the tip of the finger delimits a periphery of a target object located in the first view to obtain a delimiting trajectory from the interframe displacement, and to close the delimiting trajectory to form a full-perimeter geometry; and

a projecting module configured to project the full-perimeter geometry on a plane where a direction of the sight is perpendicular to a plane where the target object is located so as to obtain a projection region, so as to separate the projection region from the first view as a segmented image, and to perform an image identification using the projection region as an identification region of the target object.

5. The image identification device according to claim 4, wherein the recording module is configured to close the delimiting trajectory to form the full-perimeter geometry by a closed curve optimization algorithm.

6. The image identification device according to claim 4, wherein the one or more software modules further comprises:

a receiving module configured to receive an image identification request input via speech, a touch screen or a dynamic sensor so as to trigger an operation of acquiring the hand feature region within the sight from the first view by the skin color detection.

7. A non-transitory computer readable storage medium, comprising a computer program configured to execute the following steps on one or more computing devices:

acquiring a hand feature region within a sight from a first view by a skin color detection, and capturing and tracking the hand feature region in real time in the first view;

acquiring a feature and a position of a tip of a finger from the hand feature region by performing a pattern recognition for a morphological feature of a stretched hand, and capturing and tracking the feature and the position of the tip of the finger in real time;

recording an interframe displacement of a feature point of the tip of the finger when the tip of the finger delimits a periphery of a target object located in the first view to obtain a delimiting trajectory from the interframe displacement, and closing the delimiting trajectory to form a full-perimeter geometry; and

projecting the full-perimeter geometry on a plane where a direction of the sight is perpendicular to a plane where the target object is located so as to obtain a projection region, so as to separate the projection region from the first view as a segmented image, and performing an image identification using the projection region as an identification region of the target object.