METHOD AND SYSTEM FOR RECOGNIZING AN OBJECT

• US 20160048727A1
• Filed: 08/15/2014
• Published: 02/18/2016
• Est. Priority Date: 08/15/2014
• Status: Active Grant

• Current Assignee: Konica Minolta Laboratory U.S.A., Inc. (Konica Minolta Inc.)
• Original Assignee: Konica Minolta Laboratory U.S.A., Inc. (Konica Minolta Inc.)
• Inventors: ZHANG, Yongmian, Gu, Haisong, Tai, Hung-Shou
• Granted Patent: US 9,922,245 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths

Abstract

A method, a system, and a non-transitory computer readable medium for recognizing an object are disclosed, the method including: emitting an array of infrared rays from an infrared emitter towards a projection region, the projection region including a first object; generating a reference infrared image by recording an intensity of ray reflection from the projection region without the first object; generating a target infrared image by recording the intensity of ray reflection from the projection region with the first object; comparing the target infrared image to the reference infrared image to generate a predetermined intensity threshold; and extracting the first object from the target infrared image, if the intensity of ray reflection of the target infrared image of the first object exceeds the predetermined intensity threshold.

Claims

1. A method for recognizing an object, the method comprising:

emitting an array of infrared rays from an infrared emitter towards a projection region, the projection region including a first object;

generating a reference infrared image by recording an intensity of ray reflection from the projection region without the first object;

generating a target infrared image by recording the intensity of ray reflection from the projection region with the first object;

comparing the target infrared image to the reference infrared image to generate a predetermined intensity threshold; and

extracting the first object from the target infrared image, if the intensity of ray reflection of the target infrared image of the first object exceeds the predetermined intensity threshold.

2. The method of claim 1, wherein the first object is a hand of a user, and wherein if the intensity of ray reflection of the target infrared image of the hand exceeds the predetermined intensity threshold:

generating a binarized image of the hand from the infrared image; and

comparing the binarized image of the hand to a model hand to detect and track movement of the one or more fingers of the hand.

3. The method of claim 2, wherein the hand model comprises a palm, a plurality of finger tips, and corresponding roots between each of a plurality of fingers, and wherein generating the binarized image of the hand comprises:

generating a segmented hand image from the binarized image of the hand;

extracting a hand profile from the segmented hand image;

extracting hand convexities and convexity defects from the hand profile;

removing convexity points that have a depth of the corresponding roots, which are less than a predetermined depth threshold;

determining a finger with a deepest root and eliminating other fingers that point in an opposite direction of the finger;

determining a finger length from a deepest finger root along the opposite direction of the finger as an upper boundary of the palm and a lower boundary of the palm being a point of the deepest root; and

determining a gravity center between the upper boundary and the lower boundary to define a center of the palm, and wherein a radius of the palm is a distance between the gravity center and the point of the deepest root.

4. The method of claim 3, comprising:

eliminating any tips of fingers within a given threshold of the center of the palm;

classifying the non-eliminated tips as finger tips.

5. The method of claim 4, wherein a touch is differentiated from a hover by:

detecting a plurality of pixels around an edge of a finger tip;

superimposing a box around a region of the finger tip and a contact surface; and

determining a number of pixels in the box, wherein if the number of pixels are equal to or exceeds a predetermined pixel threshold, then the finger tip is touching the contract surface and if the number of pixels are less than the predetermined pixel threshold, then the finger tip is not touching the contact surface.

6. The method of claim 3, comprising:

tracking a finger tip using a global nearest neighbor, wherein the global nearest neighbor includes a gate around a predicted position of the palm center and a predicted position of finger tips associated with the palm by Kalman filtering.

7. The method of claim 6, comprising;

touch screen gesture recognition of the finger tip on a surface using a feature vector extracted along a trajectory of a moving finger tip on the touch surface, the feature vector calculated by the following equations;

9. The system of claim 1, wherein the first object is a hand of a user, and wherein if the intensity of ray reflection of the target infrared image of the hand exceeds the predetermined intensity threshold configuring the processor to:

generate a binarized image of the hand from the infrared image; and

compare the binarized image of the hand to a model hand to detect and track movement of the one or more fingers of the hand.

10. The system of claim 9, wherein the hand model comprises a palm, a plurality of finger tips, and corresponding roots between each of a plurality of fingers, and wherein generating the binarized image of the hand comprises:

generating a segmented hand image from the binarized image of the hand;

extracting a hand profile from the segmented hand image;

extracting hand convexities and convexity defects from the hand profile;

removing convexity points that have a depth of the corresponding roots, which are less than a predetermined depth threshold;

determining a finger with a deepest root and eliminating other fingers that point in an opposite direction of the finger;

determining a finger length from a deepest finger root along the opposite direction of the finger as an upper boundary of the palm and a lower boundary of the palm being a point of the deepest root; and

determining a gravity center between the upper boundary and the lower boundary to define a center of the palm, and wherein a radius of the palm is a distance between the gravity center and the point of the deepest root.

8. A system for recognizing an object, the system comprising:

an infrared emitter configured to emit an array of infrared rays from an infrared emitter towards a projection region, the projection region including a first object;

an infrared camera for recording an intensity of ray reflection from the projection region without the first object as a reference infrared image and the intensity of ray reflection of the projection region with the first object infrared image of the first object as a target infrared image; and

a processor for;

comparing the target infrared image to the reference infrared image to generate a predetermined intensity threshold; and

extracting the first object from the target infrared image, if the intensity of ray reflection of the target infrared image of the first object exceeds the predetermined intensity threshold.

11. The system of claim 8, comprising:

an infrared pass filter configured to increase a contrast of the intensity of ray reflection of the target and reference infrared images.

12. A non-transitory computer readable medium containing a computer program having computer readable code embodied to carry out a method for recognizing an object, the method comprising:

emitting an array of infrared rays from an infrared emitter towards a projection region, the projection region including a first object;

generating a reference infrared image by recording an intensity of ray reflection from the projection region without the first object;

generating a target infrared image by recording the intensity of ray reflection from the projection region with the first object;

comparing the target infrared image to the reference infrared image to generate a predetermined intensity threshold; and

extracting the first object from the target infrared image, if the intensity of ray reflection of the target infrared image of the first object exceeds the predetermined intensity threshold.

13. The computer readable medium of claim 12, wherein the first object is a hand of a user, and wherein if the intensity of ray reflection of the target infrared image of the hand exceeds the predetermined intensity threshold:

generating a binarized image of the hand from the infrared image; and

comparing the binarized image of the hand to a model hand to detect and track movement of the one or more fingers of the hand.

14. The computer readable medium of claim 13, wherein the hand model comprises a palm, a plurality of finger tips, and corresponding roots between each of a plurality of fingers, and wherein generating the binarized image of the hand comprises:

generating a segmented hand image from the binarized image of the hand;

extracting a hand profile from the segmented hand image;

extracting hand convexities and convexity defects from the hand profile;

removing convexity points that have a depth of the corresponding roots, which are less than a predetermined depth threshold;

determining a finger with a deepest root and eliminating other fingers that point in an opposite direction of the finger;

determining a finger length from a deepest finger root along the opposite direction of the finger as an upper boundary of the palm and a lower boundary of the palm being a point of the deepest root; and

determining a gravity center between the upper boundary and the lower boundary to define a center of the palm, and wherein a radius of the palm is a distance between the gravity center and the point of the deepest root.

15. The computer readable medium of claim 14, comprising:

eliminating any tips of fingers within a given threshold of the center of the palm;

classifying the non-eliminated tips as finger tips.

16. The computer readable medium of claim 15, wherein a touch is differentiated from a hover by:

detecting a plurality of pixels around an edge of a finger tip;

superimposing a box around a region of the finger tip and a contact surface; and

determining a number of pixels in the box, wherein if the number of pixels are equal to or exceeds a predetermined pixel threshold, then the finger tip is touching the contract surface and if the number of pixels are less than the predetermined pixel threshold, then the finger tip is not touching the contact surface.

17. The computer readable medium of claim 14, comprising:

tracking a finger tip using a global nearest neighbor, wherein the global nearest neighbor includes a gate around a predicted position of the palm center and a predicted position of finger tips associated with the palm by Kalman filtering.

18. The computer readable medium of claim 17, comprising;

touch screen gesture recognition of the finger tip on a surface using a feature vector extracted along a trajectory of a moving finger tip on the touch surface, the feature vector calculated by the following equations;