Shape recognition device, shape recognition program, and shape recognition method

• US 10,295,826 B2
• Filed: 02/19/2013
• Issued: 05/21/2019
• Est. Priority Date: 02/19/2013
• Status: Active Grant

• Current Assignee: Chikara Minami
• Original Assignee: Miramar Services Inc.
• Inventors: Lundberg, Johannes
• Primary Examiner(s): Potts, Ryan P
• Application Number:
  US14/768,479
• Publication Number:
  US 20150370321A1
• Time in Patent Office: 2,282 Days
• Field of Search: None
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: G06V 40/11 : Hand-related biometrics; Hand pose recognition
• CPC Class Codes: G06V 40/113 : Recognition of static hand signs
• CPC Class Codes: G06V 40/28 : Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 13/332 : Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/395 : with depth sampling, i.e. the volume being constructed from a stack or sequence of 2D image planes

Abstract

A shape recognition device, a shape recognition program, and a shape recognition method that can obtain more precise information for recognizing an external shape of an object are provided. A shape recognition device of the present invention includes an external-shape detecting unit that detects an external shape of an object, a first extracting unit that extracts a first feature point on the external shape based on the external shape detected by the external-shape detecting unit, and a second extracting unit that extracts a second feature point that exists on the external shape or in the external shape and differs from the first feature point. As a result, more precise information for recognizing the external shape of the object can be obtained.

Claims

1. A shape recognition device comprising:

a depth level sensor that measures a distance to an object and detects an external shape of a hand of a human body according to measured distance; and

a control unit that comprises a memory and a processor and performs image recognition functions that recognize fingers of the hand, the image recognition functions comprising;

a polygon creation function that creates a polygon having a vertex at each of a plurality of external-shape points based on the external shape;

a tip point extraction function that extracts a tip point of each finger from common vertexes between the polygon and a convex hull of the polygon;

a vector calculation function that calculates a vector of a reference line segment that connects a pair of right and left vertexes of the polygon adjacent to the tip point and a vector of each side that constitutes the polygon;

a base point extraction function that extracts a base point of each finger based on a position of a side which is most parallel to the reference line segment; and

an extending-direction detecting function that extracts a skeleton of each finger by connecting the tip point and the base point of each finger, and calculates a direction of the skeleton.

2. The shape recognition device according to claim 1, wherein

the depth level sensor further detects an external shape of an arm of the human body, and

the image recognition functions further comprising:

a frame detecting function that detects a quadrangular frame circumscribed around the external shape of the arm; and

an extending-direction detecting function that calculates a direction of a longer side constituting the quadrangular frame.

3. The shape recognition device according to claim 1,

the image recognition functions further comprising:

a palm recognition function that recognizes a position of a palm based on a maximum inscribed circle inscribed in the external shape of the hand.

4. The shape recognition device according to claim 3, the image recognition functions further comprising:

a thumb recognition function that recognizes a thumb of the hand based on the position of the palm and each finger recognized.

5. The shape recognition device according to claim 4, the control unit further performing:

an anatomy recognition function that recognizes an anatomic feature based on the fingers, the position of the palm and the thumb recognized; and

a gesture recognition function that recognizes a gesture based on the anatomic feature.

6. The shape recognition device according to claim 1, wherein the depth level sensor is an infrared camera.

7. The shape recognition device according to claim 1, further comprising a display device that can display a stereoscopic image.

8. A head-mounted display device comprising the shape recognition device according to claim 1.

9. A shape recognition system comprising:

a depth level sensor measures a distance to an object and detects an external shape of a hand having fingers of a human body according to measured distance;

a memory to store computer executable instructions; and

a processor to execute the computer executable instructions to perform processes, the processes comprising;

a polygon creation process of creating a polygon having a vertex at each of a plurality of external-shape points based on the external shape;

a tip point extraction process of extracting a tip point of each finger of the hand having fingers from common vertexes between the polygon and a convex hull of the polygon;

a vector calculation process of calculating a vector of a reference line segment that connects a pair of right and left vertexes of the polygon adjacent to the tip point and a vector of each side that constitutes the polygon;

a base point extraction process of extracting a base point of each finger based on a position of a side which is most parallel to the reference line segment; and

an extending-direction detecting process of extracting a skeleton of each finger by connecting the tip point and the base point of each finger, and calculating a direction of the skeleton.

10. The shape recognition system according to claim 9, further comprising a depth level sensor that detects an external shape of an arm of the human body,

the processes further comprising:

a frame detecting process of detecting a quadrangular frame circumscribed around the external shape of the arm; and

an extending-direction detecting process of calculating a direction of a longer side constituting the quadrangular frame.

11. The shape recognition system according to claim 9, the processes further comprising:

a palm recognition process of recognizing a position of a palm based on a maximum inscribed circle inscribed in the external shape of the hand.

12. The shape recognition system according to claim 11, the processes further comprising:

a thumb recognition process of recognizing a thumb of the hand based on the position of the palm and each finger.

13. The shape recognition system according to claim 12, the processes further comprising:

an anatomy recognition process of recognizing an anatomic feature based on the fingers, the position of the palm and the recognized thumb; and

a gesture recognition process of recognizing a gesture based on the anatomic feature.

14. The shape recognition system according to claim 9, further comprising a display process capable of displaying a stereoscopic image.

15. A shape recognition method comprising:

a shape detecting step of detecting an external shape of a hand having fingers of a human body using data of distance to an object measured by a depth level sensor;

a polygon creation step of creating a polygon having a vertex at each of a plurality of external-shape points based on the external shape;

a tip point extraction step of extracting a tip point of each finger of the hand having fingers from common vertexes between the polygon and a convex hull of the polygon;

a vector calculation step of calculating a vector of a reference line segment that connects a pair of right and left vertexes of the polygon adjacent to the tip point and a vector of each side that constitutes the polygon;

a base point extraction step of extracting a base point of each finger based on a position of a side which is most parallel to the reference line segment; and

an extending-direction detecting step of extracting a skeleton of each finger by connecting the tip point and the base point of each finger, and calculating a direction of the skeleton.

16. The shape recognition method according to claim 15, further comprising

a shape detecting step of detecting an external shape of an arm of the human body using the depth level sensor;

a frame detecting step of detecting a quadrangular frame circumscribed around the external shape of the arm; and

an extending-direction detecting step of calculating a direction of a longer side constituting the quadrangular frame.

17. The shape recognition method according to claim 15, further comprising

a palm recognition step of recognizing a position of a palm based on a maximum inscribed circle inscribed in the external shape of the hand.

18. The shape recognition method according to claim 17, further comprising:

a thumb recognition step of recognizing a thumb of the hand based on the position of the palm and each finger.

19. The shape recognition method according to claim 18, further comprising:

an anatomy recognition step of recognizing an anatomic feature based on the fingers, the position of the palm and the recognized thumb; and

a gesture recognition step of recognizing a gesture based on the anatomic feature.

20. The shape recognition method according to claim 15, further comprising a display step of displaying a stereoscopic image.