Gesture registration device, gesture registration program, and gesture registration method

• US 9,857,589 B2
• Filed: 04/12/2013
• Issued: 01/02/2018
• Est. Priority Date: 02/19/2013
• Status: Active Grant

• Current Assignee: Chikara Minami
• Original Assignee: Miramar Services Inc.
• Inventors: Lundberg, Johannes
• Primary Examiner(s): MARINELLI, PATRICK
• Application Number:
  US14/768,241
• Publication Number:
  US 20150378158A1
• Time in Patent Office: 1,726 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: G06V 40/11 : Hand-related biometrics; Hand pose recognition
• CPC Class Codes: G06V 40/113 : Recognition of static hand signs
• CPC Class Codes: G06V 40/28 : Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 13/332 : Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/395 : with depth sampling, i.e. the volume being constructed from a stack or sequence of 2D image planes

Abstract

A gesture registration device, a gesture registration program and a gesture registration method that can rapidly and simply obtain and stock gesture data are provided. The gesture registration device of the present invention includes an external-shape detecting unit, an analyzing unit, and a recording unit. The external-shape detecting unit detects an external shape of an object having a predetermined form. The analyzing unit performs at least analysis of an external-shape feature part and analysis of the value for at least either the position or the direction of the external shape feature part, from the detected the external shape of the object. The recording unit records a result by the analysis unit as gesture data.

Claims

1. A gesture registration device comprising:

an external-shape detecting unit that detects an external shape of an object having a predetermined form comprising at least a finger, a palm and a hand;

an analyzing unit that performs, from the detected external shape of the object comprising at least the finger, the palm, and the hand in a three-dimensional space having imperfect spherical shapes about a center of rotation of which is a right shoulder joint and a left shoulder joint of a user, respectively, at least analysis of an external-shape feature part by a tip point of the finger and a base point of the finger by using finger recognition, by palm recognition by using a maximum inscribed circle and by thumb recognition by using the palm recognition, and analysis of a value for at least either a position or a direction of the external-shape feature part;

a recording unit that records a result by the analysis unit as gesture data; and

an associating unit that causes the recording unit to record the gesture data in association with a predetermined operation.

2. The gesture registration device according to claim 1, further comprising a deleting unit that deletes the gesture data from the recording unit.

3. The gesture registration device according to claim 1, further comprising:

a releasing unit that releases association of the gesture data with the predetermined operation.

4. The gesture registration device according to claim 1, wherein the external-shape detecting unit detects a serial external shape of the object varying with time, and

the analyzing unit further performs analysis of a time variation of the external-shape feature part.

5. The gesture registration device according to claim 1, wherein the external-shape detecting unit detects the external shape of the object a plurality of times.

6. The gesture registration device according to claim 1, wherein the external-shape detecting unit is a moving-image photographing device, and

the external-shape detecting unit detects a serial external shape by continuing to image the object having the predetermined form for a predetermined time.

7. The gesture registration device according to claim 1, wherein the external-shape detecting unit is a still-image photographing device, and

the external-shape detecting unit detects a plurality of the external shapes by imaging the object having the predetermined form a plurality of times.

8. The gesture registration device according to claim 1, wherein the external-shape feature part is at least any one of a number, a position, an angle of fingers having a predetermined form, and an angle of an arm.

9. A head-mounted display device comprising the gesture registration device according to claim 1.

10. A non-transitory computer readable medium storing thereon a gesture registration program comprising instructions that when executed by a processor initiates processes comprising:

an external-shape detection process of detecting an external shape of an object having a predetermined form comprising at least a finger, a palm, and a hand;

an analysis process of analyzing, from the detected external shape of the object comprising at least the finger, the palm and the hand in a three-dimensional space having imperfect spherical shapes about a center of rotation of which is a right shoulder joint and a left shoulder joint of a user, respectively, at least analysis of an external-shape feature part by a tip point of the finger and a base point of the finger by using finger recognition, by palm recognition by using a maximum inscribed circle and by thumb recognition by using the palm recognition, and analysis of a value for at least either a position or a direction of the external-shape feature part;

a recording process of recording a result in the analysis process as gesture data; and

an association process of causing a recording unit to record the gesture data in association with a predetermined operation.

11. The non-transitory computer readable medium according to claim 10, further comprising a deletion process of deleting the gesture data.

12. The non-transitory computer readable medium according to claim 10, further comprising a release process of releasing association of the gesture data with the predetermined operation.

13. The non-transitory computer readable medium according to claim 10, wherein the external-shape detection process is a moving-image photographing process, and

the external-shape detection process includes a process of detecting a serial external shape by continuing to image the object having the predetermined form for a predetermined time to detect a series of contours.

14. The non-transitory computer readable medium according to claim 10, wherein the external-shape detection process is a still-image photographing process, and

the external-shape detection process includes a process of detecting a plurality of the external shapes by imaging the object having the predetermined form a plurality of times.

15. A gesture registration method comprising:

an external-shape detection step in which an external shape of an object having a predetermined form comprising at least a finger, a palm, and a hand is detected;

an analysis step in which, from the detected external shape of the object comprising at least the finger, the palm and the hand in a three-dimensional space having imperfect spherical shapes about a center of rotation of which is a right shoulder joint and a left shoulder joint of a user, respectively, at least analysis of an external-shape feature part by a tip point of the finger and a base point of the finger by using finger recognition, by palm recognition by using a maximum inscribed circle and by thumb recognition by using the palm recognition, and analysis of a value of at least either a position or a direction of the external-shape feature part, are performed;

a recording step in which a result in the analysis step is recorded as gesture data; and

an association step of causing a recording unit to record the gesture data in association with a predetermined operation.

16. The gesture registration method according to claim 15, further comprising a deletion step in which the gesture data is deleted.

17. The gesture registration method according to claim 15, further comprising a release step in which association of gesture data indicating the predetermined form with the predetermined operation is released.

18. The gesture registration method according to claim 15, wherein the external-shape detection step is a moving-image photographing step, and includes a step in which a serial external shape is detected by continuing to image the object having the predetermined form for a predetermined time.

19. The gesture registration method according to claim 15, wherein the external-shape detection step is a still-image photographing step, and includes a step in which a plurality of the external shapes are detected by imaging the object having the predetermined form a plurality of times.