Method by which eyeglass-type display device recognizes and inputs movement

• US 9,904,372 B2
• Filed: 08/28/2014
• Issued: 02/27/2018
• Est. Priority Date: 10/04/2013
• Status: Active Grant

• Current Assignee: Macron Co., Ltd.
• Original Assignee: Macron Co., Ltd.
• Inventors: Lee, Kil-Jae
• Primary Examiner(s): Amadiz, Rodney
• Application Number:
  US15/027,174
• Publication Number:
  US 20160252966A1
• Time in Patent Office: 1,279 Days
• Field of Search: 345 7- 8
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Provided is a method of performing an input operation through motion recognition in a glasses-type display device, which is capable of controlling the glasses-type display device through the most natural and intuitive hand motion. The method of performing the input operation through motion recognition in the glasses-type display device includes: an image acquiring operation of acquiring sequential images by using a camera mounted on the glasses-type display device; a hand recognizing operation of recognizing a hand of a user from the acquired sequential images; a hand tracking operation of tracking the recognized hand; a hand motion recognizing operation of recognizing a motion of the hand based on a position of the hand acquired in the hand tracking operation; and a control signal outputting operation of outputting a control signal corresponding to the motion of the hand. When the hand is recognized in the hand recognizing operation, the position of the hand is displayed on a screen of the glasses-type display device.

Claims

1. A method of performing an input operation through motion recognition in a glasses-type display device, the method comprising:

an image acquiring operation of acquiring sequential images by using a camera mounted on the glasses-type display device;

a hand recognizing operation of recognizing a hand of a user from the acquired sequential images;

a hand tracking operation of tracking the recognized hand;

a hand motion recognizing operation of recognizing a motion of the hand based on a position of the hand acquired in the hand tracking operation; and

a control signal outputting operation of outputting a control signal corresponding to the motion of the hand;

the method further comprising;

extracting an offset value for movement of a background except for the hand through matching between images from the plurality of sequential images acquired in the image acquiring operation; and

correcting the position of the hand acquired in the hand tracking operation by using the extracted offset value, so as to correct a change in the position of the hand due to movement of the camera,

wherein, when the hand is recognized in the hand recognizing operation, the position of the hand is displayed on a screen of the glasses-type display device.

2. The method of claim 1, wherein the hand recognizing operation comprises recognizing the hand having a predetermined hand shape, the predetermined hand shape including a shape in which an index finger is spread out.

3. The method of claim 2, wherein the hand recognizing operation comprises:

extracting a contour shape of the hand from the acquired sequential images;

recognizing whether the extracted contour shape of the hand is a hand shape in which the index finger is spread out; and

recognizing whether a direction of hand movement is changed a plurality of number of times from side to side or up and down.

4. The method of claim 2, wherein the hand motion recognizing operation comprises recognizing a motion of bending and spreading out the index finger by analyzing a change in a position of a color code that is attached to an index finger and has a specific color.

5. The method of claim 4, wherein the color code has a ring shape so that the color code is worn on the index finger.

6. The method of claim 4, wherein the color code has a combination of two or more colors.

7. The method of claim 2, wherein the hand motion recognizing operation comprises:

recognizing whether a color code having a specific color is attached to the index finger;

when the color code is not attached to the index finger, recognizing a motion in which the index finger has stopped on a menu for a predetermined time, as a click motion; and

when the color code is attached to the index finger, recognizing a motion of bending and spreading out the index finger as the click motion.

8. The method of claim 1, wherein the hand tracking operation tracks the position of the hand in a plurality of sequential images that are input after the hand is recognized in the hand recognizing operation, and

the hand tracking operation comprises:

recognizing whether the hand is within a predetermined distance by using information about the position of the hand in a previous image; and

tracking the hand by using information about a hand color and a hand shape acquired in the hand recognizing operation.

9. The method of claim 1, wherein the image acquired in the image acquiring operation is displayed on a partial region of a screen of the glasses-type display device or is overlaid on the entire screen of the glasses-type display device.

10. The method of claim 1, wherein, in the hand motion recognizing operation, a trajectory of the hand that draws a circle clockwise is interpreted as a zoom-in motion, and a trajectory of the hand that draws a circle counterclockwise is interpreted as a zoom-out motion.