SMART GLASSES, AND SYSTEM AND METHOD FOR PROCESSING HAND GESTURE COMMAND THEREFOR

• US 20160364008A1
• Filed: 06/10/2016
• Published: 12/15/2016
• Est. Priority Date: 06/12/2015
• Status: Abandoned Application

• Current Assignee: INSIGNAL Co.,Ltd., University-Industry Cooperation Foundation of Korea Aerospace University
• Original Assignee: INSIGNAL Co.,Ltd., University-Industry Cooperation Foundation of Korea Aerospace University
• Inventors: CHUN, Sung Moon, KO, Hyun Chul, KIM, Jea Gon
• Application Number:
  US15/179,028
• Publication Number:
  US 20160364008A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

Smart glasses, and a system and method for processing a hand gesture command using the smart glasses. According to an exemplary embodiment, the system includes smart glasses to capture a series of images including a hand gesture of a user and represent and transmit a hand image, included in each of the series of images, as hand representation data that is represented in a predetermined format of metadata; and a gesture recognition apparatus to recognize the hand gesture of a user by using the hand representation data of the series of images received from the smart glasses, and generate and transmit a gesture command corresponding to the recognized hand gesture.

Claims

1. Smart glasses for a gesture recognition apparatus that recognizes a hand gesture of a user and generates a gesture command corresponding to the recognized hand gesture, the smart glasses comprising:

a camera unit configured to capture a series of images including the hand gesture of a user;

a detection and representation unit configured to represent a hand image, included in each of the series of images, as hand representation data that is represented in a predetermined format of metadata; and

a communication unit configured to transmit the hand representation data, generated by the detection and representation unit, to the gesture recognition apparatus.

2. The smart glasses of claim 1, wherein the camera unit comprises a stereoscopic camera, and the series of images are a series of left and right images that are captured by using the stereoscopic camera.

3. The smart glasses of claim 1, wherein the camera unit comprises a depth camera, and the series of images are a series of depth-map images that are captured by using the depth camera.

4. The smart glasses of claim 1, wherein the detection and representation unit is configured to distinguish between a hand area and a background area by using a depth map of each of the series of images, and represent the hand area as hand representation data.

5. The smart glasses of claim 4, wherein the hand representation data represents a boundary line of the hand area with a Bé

zier curve.

6. The smart glasses of claim 4, wherein the detection and representation unit is configured to determine pixels, located within a predetermined distance, as the hand area by using the depth map.

7. The smart glasses of claim 4, wherein the detection and representation unit is configured to convert the depth map of each of the series of images into a depth-map image that is represented in a predetermined bit gray level, distinguish between the hand area and the background area from the depth-map image, represent the background area all in a gray level of ‘

0’

, perform filtering on the hand area, and represent the hand area as the hand representation data.

8. The smart glasses of claim 7, wherein the detection and representation unit is configured to generate a histogram of a pixel frequency, and distinguish between the hand area and the background area by defining, as a boundary value, a gray level of which pixel frequency is relatively small, but the pixel frequencies before and after the gray level are bigger.

9. A system for processing a hand gesture command, the system comprising:

smart glasses configured to capture a series of images including a hand gesture of a user, and represent and transmit a hand image, included in each of the series of images, as hand representation data that is represented in a predetermined format of metadata; and

a gesture recognition apparatus configured to recognize the hand gesture of a user by using the hand representation data of the series of images received from the smart glasses, and generate and transmit a gesture command corresponding to the recognized hand gesture.

10. The system of claim 9, wherein the smart glasses are configured to distinguish between a hand area and a background area by using a depth map of each of the series of images, and represent the hand area as hand representation data.

11. The system of claim 10, wherein the hand representation data represents a boundary line of the hand area with a Bé

zier curve.

12. The system of claim 10, wherein the smart glasses are configured to determine pixels, located within a predetermined distance, as the hand area by using the depth map.

13. The system of claim 10, wherein the smart glasses are configured to convert the depth map of each of the series of images into a depth-map image that is represented in a predetermined bit gray level, distinguish between the hand area and the background area from the depth-map image, represent the background area all in a gray level of ‘

0’

, perform filtering on the hand area, and represent the hand area as the hand representation data.

14. The system of claim 13, wherein the smart glasses are configured to generate a histogram of a pixel frequency, and distinguish between the hand area and the background area by defining, as a boundary value, a gray level of which a pixel frequency is relatively small, but the pixel frequencies before and after the gray level are bigger.

15. The system of claim 9, wherein the gesture recognition apparatus is configured to store a gesture and command comparison table, which represents a correspondence relation between a plurality of hand gestures and gesture commands that correspond to each of the plurality of hand gestures, and based on the gesture and command comparison table, determine a gesture command corresponding to the recognized hand gesture.

16. The system of claim 15, wherein the gesture and command comparison table is set by the user.

17. The system of claim 9, wherein the gesture recognition apparatus is configured to transmit the generated gesture command to the smart glasses or another electronic device to be controlled by the user.

18. A method of processing a hand gesture, the method comprising:

capturing a series of images including a hand gesture of a user;

representing a hand image, included in each of the series of images, as hand representation data that is represented in a predetermined format of metadata;

transmitting the hand representation data to a gesture recognition apparatus;

recognizing, by the gesture recognition apparatus, the hand gesture of a user by using the hand representation data of the series of images received from the smart glasses, and generate and transmit a gesture command corresponding to the recognized hand gesture; and

generating and transmitting a gesture command corresponding to the recognized hand gesture.

19. The method of claim 18, wherein the representing of the hand image as the hand representation data comprises distinguishing between the hand area and the background area by using a depth map of each of the series of images, and then representing the hand area as the hand representation data.