USER INTERFACE APPARATUS AND METHOD USING MOVEMENT RECOGNITION

• US 20120106792A1
• Filed: 10/31/2011
• Published: 05/03/2012
• Est. Priority Date: 10/29/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: KANG, Woo-Sung, Kwon, Mu-Sik, Kim, Jung-Rim, Hwang, Seong-Taek, Lee, Dong-Hyuk
• Granted Patent: US 8,675,916 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A movement recognition method and a user interface are provided. A skin color is detected from a reference face area of an image. A movement-accumulated area, in which movements are accumulated, is detected from sequentially accumulated image frames. Movement information corresponding to the skin color is detected from the detected movement-accumulated area. A user interface screen is created and displayed using the detected movement information.

Claims

1. A user interface apparatus that uses movement recognition, comprising:

a skin color learning unit for detecting a skin color from a reference face area of an image;

a movement-accumulated area detector for detecting a movement-accumulated area, in which movements are accumulated, from sequentially accumulated image frames;

a movement detector for detecting movement information corresponding to the skin color from the detected movement-accumulated area; and

a user interface forming unit for creating and displaying a user interface screen using the detected movement information.

2. The user interface apparatus of claim 1, further comprising a face detector for detecting the reference face area from the image

3. The user interface apparatus of claim 2, wherein the face detector determines whether a plurality of faces are detected from the image, and when the plurality of faces are detected, the face detector detects the reference face area in accordance with whether one or more of the plurality of faces are located in a center of the image and relative sizes of the plurality of faces.

4. The user interface apparatus of claim 1, wherein the skin color learning unit learns a color probability distribution value of the reference face area.

5. The user interface apparatus of claim 1, further comprising a movement area detector for detecting a movement area corresponding to the skin color from the detected movement-accumulated area, wherein the movement information is detected from the movement area of the movement-accumulated area.

6. The user interface apparatus of claim 5, wherein the movement area detector detects the movement area as having pixels with color values that correspond to the skin color in the detected movement-accumulated area, while a size of the movement-accumulated area is increasing.

7. The user interface apparatus of claim 5, wherein the movement information comprises a direction and a position of the movement area.

8. The user interface apparatus of claim 7, wherein the direction of the movement area is a direction in which a size of a movement trajectory area increases, and wherein the movement trajectory area is obtained by accumulating a trajectory along which the movement area has moved.

9. The user interface apparatus of claim 1, wherein the user interface forming unit generates an icon corresponding to the movement area and creates a screen in which the generated icon corresponds to the movement information.

10. The user interface apparatus of claim 1, further comprising a movement-accumulated image generator for generating, from the image, the movement-accumulated image in which the image frames, which show the movements, are sequentially accumulated.

11. A movement recognition method for a user interface, the method comprising the steps of:

detecting a skin color from a reference face area of an image;

detecting a movement-accumulated area, in which movements are accumulated, from sequentially accumulated image frames;

detecting movement information corresponding to the skin color from the detected movement-accumulated area; and

creating and displaying a user interface screen using the detected movement information.

12. The method of claim 11, further comprising detecting the reference face area from the image.

13. The method of claim 11, wherein detecting the reference face area comprises:

determining whether a plurality of faces are detected from the image; and

when the plurality of faces are detected, detecting the reference face area in accordance with whether one or more of the plurality of faces are located in a center of the image and relative sizes of the plurality of faces.

14. The method of claim 11, wherein detecting the skin color comprises:

calculating a color probability distribution value of the reference face area.

15. The method of claim 11, further comprising detecting a movement area corresponding to the skin color from the detected movement-accumulated area, wherein the movement information is detected from the movement area of the movement-accumulated area.

16. The method of claim 15, wherein detecting the movement area comprises detecting the movement area as having pixels with color values that correspond to the skin color in the detected movement-accumulated area, while a size of the movement-accumulated area is increasing.

17. The method of claim 15, wherein the movement information comprises a direction and a position of the movement area.

18. The method of claim 17, wherein the direction of the movement area is a direction in which a size of a movement trajectory area increases, and wherein the movement trajectory area is obtained by accumulating a trajectory along which the movement area has moved.

19. The method of claim 11, wherein forming the user interface screen comprises:

generating an icon corresponding to the movement area; and

creating a screen in which the generated icon corresponds to the movement information.

20. The method of claim 11, further comprising generating, from the image, the movement-accumulated image in which the image frames, which show the movements, are sequentially accumulated.