METHOD FOR USER GESTURE RECOGNITION IN MULTIMEDIA DEVICE AND MULTIMEDIA DEVICE THEREOF

• US 20120119985A1
• Filed: 02/24/2011
• Published: 05/17/2012
• Est. Priority Date: 11/12/2010
• Status: Abandoned Application

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: KANG, Mingoo
• Application Number:
  US13/033,988
• Publication Number:
  US 20120119985A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A display device includes a first sensor to detect a first image of a person and a second sensor to detect a second image of the person. A storage device stores first information and second information, where the first information identifies a plurality of gestures mapped to respective ones of a plurality of functions identified by the second information. A processor recognizes a gesture of the person based on the first and second images, and performs a function corresponding to the recognized gesture based on the first and second information.

Claims

1. A display device, comprising:

a first sensor to acquire a first image of a person;

a second sensor to acquire a second image of the person;

a storage device to store first information and second information, the first information identifying a plurality of gestures mapped to respective ones of a plurality of functions identified by the second information; and

a processor to recognize a gesture of the person based on the first and second images, and to perform a function corresponding to the recognized gesture based on the first and second information stored in the storage device, wherein;

the second image is detected based on the first image,

the functions are different ones of the display device, and

the gestures are made with at least one body part of the person captured in the second image.

2. The display device of claim 1, wherein the processor determines coordinate information of the person in the first image and extracts the gesture of the person in the second image based on the coordinate information.

3. The display device of claim 2, wherein the coordinate information includes distance information.

4. The display device of claim 1, wherein the first sensor is a depth image sensor and the second sensor is a color image sensor.

5. The display device of claim 1, wherein the second image includes a zoomed view of the at least one body part.

6. The display device of claim 1, wherein the recognized gesture corresponds to an orientation of the at least one body part.

7. The display device of claim 1, wherein the second image is a higher resolution image than the first image.

8. The display device of claim 1, wherein the processor:

forms combined info illation based on the first and second images, and

recognizes the gesture based on the combined information.

9. The display device of claim 1, wherein when the body part is in motion:

information from the first and second sensors are combined to recognize the gesture, the information from the first sensor including distance information and the information from the second sensor including color information.

10. The display device of claim 1, wherein the second sensor tracks movement of the at least one body part based on information from the first sensor.

11. The display device of claim 1, wherein the gesture corresponds to the trace of a letter in air and wherein the at least one body part is a finger or hand used to trace the letter.

12. The display device of claim 1, further comprising:

a storage device to store information identifying a plurality of gestures for each of a plurality of users, each of the plurality of gestures corresponds to a different one of the functions and wherein one or more functions mapped to the gestures of a first user are different from one or more functions mapped to the same gestures of the second user.

13. A display device, comprising:

a first sensor to acquire a first image of a person;

a second sensor to acquire a second image of the person;

a storage device to store first information and second information, the first information identifying a plurality of gestures mapped to respective ones of a plurality of functions identified by the second information; and

a processor to perform a function based on the first and second images,

wherein, in a first mode, the processor recognizes a gesture of the person based on the first and second images, and to perform a function corresponding to the recognized gesture based on the first and second information stored in the storage device, and

wherein, in a second mode, the processor recognizes a trajectory of a body part moved by a user, the processor recognizing movement of the body part as a letter or number based on the first and second images, the processor to perform a function corresponding to the recognized movement, and wherein;

the second image is detected based on the first image, and

the functions are different ones of the display device.

14. The display device of claim 13, wherein the first sensor is a depth image sensor and the second sensor is a color image sensor.

15. The display device of claim 13, wherein, in the first mode, the processor determines coordinate information of the person in the first image and extracts the gesture of the person in the second image based on the coordinate information.

16. The display device of claim 13, wherein the coordinate information includes distance information.

17. The display device of claim 13, wherein the second image includes a zoomed view of the at least one body part.

18. A control method comprising:

storing first information and second information, the first information identifying a plurality of gestures mapped to respective ones of a plurality of functions identified by the second information; and

detecting a first image of a person;

detecting a second image of the person;

recognizing a gesture of the person based on the first and second images; and

performing a function corresponding to the recognized gesture based on the first and second information, wherein the second image is detected based on the first image, the functions are different ones of a display device, and the gestures are made with at least one body part of the person captured in the second image.

19. The method of claim 18, wherein said recognizing includes:

determining coordinate information of the person in the first image, and

extracting the gesture of the person in the second image based on the coordinate information.

20. The method of claim 19, wherein the coordinate information includes distance information.