METHOD AND APPARATUS FOR PERFORMING GESTURE RECOGNITION USING OBJECT IN MULTIMEDIA DEVICES

• US 20120119987A1
• Filed: 11/07/2011
• Published: 05/17/2012
• Est. Priority Date: 11/12/2010
• Status: Abandoned Application

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: IM, Soungmin, Park, Soungsoo, Kang, Mingoo
• Application Number:
  US13/290,833
• Publication Number:
  US 20120119987A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

According to an embodiment of the present invention, a gesture recognition method for use in a multimedia device includes capturing, via an image sensing unit of the multimedia device, a peripheral image, recognizing a first object contained in the captured peripheral image and a gesture made using the first object, mapping a multimedia device operation to the gesture, and entering into an input standby mode associated with the gesture.

Claims

1. A gesture recognition method for use in a multimedia device, the method comprising:

capturing, via an image sensing unit of the multimedia device, a peripheral image;

recognizing a first object contained in the captured peripheral image and a gesture made using the first object;

mapping a multimedia device operation to the gesture; and

entering into a gesture input standby mode for receiving another gesture.

2. The gesture recognition method according to claim 1, wherein the capturing of the peripheral image includes:

acquiring, via a depth image sensor of the image sensing unit, location information of the first object contained in the peripheral image; and

acquiring, via an RGB image sensor of the image sensing unit, an image of a specific part at which the first object is located according to the acquired location information.

3. The gesture recognition method according to claim 1, wherein the recognizing of the first object includes:

extracting characteristic information of each object including the first object from the captured image;

searching for a second object associated with the extracted characteristic information of the first object from a database (DB) stored in the multimedia device; and

recognizing information of the searched second object corresponds to information of the first object.

4. The gesture recognition method according to claim 1, wherein the mapping between the multimedia device operation and the gesture includes:

receiving, via the image sensing unit, the gesture made using the recognized object;

receiving a selection signal of a multimedia device operation mapped to the received gesture; and

storing mapping data in response to the selection signal.

5. A gesture recognition method for use in a multimedia device, the method comprising:

capturing, via an image sensing unit of the multimedia device, a peripheral image;

recognizing a first object contained in the captured image and a gesture made using the first object;

executing an application associated with the recognized first object;

mapping a multimedia device operation to the gesture; and

entering into an input standby mode associated with the gesture and the executed application.

6. The gesture recognition method according to claim 5, wherein the executing of the application includes:

searching for appearance information of the recognized first object;

searching for an application corresponding to the recognized first object in a database (DB) of the multimedia device based on the searched appearance information; and

executing the searched application.

7. The gesture recognition method according to claim 5, wherein the capturing of the peripheral image includes:

acquiring, via a depth image sensor of the image sensing unit, location information of the first object contained in the peripheral image; and

acquiring, via an RGB image sensor of the image sensing unit, an image of a specific part at which the first object is located according to the acquired location information.

8. The gesture recognition method according to claim 5, wherein the recognizing of the first object includes:

extracting characteristic information of each object including the first object from the captured image;

searching for a list of objects associated with the extracted characteristic information of the each object from the captured image from a database (DB) stored in the multimedia device; and

recognizing that information of a second object from the retrieved list of objects corresponds to information of the first object.

9. The gesture recognition method according to claim 5, wherein the mapping between the multimedia device operation and the gesture includes:

receiving, via the image sensing unit, the gesture made using the recognized first object;

receiving a selection signal of an operation of the executed application in order to map the operation of the executed application to the received gesture; and

storing mapping data in response to the selection signal.

10. A multimedia device for recognizing a user gesture, the multimedia device comprising:

an image sensing unit configured to capture a peripheral image;

an image recognition unit configured to analyze the peripheral image captured by the image sensing unit and to recognize a first object contained in the captured image and a gesture made using the first object;

a storage unit configured to store mapping data between the gesture made using the first object and a multimedia device operation; and

a controller configured to search for the mapping data of the first object recognized by the image recognition unit in the storage unit, to load the mapping data, and to enter into a gesture input standby mode for receiving another gesture.

11. The multimedia device according to claim 10, wherein the image sensing unit includes:

a depth image sensor configured to acquire information about a distance from the depth image sensor to a target object; and

an RGB image sensor configured to acquire color information.

12. The multimedia device according to claim 10, wherein the storage unit stores characteristic information of the first object, and the image recognition unit extracts characteristic information of each object including the first object from the peripheral image captured by the image sensing unit, searches for a second object associated with the extracted characteristic information of the first object from the storage unit, and recognizes information of the second object corresponds to information of the first object.

13. The multimedia device according to claim 10, wherein the controller receives the gesture from a user who handles the first object through the image sensing unit, receives a selection signal of a multimedia device operation mapped to the received gesture, and stores mapping data in the storage unit.

14. A multimedia device for recognizing a user gesture, the multimedia device comprising:

an image sensing unit configured to capture a peripheral image;

an image recognition unit configured to analyze the image captured by the image sensing unit, and to recognize a first object contained in the captured image and a gesture made using the first object;

an application execution unit configured to search and execute an application corresponding to the recognized first object;

a storage unit configured to store mapping data between the gesture made using the first object and an application operation; and

a controller configured to load the mapping data corresponding to the executed application from the storage unit, and to enter into an input standby mode associated with the gesture and the executed application operation.

15. The multimedia device according to claim 14, further comprising:

a display configured to display an image, wherein the application execution unit searches for appearance information of the first object recognized by the image recognition unit, searches for an application corresponding to the recognized first object, and executes the searched application.

16. The multimedia device according to claim 14, wherein the image sensing unit includes:

a depth image sensor configured to acquire information about a distance from the depth image sensor to a target object; and

an RGB image sensor configured to acquire color information.

17. The multimedia device according to claim 14, wherein the storage unit stores characteristic information of the first object, and the image recognition unit extracts characteristic information of each object including the first object from the peripheral image captured by the image sensing unit, searches for a second object associated with the extracted characteristic information of the first object from the storage unit, and recognizes information of the second object corresponds to information of the first object.

18. The multimedia device according to claim 14, wherein the controller receives the gesture from a user who handles the first object through the image sensing unit, receives a selection signal of a multimedia device operation mapped to the received gesture, and stores mapping data in the storage unit.