GESTURE-BASED USER INTERFACE METHOD AND APPARATUS

• US 20130050425A1
• Filed: 08/24/2011
• Published: 02/28/2013
• Est. Priority Date: 08/24/2011
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: Sangki Kim, Yongwon Cho
• Inventors: Im, Soungmin, Yu, Sunjin, Lim, Kyungyoung, Kim, Sangki, Cho, Yongwon
• Granted Patent: US 8,830,302 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence

Abstract

A gesture-based user interface method and corresponding apparatus that includes a light source configured to irradiate light to a user, an image sensor configured to receive light reflected from the user and output a depth image of the user, an image processor configured to recognize a user gesture based on the depth image output from the image sensor, and a controller configured to control the light source and the image sensor such that at least one of an optical wavelength of the light source, an optical power level of the light source, a frame rate of the light source and a resolution of the depth image is adjusted according to a gesture recognition mode.

Claims

1. An apparatus configured to recognize a user gesture, comprising:

a light source configured to irradiate light;

an image sensor configured to

receive a portion of the light that is reflected from a user,

convert the received light into a depth image of the user, and

output the depth image; and

a controller operatively connected to the light source and the image sensor, the controller configured to control at least one of an optical wavelength of the light source, an optical power level of the light source, a frame rate of the light source and a resolution of the image sensor is controlled according to a gesture recognition mode of the apparatus.

2. The apparatus of claim 1,

wherein the gesture recognition mode includes a first mode for recognizing a whole-body gesture and a second mode for recognizing a gesture involving a part of the body, and

wherein a recognition range of the second mode is narrower than a recognition range of the first mode.

3. The apparatus of claim 2,

wherein the controller is configured to control the light source such that the optical wavelength is set to a first wavelength in the first mode and the optical wavelength is set to a second wavelength in the second mode, and

wherein the second wavelength is shorter than the first wavelength.

4. The apparatus of claim 2,

wherein the controller is configured to control the light source such that the frame rate is set to a first frame rate in the first mode and the frame rate is set to a second frame rate in the second mode, and

wherein the second frame rate is greater than the first frame rate.

5. The apparatus of claim 2,

wherein the controller is configured to control the image sensor such that the resolution of the depth image is set to a first resolution in the first mode and the resolution of the depth image is set to a second resolution in the second mode, and

wherein the second resolution is higher than the first resolution.

6. The apparatus of claim 1,

wherein the gesture recognition mode includes

a short distance mode in which a distance between the apparatus and the user is less than a reference distance, and

a long distance mode in which the distance is greater than the reference distance,

wherein the controller is configured to set the optical power level to a first power value in the short distance mode and set the optical power level to a second power value in the long distance mode, and

wherein the second power value is greater than the first power value.

7. The apparatus of claim 1, further comprising:

an image processor operatively connected to the controller and configured to recognize the gesture of the user based on the depth image.

8. The apparatus of claim 7, further comprising:

a memory operatively connected to the controller and configured to store information regarding a command corresponding to the recognized gesture of the user,

wherein, upon recognizing the gesture of the user, the controller is configured to output the command to a content display apparatus based on the information regarding the command.

9. An apparatus configured to recognize a user gesture, comprising:

a camera system that includes

a electromagnetic wave source configured to irradiate electromagnetic waves; and

an image sensor configured to

receive a portion of the irradiated electromagnetic waves that is reflected from a user,

convert the received electromagnetic waves into an electrical signal, and

output the electrical signal; and

a controller operatively connected to the camera system, the controller configured to select at least one of a first field of view and a second field of view of the camera system according to a gesture recognition mode of the apparatus, the second field of view being wider than the first field of view.

10. The apparatus of claim 9, wherein the camera system includes a first depth camera and a second depth camera.

11. The apparatus of claim 10, wherein the first depth camera has the first field of view and the second depth camera has the second field of view.

12. The apparatus of claim 10, wherein the first depth camera and the second depth camera are configurable to photograph gestures of different users.

13. The apparatus of claim 10, wherein the first depth camera is configurable to provide a front image of the user and the second depth camera is configurable to provide a side image of the user.

14. The apparatus of claim 10, wherein the first depth camera is a time of flight (TOF) camera and the second depth camera is a structured light camera.

15. The apparatus of claim 9, further comprising:

an image processor operatively connected to the controller and configured to recognize the gesture of the user based on the electrical signal.

16. A method of recognizing a user gesture by an apparatus having a light source and an image sensor, the method comprising:

controlling an operation of the apparatus according to a gesture recognition mode of the apparatus, the gesture recognition mode including settings for controlling at least one of an optical wavelength of the light source, an optical power level of the light source, a frame rate of the light source and an image resolution of the image sensor,

wherein the operation of the apparatus includes

irradiating light by the light source;

receiving a portion of the light that is reflected from a user;

converting the received light into a depth image of the user by the image sensor; and

outputting the depth image.

17. The method of claim 16,

wherein the gesture recognition mode includes a first mode for recognizing a whole- body gesture and a second mode for recognizing a gesture involving only a part of the body, and

wherein a recognition range of the second mode is narrower than a recognition range of the first mode.

18. The method of claim 17,

wherein the step of controlling includes setting the optical wavelength a first wavelength in the first mode and setting the optical wavelength to a second wavelength in the second mode, and

wherein the second wavelength is shorter than the first wavelength.

19. The method of claim 17,

wherein the step of controlling includes setting the frame rate to a first frame rate in the first mode and setting the frame rate to a second frame rate in the second mode, and

wherein the second frame rate is greater than the first frame rate.

20. The gesture-based user interface method of claim 17,

wherein the step of controlling includes setting the image resolution to a first resolution in the first mode and setting the image resolution to a second resolution in the second mode, and

wherein the second resolution is higher than the first resolution.

21. The gesture-based user interface method of claim 16,

wherein the gesture recognition mode includes a short distance mode in which a distance between the apparatus and the user is less than a reference distance and a long distance mode in which the distance is greater than the reference distance,

wherein the step of controlling includes setting the optical power level to a first power value in the short distance mode and setting the optical power level is set to a second power value in the long distance mode, and

wherein the second power level is greater than the first power value.

22. The method of claim 16, further comprising:

recognizing the gesture of the user based on the depth image.

23. The method of claim 22, further comprising:

storing information regarding a command corresponding to the recognized gesture of the user; and

outputting the command corresponding to the recognized gesture of the user to a content display apparatus based on the information regarding the command.

24. A method of recognizing a user gesture by an apparatus including a camera system, the method comprising:

selecting a field of view of the camera system according to a gesture recognition mode of the apparatus, the selected field of view being one of a first field of view and a second field of view, the second field of view being wider than the first field of view;

irradiating electromagnetic waves from the apparatus;

receiving, by the camera system according to the selected field of view, a portion of the irradiated electromagnetic waves that is reflected from a user;

converting the received electromagnetic waves into an electrical signal; and

outputting the electrical signal.

25. The method of claim 24, wherein the camera system includes a first depth camera and a second depth camera.

26. The method of claim 25, wherein the first depth camera has the first field of view and the second depth camera has the second field of view.

27. The method of claim 25, further comprising:

configuring the first depth camera and the second depth camera to photograph gestures of different users.

28. The method of claim 25, wherein the first depth camera is configurable to provide a front image of the user and the second depth camera is configurable to provide a side image of the user.

29. The method of claim 25, wherein the first depth camera is a time of flight (TOF) camera and the second depth camera is a structured light camera.

30. The method of claim 24, further comprising:

recognizing the gesture of the user based on electrical signal.