MOBILE TERMINAL AND METHOD FOR CONTROLLING OPERATION OF THE MOBILE TERMINAL

• US 20120062558A1
• Filed: 09/01/2011
• Published: 03/15/2012
• Est. Priority Date: 09/15/2010
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: LEE, Jinsool, Kim, Dongok, Kim, Taeyun, Seen, Seungmin
• Granted Patent: US 9,513,710 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A mobile terminal and a method for controlling the operation of the mobile terminal are provided. In the method, a stereoscopic 3D image that utilizes binocular disparity between a left-eye image and a right-eye image is displayed on a display and a stereoscopic 3D pointer is displayed in the stereoscopic 3D image when a user'"'"'s hand is detected according to analysis of visual data acquired by a camera. The position of the stereoscopic 3D pointer is moved according to movement of the hand when the movement of the hand is detected according to analysis of the visual data. According to this method, the operation of the mobile terminal can be controlled by the user'"'"'s hand movement or gesture.

Claims

1. A method of a mobile terminal, the method comprising:

displaying a perceived three dimensional (3D) image on a display;

receiving first visual data of an indicator from at least one camera;

displaying a perceived 3D pointer in the displayed 3D image based on the received first visual data of the indicator;

receiving second visual data of the indicator from the at least one camera; and

changing a position of the displayed 3D pointer on the display based on the received second visual data.

2. The method of claim 1, wherein the indicator is a hand.

3. The method of claim 1, wherein changing the position of the displayed 3D pointer includes changing the position of the displayed 3D pointer based on movement of the indicator.

4. The method of claim 1, further comprising:

detecting a gesture based on received visual data from the camera; and

performing an operation associated with an object indicated by the 3D pointer based on the detected gesture.

5. The method of claim 1, further comprising changing a perceived 3D depth of the displayed 3D pointer based on movement of the indicator.

6. The method of claim 1, further comprising:

determining that the indicator has not moved for a predetermined time based on received visual data from the camera; and

performing, in response to the determination that the indicator has not moved for the predetermined time, an operation associated with an object indicated by the 3D pointer.

7. The method of claim 1, further comprising outputting a feedback signal in response to a change of a perceived depth of the 3D pointer.

8. The method of claim 1, further comprising displaying depth information corresponding to a perceived depth of the 3D pointer in the 3D image.

9. The method of claim 8, wherein the depth information is displayed as a number, a graph, or an image.

10. The method of claim 1, further comprising displaying a menu for setting a function corresponding to a specific gesture of the indicator.

11. A mobile terminal comprising:

a display to display a perceived three-dimensional (3D) image;

at least one camera to acquire visual data; and

a controller to receive, from the camera, first visual data of an indicator, to display a perceived 3D pointer in the 3D image based on the received first visual data of the indicator, to receive second visual data of the indicator from the camera, and to change a position of the displayed 3D pointer based on the received second visual data.

12. The mobile terminal of claim 11, wherein the indicator is a hand.

13. The mobile terminal of claim 11, wherein the controller changes the position of the displayed 3D pointer based on movement of the indicator.

14. The mobile terminal of claim 11, wherein the controller detects a gesture based on received visual data from the camera, and the controller performs an operation associated with an object indicated by the 3D pointer based on the detected gesture.

15. The mobile terminal of claim 11, wherein the controller determines that the indicator has not moved for a predetermined time based on received visual data from the camera, and the controller performs an operation associated with an object indicated by the 3D pointer upon the determination that the indicator has not moved for a predetermined time.

16. The mobile terminal of claim 11, wherein the controller displays depth information of the 3D pointer in the 3D image.

17. The mobile terminal of claim 11, wherein the controller generates visual effects to the 3D pointer as the 3D pointer is closer to a specific object.

18. The mobile terminal of claim 11, further comprising a haptic module to generate tactile effects,

wherein the controller controls the haptic module to generate tactile effects based on a change of a depth of the 3D pointer.

19. A display method of a mobile terminal, the method comprising:

displaying an image on a display;

displaying a perceived three-dimensional (3D) pointer on the image;

receiving data of a user'"'"'s hand from a camera; and

changing the displayed 3D pointer based on the received data of the user'"'"'s hand.

20. The method of claim 19, wherein changing the displayed 3D pointer includes changing a position of the displayed 3D pointer based on movement of the hand.

21. The method of claim 19, further comprising:

detecting a gesture based on received data from the camera; and

performing an operation associated with an object indicated by the 3D pointer based on the detected gesture.

22. The method of claim 19, further comprising changing a perceived 3D depth of the displayed 3D pointer based on movement of the user'"'"'s hand.

23. The method of claim 19, further comprising:

determining that the hand has not moved for a predetermined time based on received data from the camera; and

performing, in response to the determination that the hand has not moved for the predetermined time, an operation associated with an object indicated by the 3D pointer.

24. The method of claim 19, further comprising selecting an object on the displayed image using the displayed 3D pointer.

25. The method of claim 19, wherein displaying the image includes displaying a perceived 3D image that utilizes binocular disparity between a first eye image and a second eye image.