Eyewear-type terminal and method for controlling the same

• US 10,061,391 B2
• Filed: 11/24/2015
• Issued: 08/28/2018
• Est. Priority Date: 05/26/2015
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kwon, Chohee, Han, Dongsu, Lim, Gukchan
• Primary Examiner(s): Rayan, Mihir K
• Application Number:
  US14/951,318
• Publication Number:
  US 20160349849A1
• Time in Patent Office: 1,008 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/63 by using electronic viewfinders

Abstract

Provided is a eyewear-type terminal capable of communicating with a drone. The eyewear-type terminal includes a communication unit configured to communicate with a drone, a display configured to output visual information received from the drone, an optical unit comprising at least one lens and configured to form an image of the visual information in an external virtual region such that the image is visible only to a user wearing the terminal, a sensor configured to sense a user gesture applied with respect to the image and a controller configured to control the drone based on the sensed user gesture.

Claims

1. An eyewear-type terminal comprising:

a communication unit configured to communicate with a drone;

a display configured to output visual information received from the drone;

an optical unit comprising at least one lens and configured to form a virtual image of the visual information at an external virtual region that is formed outside of the terminal such that the virtual image is visible only to a user wearing the terminal;

a sensor configured to sense a user gesture applied with respect to the virtual image; and

a controller configured to;

control the drone based on the sensed user gesture; and

control movement of the drone based on the sensed user gesture that is applied subsequently to a gesture of grabbing the virtual image such that;

the drone is moved when a manner of the gesture of grabbing the virtual image is a first manner; and

the drone is rotated when the manner of the gesture of grabbing the virtual image is a second manner.

2. The terminal of claim 1, wherein the controller is further configured to control at least one of a movement of the drone or a camera of the drone in response to a preset user gesture applied with respect to the virtual image.

3. The terminal of claim 2, wherein the controller is further configured to:

control the movement of the drone when the preset user gesture is a first user gesture; and

control the camera when the preset user gesture is a second user gesture.

4. The terminal of claim 3, wherein:

the first user gesture is a gesture that is applied subsequently to a gesture of grabbing the virtual image; and

the controller is further configured to cause the drone to move in response to the subsequently applied gesture.

5. The terminal of claim 4, wherein when the first user gesture is applied in a state in which the virtual image is displayed in a first display manner, the virtual image is displayed in a second display manner.

6. The terminal of claim 5, wherein the virtual image displayed in the second display manner is restored to be displayed in the first display manner when the first user gesture is released.

7. The terminal of claim 3, wherein:

the second user gesture comprises a pinch-in gesture or a pinch-out gesture applied with respect to the virtual image; and

the controller is further configured to control the camera to execute a zoom-out function or a zoom-in function in response to the pinch-in gesture or the pinch-out gesture.

8. The terminal of claim 7, wherein the controller is further configured to cause the drone to move when the pinch-out gesture is applied to be moved out of the virtual image.

9. The terminal of claim 1, further comprising a camera, wherein:

the controller is further configured to cause the display to display a preview image received via the camera; and

the optical unit is further configured to form an image of the preview image in addition to the virtual image of the visual information.

10. The terminal of claim 9, wherein the image of the preview image and the virtual image of the visual information are formed to be spaced apart from one another.

11. The terminal of claim 9, wherein when a preset type of gesture is applied to at least one of the image of the preview image or the virtual image of the visual information, the controller is further configured to cause the at least one of the image of the preview image or the virtual image of the visual information to move.

12. The terminal of claim 11, wherein when the image of the preview image and the virtual image of the visual information come into contact with each other due to the movement, the controller is further configured to capture the image of the preview image and the virtual image of the visual information.

13. An eyewear-type terminal comprising:

a communication unit configured to communicate with a drone;

a display configured to output visual information received from the drone;

an optical unit comprising at least one lens and configured to form a virtual image of the visual information at an external virtual region such that the virtual image is visible only to a user wearing the terminal;

a sensor configured to sense a user gesture applied with respect to the virtual image; and

a controller configured to;

control movement of the drone when the sensed user gesture is a first user gesture that is applied subsequently to a gesture of grabbing the virtual image, causing the drone to move in response to the subsequently applied gesture, wherein the drone is moved in a different manner according to a manner of the gesture of grabbing the virtual image; and

control a camera of the drone when the sensed user gesture is a second user gesture.

14. A method for controlling an eyewear-type terminal, the method comprising:

outputting visual information received from a drone via a display of the terminal;

forming a virtual image of the visual information, by an optical unit comprising at least one lens, in an external virtual region that is formed outside of the terminal such that the virtual image is visible only to a user wearing the terminal;

controlling the drone based on a user gesture applied with respect to the virtual image; and

controlling movement of the drone based on the user gesture that is applied subsequently to a gesture of grabbing the virtual image such that;

the drone is moved when a manner of the gesture of grabbing the virtual image is a first manner; and

the drone is rotated when the manner of the gesture of grabbing the virtual image is a second manner.

15. The method of claim 14, wherein:

the controlling of the drone comprises controlling at least one of a movement of the drone or a camera of the drone in response to a preset user gesture applied with respect to the virtual image;

the movement of the drone is controlled when the preset user gesture is a first user gesture; and

the camera is controlled when the preset user gesture is a second user gesture.

16. The method of claim 15, wherein:

the first user gesture is a gesture that is applied subsequently to the gesture of grabbing the virtual image; and

the controller is further configured to cause the drone to move in response to the subsequently applied gesture.

17. The method of claim 15, wherein:

the second user gesture comprises a pinch-in gesture or a pinch-out gesture applied with respect to the virtual image; and

the controller is further configured to control the camera to execute a zoom-out function or a zoom-in function in response to the pinch-in gesture or the pinch-out gesture.

18. The method of claim 14, further comprising:

forming an image of a preview image received via a camera of the terminal in addition to the virtual image of the visual information,

wherein at least one of the image of the preview image or the virtual image of the visual information is moved in response to a preset type of gesture applied to the at least one of the image of the preview image or the virtual image of the visual information.

19. The method of claim 18, further comprising:

capturing the image of preview image and the virtual image of the visual information when the image of the preview image and the virtual image of the visual information come into contact with each other due to the movement.