Mobile terminal and method of controlling operation thereof
First Claim
1. A method of controlling an operation of a mobile terminal, the method comprising:
- communicatively connecting the mobile terminal with a head mounted display for displaying an augmented reality (AR) screen;
receiving first data from the head mounted display, the received first data includes gesture input data acquired by a first camera at the head mounted display and image data of an image corresponding to an environment acquired by the first camera;
receiving second data from the head mounted display, the received second data includes image data acquired by a second camera at the head mounted display;
analyzing the first and second data;
determining a predetermined gesture based on the analyzed first data and determining an eye position of a user based on the analyzed second data; and
controlling the head mounted display to display an indicator corresponding to a shape of the determined gesture on the AR screen, wherein the indicator is displayed at the determined eye position of the user, and wherein the controlling of the head mounted display includes controlling the head mounted display to display a closed curve region on the AR screen when the predetermined gesture is determined to correspond to a closed curve gesture.
1 Assignment
0 Petitions
Accused Products
Abstract
A mobile terminal and a method of controlling an operation thereof are disclosed. In the method of controlling the operation of the mobile terminal, the mobile terminal is communicatively with a head mounted display for displaying an augmented reality (AR) screen obtained by combining a virtual image with a real environment. If image data acquired through a camera is analyzed and a predetermined gesture input is detected, the head mounted display is controlled such that a display change corresponding to the detected gesture input is displayed on the AR screen. In a state of displaying the AR screen through the head mounted display, various operations related to the mobile terminal can be controlled according to the predetermined gesture input.
-
Citations
24 Claims
-
1. A method of controlling an operation of a mobile terminal, the method comprising:
-
communicatively connecting the mobile terminal with a head mounted display for displaying an augmented reality (AR) screen; receiving first data from the head mounted display, the received first data includes gesture input data acquired by a first camera at the head mounted display and image data of an image corresponding to an environment acquired by the first camera; receiving second data from the head mounted display, the received second data includes image data acquired by a second camera at the head mounted display; analyzing the first and second data; determining a predetermined gesture based on the analyzed first data and determining an eye position of a user based on the analyzed second data; and controlling the head mounted display to display an indicator corresponding to a shape of the determined gesture on the AR screen, wherein the indicator is displayed at the determined eye position of the user, and wherein the controlling of the head mounted display includes controlling the head mounted display to display a closed curve region on the AR screen when the predetermined gesture is determined to correspond to a closed curve gesture. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A mobile terminal comprising:
-
a wireless communication unit configured to communicatively connect the mobile terminal with a head mounted display for displaying an augmented reality (AR) screen; and a controller configured to; receive first data from the head mounted display, the received first data includes gesture input data acquired by a first camera at the head mounted display and image data of an image corresponding to an environment acquired by the first camera; receive second data from the head mounted display, the received second data includes image data acquired by a second camera at the head mounted display; analyze the first and second data; determine a predetermined gesture based on the analyzed first data and determine an eye position of a user based on the analyzed second data; and control the head mounted display to display an indicator corresponding to a shape of the determined gesture on the AR screen, wherein the indicator is displayed at the determined eye position of the user, and wherein the controller enables to display a closed curve region on the AR screen when the predetermined gesture is determined to correspond to a closed curve gesture. - View Dependent Claims (15, 16, 17, 18, 19)
-
-
20. A mobile terminal comprising:
-
a wireless communication unit configured to communicatively connect the mobile terminal with a head mounted display for displaying an augmented reality (AR) screen; and a controller configured to receive data from the head mounted display, the received data includes gesture input data acquired by a camera at the head mounted display and image data of an image corresponding to an environment acquired by the camera, the controller to analyze the data and determine a predetermined gesture based on the analyzed data, and to control the head mounted display on the AR screen based on the determined gesture, wherein the controller enables to display a closed curve region on the AR screen when the predetermined gesture is determined to correspond to a closed curve gesture, and wherein the controller enables to display a text input window for attaching a tag to the closed curve region on a display and sets text input through the text input window as a tag of the closed curve region, when a gesture input of tapping a boundary of the closed curve region is detected.
-
-
21. A system comprising:
-
a head mounted display configured to display an augmented reality (AR) screen; and a mobile terminal communicatively connected to the head mounted display and configured to; receive first data from the head mounted display, the received first data includes gesture input data acquired by a first camera at the head mounted display and image data of an image corresponding to an environment acquired by the first camera; receive second data from the head mounted display, the received second data includes image data acquired by a second camera at the head mounted display; analyze the first and second data; determine a predetermined gesture based on the analyzed first data, and determine an eye position of a user based on the analyzed second data; and control the head mounted display to display an indicator corresponding to a shape of the determined gesture on the AR screen, wherein the indicator is displayed at the determined eye position of the user, and wherein the mobile terminal controls the head mounted display to display a closed curve region on the AR screen when the predetermined gesture is determined to correspond to a closed curve gesture. - View Dependent Claims (22, 23, 24)
-
Specification