MOBILE TERMINAL AND METHOD OF OPERATING THEREOF

• US 20180061137A1
• Filed: 08/29/2017
• Published: 03/01/2018
• Est. Priority Date: 08/30/2016
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: JUNG, Jaeduck
• Granted Patent: US 10,373,389 B2
• Time in Patent Office: Days
• CPC Class Codes: A47L 2201/04 Automatic control of the travelling movement; Automatic obstacle detection
• CPC Class Codes: A47L 9/009 Carrying-vehicles; Arrangements of trollies or wheels; Means for avoiding mechanical obstacles
• CPC Class Codes: A47L 9/2805 Parameters or conditions being sensed
• CPC Class Codes: A47L 9/2852 Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles
• CPC Class Codes: A47L 9/2857 User input or output elements for control, e.g. buttons, switches or displays
• CPC Class Codes: A47L 9/2894 Details related to signal transmission in suction cleaners
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A mobile terminal includes a camera, a display configured to display a real image projected by the camera, and a controller configured to recognize the real image to request a traveling path corresponding to the real image from a robot cleaner, to generate an augmented reality (AR) image of the traveling path based on the traveling path received from the robot cleaner, and to display the AR image to be superimposed on the real image.

Claims

1. A mobile terminal comprising:

a camera;

a display configured to display a real image projected by the camera; and

a controller configured to recognize the real image to request a traveling path corresponding to the real image from a robot cleaner, to generate an augmented reality (AR) image of the traveling path based on the traveling path received from the robot cleaner, and to display the AR image to be superimposed on the real image.

2. The mobile terminal according to claim 1, wherein the AR image of the traveling path includes at least one of a cleaned region and an uncleaned region.

3. The mobile terminal according to claim 2, wherein, if the cleaned region and the uncleaned region are included in the AR image of the traveling path, the controller distinguishably displays the cleaned region and the uncleaned region.

4. The mobile terminal according to claim 2, wherein the controller denotes the cleaned region by a solid line and denotes the uncleaned region by a dotted line.

5. The mobile terminal according to claim 2, wherein, if the uncleaned region is included in the AR image of the traveling path, the controller controls the display to display a notification message of the uncleaned region.

6. The mobile terminal according to claim 1, wherein the controller displays the AR image of the traveling path in the form of at least one of a stereoscopic image, a holographic image and a moving image.

7. The mobile terminal according to claim 1, wherein the real image includes a region where cleaning is performed by the robot cleaner.

8. The mobile terminal according to claim 1, wherein the controller generates and displays the AR image to be superimposed on the real image, when the robot cleaner in a docking state is projected onto the real image.

9. An augmented reality system comprising:

a robot cleaner configured to perform cleaning while automatically traveling; and

a mobile terminal including a display configured to display a real image projected by a built-in camera and a controller configured to recognize the real image to request a traveling path corresponding to the real image from a robot cleaner, to generate an augmented reality (AR) image of the traveling path based on the traveling path received from the robot cleaner, and to display the AR image to be superimposed on the real image.

10. The AR system according to claim 9, wherein the AR image of the traveling path includes at least one of a cleaned region and an uncleaned region.

11. The AR system according to claim 10, wherein, if the cleaned region and the uncleaned region are included in the AR image of the traveling path, the controller distinguishably displays the cleaned region and the uncleaned region.

12. The AR system according to claim 10, wherein the controller denotes the cleaned region by a solid line and denotes the uncleaned region by a dotted line.

13. The AR system according to claim 9, wherein the controller displays the AR image of the traveling path in the form of at least one of a stereoscopic image, a holographic image and a moving image.

14. The AR system according to claim 9, wherein the controller generates and displays the AR image to be superimposed on the real image, when the robot cleaner in a docking state is projected on the real image.

15. A recording medium having recorded thereon a computer-readable program for performing a method of operating a mobile terminal, the method comprising

displaying a real image projected by a built-in camera;

recognizing the real image to request a traveling path corresponding to the real image from a robot cleaner;

generating an augmented reality (AR) image of the traveling path based on the traveling path received from the robot cleaner; and

displaying the AR image to be superimposed on the real image.

16. The recording medium according to claim 15, wherein the AR image of the traveling path includes at least one of a cleaned region and an uncleaned region.

17. The recording medium according to claim 16, wherein the displaying of the AR image to be superimposed on the real image includes distinguishably displaying the cleaned region and the uncleaned region if the cleaned region and the uncleaned region are included in the AR image of the traveling path.

18. The recording medium according to claim 15, wherein the generating of the AR image includes:

mapping the traveling path received from the robot cleaner to a traveling path of a virtual space; and

generating the AR image of the traveling path in the mapped virtual space.

19. The recording medium according to claim 15, wherein the real image includes a region where cleaning is performed by the robot cleaner.

20. The recording medium according to claim 15, wherein the method further includes generating and displaying the AR image to be superimposed on the real image, when the robot cleaner in a docking state is projected onto the real image.