Glass type mobile terminal

• US 9,330,313 B2
• Filed: 05/29/2013
• Issued: 05/03/2016
• Est. Priority Date: 12/05/2012
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Jung, Juyung, Song, Hyunghun
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US13/904,871
• Publication Number:
  US 20140152696A1
• Time in Patent Office: 1,070 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 10/109 Time management, e.g. calendars, reminders, meetings or time accounting
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32

Abstract

A glass type mobile terminal including a transparent screen; a frame configured to secure the transparent screen in front of a user'"'"'s eyes wearing the glass type mobile terminal; a camera mounted to the frame and configured to photograph an image in front of the user'"'"'s eyes; a memory; an image recognition unit configured to extract information from the image photographed by the camera; and a controller configured to compare the extracted information with related information stored in the memory, and display the related information to the transparent screen on the transparent screen along with the captured image.

Claims

1. A glass type mobile terminal comprising:

a transparent screen;

a frame configured to secure the transparent screen in front of a user'"'"'s eyes wearing the glass type mobile terminal;

a camera mounted to the frame and configured to photograph an image in front of the user'"'"'s eyes;

a memory;

an image recognition unit configured to extract information from the image photographed by the camera; and

a controller configured to;

compare the extracted information with related information stored in the memory, and

display the related information to the transparent screen on the transparent screen along with the captured image,

wherein the image recognition unit is further configured to extract the information when one object occupies a preset range of areas included in the image.

2. The glass type mobile terminal according to claim 1, wherein the controller is further configured to only store the extracted information from the image photographed by the camera.

3. The glass type mobile terminal according to claim 1, further comprising:

an audio output module configured to output sound information,

wherein the controller is further configured to output the related information via the audio output module.

4. The glass type mobile terminal according to claim 1, wherein the image includes a calendar, the extracted information includes at least one of a year, month and date of the calendar and a memo written on the calendar, and the related information in the memory corresponds to at least one of the year, month, date and memo written on the calendar, and

wherein the controller is further configured to superimpose the related information on the calendar.

5. The glass type mobile terminal according to claim 4, wherein the controller is further configured to zoom in/out and move the related information on the transparent screen according to the user'"'"'s movement.

6. The glass type mobile terminal according to claim 4, further comprising:

a wireless communication unit connected with the internet,

wherein the controller is further configured to search the internet for the related information related to the calendar and superimpose the related information on the transparent screen.

7. The glass type mobile terminal according to claim 1, wherein the image is a memo and the extracted information is a character written on the memo,

wherein the controller is further configured to display at least part of stored-information corresponding to the written character on the transparent screen, and

wherein the controller is further configured to select whether to merge or group the information extracted from the memo with the related information or store the extracted information as independent information.

8. The glass type mobile terminal according to claim 7, wherein the controller is further configured to display the stored-information as a thumbnail image.

9. The glass type mobile terminal according to claim 1, wherein the image is a memo and the extracted information is a character written on the memo, and

wherein the image recognition unit is further configured to perform a character recognition process for recognizing a character newly written in real time on the memo, convert the recognized character into a text, and display the converted text on the transparent screen.

10. The glass type mobile terminal according to claim 1, wherein the image is a memo and the extracted information is a character written on the memo, and

wherein the controller is further configured to prompt the user to store the memo as a new document or a new page.

11. The glass type mobile terminal according to claim 1, further comprising:

a wireless communication unit connected to the internet,

wherein the image is a memo and the extracted information is a character written on the memo, and

wherein the controller is further configured to search the internet for a characteristic of a character selected from the memo and display information about the searched characteristic on the transparent screen.

12. The glass type mobile terminal according to claim 1, further comprising:

a microphone configured to convert a voice of a cashier indicating an amount of money needed to be paid into a digital signal,

wherein the image includes an image of money the user is paying, and

wherein the controller is further configured to;

compare the converted voice with the image of the money the user is paying,

calculate an amount of change due to the user, and

output the calculated amount of change on the transparent screen or the audio output module.

13. The glass type mobile terminal according to claim 1, wherein the image is a receipt, and the extracted information corresponds to individual items included in the receipt, and

wherein the controller is further configured to update an account book stored in the memory to include the extracted information.

14. A glass type mobile terminal comprising:

a transparent screen;

a frame configured to secure the transparent screen in front of a user'"'"'s eyes wearing the glass type mobile terminal;

a camera mounted to the frame and configured to photograph an image in front of the user'"'"'s eyes, the image includes an object,

a memory storing related information, the related information includes at least one image of the object photographed at one or more angles,

an image recognition unit configured to extract information or the object from the image photographed by the camera;

an audio output module configured to output sound information;

a location information module configured to recognize a location of the glass type mobile terminal; and

a controller configured to;

compare the extracted information or object with the related information stored in the memory,

determine whether the object in the photographed image corresponds to the object included in the related information,

record the location and photographed time in the memory as a log record for the object, when the object in the photographed image corresponds to the object included in the related information, and

output the related information or the log record via the transparent screen or the audio output module.

15. The glass type mobile terminal according to claim 14, wherein the controller is further configured to display additional information of the object on the transparent screen or output the additional information via the audio output module, when the object in the photographed image corresponds to the object included in the related information.

16. The glass type mobile terminal according to claim 15, wherein the additional information includes a designated time when a current time is approaching the designated time.

17. A glass type mobile terminal comprising:

a transparent screen;

a frame configured to secure the transparent screen in front of a user'"'"'s eyes wearing the glass type mobile terminal;

a camera mounted to the frame and configured to photograph an image in front of the user'"'"'s eyes, the image includes a memo;

a memory;

an image recognition unit configured to extract information from the image photographed by the camera, the extracted information is a character written on the memo; and

a controller configured to;

compare the extracted information with related information stored in the memory,

display the related information to the transparent screen on the transparent screen along with the captured image, and

prompt the user to store the memo as a new document or a new page.

18. The glass type mobile terminal according to claim 17, wherein the controller is further configured to display at least part of stored-information corresponding to the written character on the transparent screen, and

wherein the controller is further configured to select whether to merge or group the information extracted from the memo with the related information or store the extracted information as independent information.

19. The glass type mobile terminal according to claim 17, wherein the image recognition unit is further configured to perform a character recognition process for recognizing a character newly written in real time on the memo, convert the recognized character into a text, and display the converted text on the transparent screen.

20. The glass type mobile terminal according to claim 17, further comprising:

a wireless communication unit connected to the internet,

wherein the controller is further configured to search the internet for a characteristic of a character selected from the memo and display information about the searched characteristic on the transparent screen.