Image processing apparatus and method using smart glass

• US 9,460,351 B2
• Filed: 09/24/2013
• Issued: 10/04/2016
• Est. Priority Date: 11/28/2012
• Status: Expired due to Fees

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: You, Woong Shik, Kim, Kwi Hoon, Bae, Jae Hwui, Ahn, Sang Woo, Kwon, O Hyung
• Primary Examiner(s): AN, SHAWN S
• Application Number:
  US14/035,803
• Publication Number:
  US 20140146178A1
• Time in Patent Office: 1,106 Days
• US Class Current: 1/1
• CPC Class Codes: G01B 11/022 by means of tv-camera scanning
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19621 Portable camera
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/2743 Video hosting of uploaded data from client
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An image processing apparatus using a smart glass is provided, the image processing apparatus including an extractor to extract a first target to be observed by a user from a first image received from the smart glass and generate a second image using the extracted first target, an information collector to collect information related to the first target, and an image generator to reconstruct a third image using at least one of the second image and the information related to the first target, based on user settings.

Claims

1. An image processing apparatus using a smart glass, the apparatus comprising:

an extractor to extract a first target to be observed by a user from a first image received from the smart glass and generate a second image using the extracted first target;

an information collector to collect information related to the first target; and

an image generator to generate a third image using at least one of the second image and the information related to the first target, based on user settings in relation to the first target,

wherein, when the apparatus provides a time machine slow motion service to replay a scene having passed quickly in a perimeter of the user in a slow motion, the first target includes at least one of a sound, an event, and a predetermined time interval with respect to the first image, and the third image is a reconstructed image in a time machine slow motion and includes an automatic generation and indexing function with respect to a predetermined event desired by the user or a start event of the slow motion using sound recognition.

2. The apparatus of claim 1, wherein the first target further comprises at least one of a motion, an object, and a predetermined person to be observed by the user.

3. The apparatus of claim 1, wherein the first image is captured from a viewpoint corresponding to a gaze of the user.

4. The apparatus of claim 1, wherein the second image in which an image quality with respect to a portion corresponding to the first target is greater than or equal to a first resolution, and an image quality with respect to a remaining portion excluding the first target is less than the first resolution, is generated.

5. The apparatus of claim 1, wherein the extractor comprises:

a motion recognizer to recognize a predetermined motion of the user,

wherein the extractor provides the user with information related to the predetermined motion of the user.

6. The apparatus of claim 1, wherein the image generator comprises:

a comparator to compare a previous image associated with the first target to the third image and provide a result of the comparing.

7. The apparatus of claim 1, wherein the image generator recommends an image for each category of the information related to the first target, based on the information related to the first target.

8. The apparatus of claim 1, wherein the third image comprises additional information provided using at least one of a graphic and a character.

9. The apparatus of claim 1, wherein the smart glass comprises a depth camera or a 360-degree view angle camera, and is connected to the Internet.

10. The apparatus of claim 1, wherein, when the apparatus further provides a synchronized multiple fitting service to enable the user to scan a list of clothes currently owned by the user through a mirror including the smart glass and to provide the user with information relating to the clothes, the first target includes at least one of a current appearance of the user, the list of clothes, a list of recommendations, and a previous image corresponding to a current season or weather information, and

wherein the image generator provides the user with fitting images moving in identical patterns for the user to compare an appearance in current clothes to an appearance in other clothes through the mirror, recommends an image for each category of the information related to the first target, and provides the user with the recommended image along with the third image.

11. An image processing method using a smart glass, the method comprising:

receiving a first image from the smart glass, and extracting a first target to be observed by a user from the first image;

generating a second image using the extracted first image;

collecting information related to the first target; and

generating a third image using at least one of the second image and the information related to the first target, based on user settings in relation to the first target,

wherein, when the method provides a time machine slow motion service to replay a scene having passed quickly in a perimeter of the user in a slow motion, the first target includes at least one of a sound, an event, and a predetermined time interval with respect to the first image, and the third image is a reconstructed image in a time machine slow motion and includes an automatic generation and indexing function with respect to a predetermined event desired by the user or a start event of the slow motion using sound recognition.

12. The method of claim 11, wherein the first target further comprises at least one of a motion, an object, and a predetermined person to be observed by the user.

13. The method of claim 11, wherein the first image is captured from a viewpoint corresponding to a gaze of the user.

14. The method of claim 11, wherein the second image in which an image quality with respect to a portion corresponding to the first target is greater than or equal to a first resolution, and an image quality with respect to a remaining portion excluding the first target is less than the first resolution, is generated.

15. The method of claim 11, further comprising:

comparing a previous image associated with the first target to the third image and providing a result of the comparing on one side of the third image.

16. The method of claim 11, further comprising:

providing a recommended image for each category of the information related to the first target on one side of the third image.

17. The method of claim 11, wherein, when the method further provides a synchronized multiple fitting service, the method further comprises:

enabling the user to scan a list of clothes currently owned by the user through a mirror including the smart glass;

providing the user with information relating to the clothes;

providing the user with fitting images moving in identical patterns for the user to compare an appearance in current clothes to an appearance in other clothes through the mirror;

recommending an image for each category of the information related to the first target; and

providing the user with the recommended image along with the third image,

wherein the first target includes at least one of a current appearance of the user, the list of clothes, a list of recommendations, and a previous image corresponding to a current season or weather information.

18. An image processing apparatus using a smart glass, the apparatus comprising:

an extractor to extract a first target to be observed by a user from a first image received from the smart glass and generate a second image using the extracted first target;

an information collector to collect information related to the first target; and

an image generator to generate a third image using at least one of the second image and the information related to the first target, based on user settings in relation to the first target,

wherein, when the apparatus provides a virtual coexistence remote tour service to enable an image captured using the smart glass in a second place to be viewed identically in a first place and to provide the user in the first place with an effect similar to being in the second place, the first image refers to an image captured from a viewpoint corresponding to a gaze of the user with respect to the second place and the first target includes a predetermined scene or a landscape of the second place, and

wherein, for an augmented reality sharing function between user smart glasses of the first place and the second place, the third image represents a facial expression or a motion using a character for each user, or provides additional information including a speech bubble and a navigation map using a graphic.

19. An image processing method using a smart glass, the method comprising:

receiving a first image from the smart glass, and extracting a first target to be observed by a user from the first image;

generating a second image using the extracted first image;

collecting information related to the first target; and

generating a third image using at least one of the second image and the information related to the first target, based on user settings in relation to the first target,

wherein, when the method provides a virtual coexistence remote tour service, the method further comprises;

enabling an image captured using the smart glass in a second place to be viewed identically in a first place; and

providing the user in the first place with an effect similar to being in the second place,

wherein the first image refers to an image captured from a viewpoint corresponding to a gaze of the user with respect to the second place and the first target includes a predetermined scene or a landscape of the second place, and

wherein, for an augmented reality sharing function between user smart glasses of the first place and the second place, the third image represents a facial expression or a motion using a character for each user, or provides additional information including a speech bubble and a navigation map using a graphic.