Wearable device and method for providing augmented reality information

• US 10,484,673 B2
• Filed: 05/29/2015
• Issued: 11/19/2019
• Est. Priority Date: 06/05/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jin, Youngtae
• Primary Examiner(s): Hasan, Mainul
• Application Number:
  US14/725,396
• Publication Number:
  US 20150358614A1
• Time in Patent Office: 1,635 Days
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/211 using temporal multiplexing
• CPC Class Codes: H04N 13/293 Generating mixed stereoscopic images; Generating mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/339 using spatial multiplexing H04N13/337 takes precedence

Abstract

An electronic device includes a camera module configured to capturing an image through a camera module of a wearable device, a transceiver configured to perform a functionally coupling between an electronic device and a wearable device, a display module, and a processor functionally coupled to the camera module and the display module, wherein the processor is configured to transmit the image to the electronic device to generate augmented reality using the captured image through the transceiver, receive the augmented reality on a display module of the wearable device through the transceiver, detect a user'"'"'s hand gesture through the camera module, and control the display module to display a change of augmented reality information, based on the detected user'"'"'s hand gesture.

Claims

1. A method for providing augmented reality, the method comprising:

generating augmented reality using an image captured through a camera module of a wearable device, the image comprising a plurality of real objects;

displaying the augmented reality on a display of the wearable device, the augmented reality including a plurality of virtual objects, wherein each virtual object corresponds to one of a plurality of real objects that are points of interest;

detecting at least one finger of a user within the augmented reality by the camera module;

displaying a first window selecting a virtual object corresponding to the real object that is a point of interest, wherein the finger is associated with the first window; and

in response to detecting that a finger of the user is bent to at least a predetermined angle, transitioning the first window into a second window different from the first window, displayed on the display of the wearable device, that provides detailed information regarding the real object corresponding to the virtual object.

2. The method of claim 1, further comprising, when the bent finger is moved, displaying a further change of the augmented reality on the display.

3. The method of claim 1, wherein:

detecting at least one finger further comprises detecting at least two fingers of the user within the augmented reality, and

in response to detecting a gestures of the at least two fingers, performing at least one of;

displaying a reduced form of the virtual object,

displaying an enlarged form of the virtual object, or

displaying a moved form of the virtual object.

13. The method of claim 1, wherein the detailed information comprises at least one of a text or an image describing the real object that is associated with the virtual object.

14. The method of claim 1, wherein the virtual object is an icon.

15. The method of claim 1, wherein displaying the first window comprises transposing the window on a location of the virtual object within the augmented reality.

16. The method of claim 1, wherein displaying the first window comprises displaying the first window according to a location of the at least one finger within the augmented reality.

4. A wearable electronic device comprising:

a camera module;

a display module; and

a processor configured to;

generate augmented reality using an image captured through the camera module;

display the augmented reality on the display module, the augmented reality including a plurality of virtual objects, wherein each virtual object corresponds to one of a plurality of real objects that are points of interest;

detect at least one finger of a user detected within the augmented reality by the camera module;

display a first window selecting a virtual object corresponding to the real object that is a point of interest, wherein the at least one finger is associated with the first window;

detect that a finger, of the at least one detected finger, of the user is bent to at least a predetermined angle; and

in response to detecting that the finger is bent to at least the predetermined angle, transitioning the first window into a second window different from the first window, displayed on the display, that provides detailed information regarding the real object corresponding to the virtual object.

5. The wearable electronic device of claim 4, wherein the processor is further configured to, when the bent finger is moved, control the display module to display a further change of the augmented reality on the display.

6. The wearable electronic device of claim 4, wherein to detect at least one finger of the user, the processor is further configured to:

detect at least two fingers of the user within the augmented reality, and

in response to detecting at least two fingers, control the display to display at least one of;

a reduced form of the virtual object,

an enlarged form of the virtual object, or

a moved form of the virtual object.

7. A method for providing augmented reality information, the method comprising:

establishing a communication channel between an electronic device and a wearable device;

capturing, by the wearable device, an image comprising a plurality of real objects through a camera module of a wearable device;

transmitting, by the wearable device, the image to the electronic device to generate augmented reality using the captured image, the augmented reality including a plurality of virtual objects, wherein each virtual object corresponds to one of a plurality of real objects that are points of interest;

in response to detecting, by the wearable device, at least one finger of a user, transmitting, by the wearable device a location of each of the at least one finger to the electronic device to display a first window selecting a virtual object corresponding to the real object that is a point of interest, wherein the finger is associated with the first window;

receiving and displaying, by the wearable device, the augmented reality on a display module of the wearable device; and

in response to detecting that a finger, of the at least one finger, is being bent to at least a predetermined angle, transitioning the first window into a second window different from the first window, displayed on the display module, that provides detailed information regarding the real object corresponding to the virtual object,

wherein a hand gesture of the user comprises at least one of bending the finger or moving the finger.

8. The method of claim 7, wherein:

detecting at least one finger further comprises detecting at least two fingers of the user within the augmented reality, and

in response to detecting a gesture of the at least two fingers, performing at least one of;

displaying a reduced form of the virtual object,

displaying an enlarged form of the virtual object, or

displaying a moved form of the virtual object.

9. The method of claim 7, wherein displaying the augmented reality includes displaying, transparently or semitransparently, each of the virtual objects as an overlay on each of the real objects, respectively.

10. A wearable device comprising:

a camera module configured to capturing an image through a camera module of the wearable device;

a transceiver configured to establish a communication channel between an electronic device and the wearable device;

a display module; and

a processor configured to;

transmit the image to the electronic device to generate augmented reality using the captured image through the transceiver, the augmented reality including a plurality of virtual objects, wherein each virtual object corresponds to one of a plurality of real objects that are points of interest;

in response to detecting at least one finger of a user, transmit, through the transceiver, a location of each of the at least one finger to the electronic device to display a first window selecting a virtual object corresponding to the real object that is a point of interest, wherein the finger is associated with the first window;

receive the augmented reality on a display module of the wearable device through the transceiver; and

in response to detecting that a finger, of the at least one finger, is being bent to at least a predetermined angle, control the display module to transition the first window into a second window, different from the first window and displayed on the display module, that provides detailed information regarding the real object corresponding to the virtual object,

wherein a hand gesture of the user comprises at least one of bending the finger or moving the finger.

11. The wearable device of claim 10, wherein to detect at least one finger of the user, the processor is further configured to:

detect at least two fingers of the user within the augmented reality, and

in response to detecting at least two fingers, control the display to display at least one of;

a reduced form of the virtual object,

an enlarged form of the virtual object, or

a moved form of the virtual object.

12. The wearable device of claim 10, wherein the processor is further configured to control the display module to display, transparently or semitransparently, each of the virtual objects as an overlay over each of the real objects, respectively.