APPARATUS AND METHOD FOR CONTROLLING USER INTERFACE TO SELECT OBJECT WITHIN IMAGE AND IMAGE INPUT DEVICE

• US 20140096084A1
• Filed: 09/30/2013
• Published: 04/03/2014
• Est. Priority Date: 09/28/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kwon, Dong-Wook, Kim, Min-Ho, Kim, Kyung-ll, Lee, Gi-Sang, Lee, Sang-Bo, Jin, Young-Gu, Choi, Jin-Wuk, Lee, Jin-Kyung
• Granted Patent: US 10,101,874 B2
• Time in Patent Office: Days
• US Class Current: 715/835
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

A method for controlling a user interface of an apparatus to enable intuitive manipulation for an object included in an image is provided. An object with deeper depth in a rear image of the apparatus may become selectable when the user holds a user input means close to the apparatus, while an object with shallower depth in the rear image may become selectable when the user draws the user input means away from the apparatus, thereby allowing the user to intuitively select an object in the rear image.

Claims

1. An apparatus for controlling a user interface to select an object included in an image, the apparatus comprising:

a first image input unit configured to output a first image obtained by capturing a user input means;

a second image input unit placed on an opposite side of the first image input unit configured to output a second image obtained by capturing what is ahead;

a display unit configured to display the second image; and

a controller configured to control to recognize multiple objects within the second image, and to display the second image by changing a selectable object among the multiple objects based on a distance between the user input means and the display unit.

2. The apparatus of claim 1, further comprising:

a first depth sensor configured to obtain depth information about the first image; and

a second depth sensor configured to obtain depth information about the second image.

3. The apparatus of claim 1, wherein the controller controls to display the second image by changing selectable objects sequentially among the multiple objects from an object with shallower depth to an object with deeper depth as the distance between the user input means and the display unit decreases.

4. The apparatus of claim 3, wherein the controller determines a sequence of the selectable objects based on the depth information about the second image.

5. The apparatus of claim 2, wherein the controller measures the distance between the user input means and the display unit based on the depth information about the first image.

6. The apparatus of claim 1, wherein the controller displays a selectable object that depends on the distance between the user input means and the display unit in the display unit by using an indicator.

7. The apparatus of claim 1, wherein the display unit is implemented as a touch screen, and upon selection of a selectable object, displays the object to be expanded or reduced based on a touch input of the user input means on the touch screen.

8. The apparatus of claim 1, wherein the display unit displays the multiple objects within the second image with respective information about the multiple objects matched with the multiple objects.

9. An image input apparatus for outputting an image that contains multiple objects, the image input apparatus comprising:

a first image input unit configured to output a first image obtained by capturing a user input means and providing depth information about the first image that represents a distance between the user input means and the image input apparatus; and

a second image input unit arranged on an opposite side of the first image input unit configured to output a second image that includes multiple objects having different depths, such that the multiple objects become selectable sequentially from an object with shallow depth to an object with deeper depth as the distance decreases.

10. The image input apparatus of claim 9, wherein the second image input unit comprises a depth sensor configured to recognize the multiple objects within the second image and providing respective depth information of the multiple objects.

11. The image input apparatus of claim 9, wherein the first image input unit comprises a depth sensor configured to provide depth information about the first image.

12. The image input apparatus of claim 9, wherein the image input apparatus is established on a mobile terminal with a display unit, and wherein a selectable object is changed among the multiple objects based on the distance, the selectable object being identified with an indicator.

13. A method for selecting an object included in an image in an apparatus for controlling a user interface, the method comprising:

generating a first image by capturing a user input means;

generating a second image obtained by capturing what is ahead of the apparatus;

recognizing and displaying multiple objects within the second image;

measuring a distance between a user input means and the apparatus based on a first image; and

changing and displaying a selectable object among the multiple objects based on the measured distance.

14. The method of claim 13, further comprising:

capturing the first image with a first image input unit and capturing the second image with a second image input unit arranged on an opposite side of the first image input unit; and

receiving respective depth information about the first image and the second image through the first image input unit and the second image input unit, respectively.

15. The method of claim 14, wherein the distance between the user input means and the apparatus is measured using the depth information about the first image.

16. The method of claim 14, wherein changing and displaying a selectable object comprises:

changing and displaying selectable objects sequentially from an object with shallow depth to an object with deeper depth among the multiple objects based on the respective depth information about the first image and the second image, as the distance decreases.

17. The method of claim 16, further comprising:

determining the selectable sequence of the multiple objects based on depth image about the second image.

18. The method of claim 13, wherein changing and displaying a selectable object comprises:

identifying the selectable object with an indicator.

19. The method of claim 13, wherein recognizing and displaying multiple objects within a second image comprises:

searching for respective information for the multiple objects; and

displaying the multiple objects by matching with the respective information.

20. The method of claim 13, further comprising:

upon selection of a selectable object, displaying the selectable object to be expanded or reduced according to a touch input of the user input means.