Graphical user interface adjusting to a change of user's disposition

• US 8,890,812 B2
• Filed: 10/25/2012
• Issued: 11/18/2014
• Est. Priority Date: 10/25/2012
• Status: Active Grant

• Current Assignee: Lumentum Operations, LLC (Lumentum Holdings Inc.)
• Original Assignee: JDS Uniphase Corporation (Lumentum Holdings Inc.)
• Inventors: Billerbeck, Bryed
• Primary Examiner(s): OSORIO, RICARDO
• Application Number:
  US13/660,468
• Publication Number:
  US 20140118255A1
• Time in Patent Office: 754 Days
• Field of Search: 345156-158, 345/173, 348/77
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2320/028 by changing the viewing angle properties, e.g. widening the viewing angle, adapting the viewing angle to the view direction
• CPC Class Codes: G09G 2320/068 for control of viewing angle adjustment
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

A user interface apparatus and a method for adjusting the apparatus are disclosed. A position and/or a viewing angle of the user is tracked, and graphical interface objects are adjusted to keep them visible at different user'"'"'s distances and viewing angles. For example, as the user steps away from the display, the objects on the display can be proportionally enlarged to make them appear of the same size to the user. The sensitivity of a gesture recognition system to the user'"'"'s movements and gestures can be also adjusted to facilitate manipulation of the objects by the user at different distances from the display of the graphical user interface.

Claims

1. A method of operating a gesture recognition based user interface comprising a display for displaying a plurality of graphical objects, and a spatial detector for detecting a distance and/or a viewing angle of a user relative to the display, the method comprising:

(a) using the spatial detector to detect a change of the distance and/or the viewing angle of the user relative to the display, wherein the change of the distance and/or the viewing angle of the user relative to the display results in a change of a sensitivity of the gesture recognition;

(b) automatically adjusting at least one of position, orientation, size, shape, color, brightness, or contrast of at least one of the plurality of graphical objects displayed by the display, in dependence upon the change detected in step (a), so as to lessen a change of visible shape or size, and/or a change of visibility of the at least one graphical object to the user, resulting from the change detected in step (a);

(c) automatically adjusting the gesture recognition sensitivity in dependence on the change of the distance and/or the viewing angle of the user, detected in step (a), so as to lessen the change of the gesture recognition sensitivity due to the change of the distance and/or viewing angle of the user; and

(d) repeating steps (a) through (c) by continuously adjusting the at least one of position, orientation, size, shape, color, brightness, or contrast of the at least one graphical object, so as to lessen changes of the visible shape or size, and/or changes of the visibility of the at least one graphical object, resulting from changes of the distance and/or the viewing angle of the user relative to the display; and

by continuously adjusting the gesture recognition sensitivity, so as to lessen the change of the gesture recognition sensitivity due to the change of the distance and/or viewing angle of the user.

2. The method of claim 1, wherein in step (b), the adjustment of color includes adjustment of color saturation, wherein the adjustment of the at least one of position, orientation, size, color saturation, brightness, or contrast is substantially proportional to the change of the distance and/or the viewing angle of the user, detected in step (a).

3. The method of claim 1, wherein in step (c), the gesture recognition sensitivity adjustment is substantially proportional to the change of the distance and/or the viewing angle of the user, detected in step (a).

4. The method of claim 1, wherein in step (b), the adjustment of the at least one of position, orientation, size, shape, color, brightness, or contrast is performed predictively based on the user'"'"'s past movements and/or gestures.

5. The method of claim 1, wherein in step (c), the adjustment of the sensitivity of the gesture recognition is performed predictively based on the user'"'"'s past movements and/or gestures.

6. The method of claim 1, wherein the at least one graphical object comprises a cluster of neighboring graphical objects of the plurality of graphical objects.

7. The method of claim 6, wherein the at least one graphical object comprises each of the plurality of graphical objects.

8. The method of claim 1, wherein in step (b), the shape and size of the at least one graphical object are adjusted to keep angular size thereof substantially constant when the distance and the viewing angle of the user are within a range of interaction between the user and the user interface.

9. The method of claim 1, wherein the display comprises a three-dimensional display, wherein in step (b), the position and/or orientation of the at least one of the plurality of graphical objects is characterized by two lateral coordinates and a depth coordinate, wherein the depth coordinate of the at least one graphical object is adjusted so as to lessen a perceived distance variation between the user and the at least one graphical object, as the user moves within the range of interaction with the user interface.

10. The method of claim 9, wherein in step (b), at least one of the two lateral coordinates of the at least one graphical object is adjusted, so as to cause the at least one graphical object to follow a body of the user, as the user moves within the range of interaction with the user interface.

11. The method of claim 10, wherein steps (a) through (d) are performed for each of a plurality of users, each user corresponding to a particular one of the plurality of graphical objects, wherein in step (b), the position of each of the plurality of graphical objects is adjusted to follow a body of the respective user.

12. A method for dynamically adjusting a gesture recognition based graphical user interface for interaction with a user, the method comprising:

(i) determining a distance and/or a viewing angle of the user relative to the graphical user interface;

(ii) detecting a change of the distance and/or the viewing angle determined in step (i), wherein the change of the distance and/or the viewing angle of the user relative to the display results in a change of a sensitivity of the gesture recognition;

(iii) dynamically adjusting at least one of position, orientation, size, shape, color, brightness, or contrast of at least one of a plurality of graphical objects displayed by the graphical user interface, so as to lessen a change of shape and/or size of the at least one graphical object visible to the user, resulting from the change of the distance and/or the viewing angle thereof detected in step (ii); and

(iv) adjusting the gesture recognition sensitivity in dependence on the change of the distance and/or the viewing angle of the user, detected in step (i), so as to lessen the change of the gesture recognition sensitivity due to the change of the distance and/or viewing angle of the user detected in step (ii).

13. The method of claim 12, wherein in step (iii), a magnitude of the adjustment of the size of the at least one graphical object is substantially proportional to the change of the distance detected in step (ii).

14. The method of claim 13, wherein the shape of the at least one graphical object is characterized by a vertical height and a horizontal width with respect to gravity, wherein the viewing angle is horizontal with respect to the gravity, and

wherein in step (iii), the adjustment of the width of the at least one graphical object is substantially inversely proportional to cosine square of the viewing angle determined in step (i).

15. The method of claim 13, wherein the brightness and/or the contrast of the at least one graphical object are substantially proportional to the change of the distance determined in step (ii).

16. A user interface apparatus, comprising:

a display for displaying a plurality of graphical objects to a user;

a spatial detector for determining a distance and/or a viewing angle of the user relative to the display; and

a controller operationally coupled to the display and the spatial detector, the controller including;

a user locator module coupled to the spatial detector, for detecting a change of the distance and/or the viewing angle of the user relative to the display;

a graphical user interface adjustment module coupled to the user locator module and to the display, for automatically adjusting at least one of position, orientation, size, shape, color, brightness, or contrast of at least one of the plurality of graphical objects displayed by the display, in dependence upon the change of the distance and/or the viewing angle detected by the user locator module, so as to lessen a change of visible shape or size, and/or a change of visibility of the at least one graphical object to the user, resulting from the detected change of the distance and/or the viewing angle of the user;

a gesture recognition module for recognizing gestures of the user; and

a gesture sensitivity adjustment module coupled to the user locator module and to the gesture recognition module, for automatically adjusting a sensitivity of the gesture recognition module to the user'"'"'s gestures in dependence on the change of the distance and/or the viewing angle of the user detected by the user locator module, so as to lessen a change of the sensitivity of the gesture recognition module to the user'"'"'s gestures caused by the change of the distance and/or the viewing angle of the user.

17. The user interface apparatus of claim 16, wherein the controller further comprises a learning module coupled to the user locator module, for learning past changes of the distance and/or the viewing angle of the user relative to the display, to determine possible future changes of the distance and/or the viewing angle of the user, for performing a predictive adjustment of the at least one graphical object.

18. The user interface apparatus of claim 16, wherein the display comprises a three-dimensional display.

19. The user interface apparatus of claim 16, wherein the spatial detector comprises a time-of-flight camera.