System and method for navigating a 3D graphical user interface

• US 8,384,718 B2
• Filed: 01/10/2008
• Issued: 02/26/2013
• Est. Priority Date: 01/10/2008
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Mobile Communications AB (Sony Group Corp.)
• Inventors: Dahlke, Mikael
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US11/972,019
• Publication Number:
  US 20090179914A1
• Time in Patent Office: 1,874 Days
• Field of Search: 345/467
• US Class Current: 345/467
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A system enables a user viewing a plurality of graphical objects rendered on a display screen of a mobile device to manipulate the renderings of each graphical object for purposes of selecting a first of the graphical objects. The system comprises the display screen and a facial proximity system determining the distance of the mobile device from the face of the user. A graphical user interface control system drives a rendering of the graphical objects on the display screen. Further, upon receiving an indication of a change in distance between the mobile device and the face of the user and, upon determining that the distance between the mobile device and the face of the user has changed, the graphical user interface control system modifies at least one of the size of the rendering of the graphical objects and the perspective of the rendering of the graphical objects on the display screen.

Claims

1. A system for navigating a graphical user interface of a mobile device, the system comprising:

a display screen;

a facial proximity system determining the distance of the mobile device from the face of the user;

a three-dimensional graphical user interface control system configured to;

drive a base rendering of a graphical object in perspective on the display screen, wherein the graphical object represents multiple sub-objects, each of the multiple sub-objects representing individually selectable data or functions of the mobile device; and

receive an indication of a change in distance between the mobile device and the face of the user and, upon determining that the distance between the mobile device and the face of the user has changed, modify the perspective of the rendering of the graphical object on the display screen;

wherein the indication of a change in distance between the mobile device and the face of the user includes an indication that the distance between the mobile device and the face of the user has decreased; and

modifying the perspective of the rendering of the graphical object on the display screen comprises replacing a rendering of a single object with a rendering of each of the multiple sub-objects.

2. The system of claim 1, wherein the graphical user interface system further, in response to receiving an indication of a reset event, modifies the perspective of the renderings of the graphical object on the display screen to return to the base rendering of the graphical object.

3. The system of claim 2, wherein the facial proximity system comprises:

a camera directed towards the face of the user and capturing a sequence of images of the face of the user;

a distance analysis module;

determining that the distance between the mobile device and the face of the user has increased when a size measurement of the user'"'"'s face within an image is smaller than the size measurement of the user'"'"'s face within a previous image; and

determining that the distance between the mobile device and the face of the user has decreased when the size measurement of the user'"'"'s face within an image is larger than the size measurement of the user'"'"'s face within a previous image; and

in each case, generating the indication of a change in distance between the mobile device and the face of the user.

4. The system of claim 3, wherein the facial proximity system further comprises:

a viewing angle module determining a change in the user'"'"'s viewing angle of the display screen; and

the graphical user interface control system further receives an indication of the change in viewing angle of the display screen and, upon determining that the viewing angle has changed, modifying the perspective of the rendering of the graphical object on the display screen.

5. The system of claim 4, wherein the viewing angle module determines the change in the user'"'"'s viewing angle of the display screen by determining that a position of the user'"'"'s face within an image of the sequence of images is displaced from a position of the user'"'"'s face within a previous image of the sequence of images.

6. The system of claim 4, modifying the perspective of the rendering of the graphical object on the display screen comprises replacing at least one selectable object with a non-selectable object.

7. The system of claim 1:

further comprising an accelerometer generating a signal representing acceleration of the mobile device; and

the viewing angle module receives the signal representing acceleration and determines a change in viewing angle of the display screen by determining that a change in perspective of the users face between an image and a previous image and the signal representing acceleration of the mobile device both represent a similar change in viewing angle.

8. The system of claim 7, wherein the accelerometer further generates a selection signal representing a distinctive selection motion of the mobile device;

and

the graphical user interface control further receives the selection signal and, in response thereto, one of i) repositions a graphical selector between a first object and a second object on the display screen; and

ii) generates a selection signal to an application indicating selection of the graphical object on which the graphical selector is located.

9. The system of claim 2, wherein the facial proximity system comprises:

a transmitter directing a signal directed towards the face of the user;

a receiver capturing a reflection of the signal from the face of the user;

a distance analysis module;

determining that the distance between the mobile device and the face of the user has increased when a time measurement between directing the signal and the receiver capturing a reflection of the signal is greater than a previous time measurement between directing the signal and the receiver capturing a reflection of the signal; and

determining that the distance between the mobile device and the face of the user has decreased when the time measurement between directing the signal and the receiver capturing a reflection of the signal is less than a previous time measurement between directing the signal and the receiver capturing a reflection of the signal; and

in each case, generating the indication of a change in distance between the mobile device and the face of the user.

10. The system of claim 9, wherein:

the indication of a change in distance between the mobile device and the face of the user includes an indication that the distance between the mobile device and the face of the user has decreased; and

modifying the perspective of the rendering of the graphical object on the display screen comprises replacing a rendering of single object representing multiple sub objects with a rendering of each of the multiple sub-objects.

11. The system of claim 9, wherein the facial proximity system further comprises:

an accelerometer generating a signal representing acceleration of the mobile device; and

a viewing angle module determining a change in the user'"'"'s viewing angle of the display screen by determining a viewing angle change associated with the signal representing acceleration of the mobile device; and

the graphical user interface control system further receives an indication of the change in viewing angle of the display screen and, upon determining that the viewing angle has changed, modifying the perspective of the rendering of the graphical object on the display screen.

12. The system of claim 11, modifying the perspective of the rendering of the graphical object on the display screen comprises replacing at least one selectable object with a non-selectable object.

13. The system of claim 11, wherein the accelerometer further generates a selection signal representing a distinctive selection motion of the mobile device;

and

the graphical user interface control further receives the selection signal and, in response thereto, one of i) repositions a graphical selector between a first object and a second object on the display screen; and

ii) generates a selection signal to an application indicating selection of the graphical object on which the graphical selector is located.

14. The system of claim 2, wherein the facial proximity system comprises:

an accelerometer generating a signal representing acceleration of the mobile device

a distance analysis module;

determining that the distance between the mobile device and the face of the user has increased by determining that the signal is consistent with such an increase; and

determining that the distance between the mobile device and the face of the user has decreased by determining a change in viewing angle of the display screen and the face of the user by determining a viewing angle change associated with the signal representing acceleration of the mobile device; and

in each case, generating the indication of a change in distance between the mobile device and the face of the user.

15. The system of claim 14, wherein the facial proximity system further comprises:

a viewing angle module determining a change in the user'"'"'s viewing angle of the display screen by determining a viewing angle change associated with the signal representing acceleration of the mobile device; and

the graphical user interface control system further receives an indication of the change in viewing angle of the display screen and, and upon determining that the viewing angle has changed, modifying the perspective of the rendering of the graphical object on the display screen.

16. The system of claim 15, modifying the perspective of the rendering of the graphical object on the display screen comprises replacing at least one selectable object with a non-selectable object.