Graphical user interface for displaying and navigating in a directed graph structure

• US 20020060701A1
• Filed: 01/23/2002
• Published: 05/23/2002
• Est. Priority Date: 05/24/1993
• Status: Active Grant

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Frank, Edward H., Clanton, Charles H. III, Warth, Chris, LaVallee, David A., Palrang, Joseph M., Naughton, Patrick J., Gosling, James A., Sheridan, R. Michael
• Granted Patent: US 7,240,289 B2
• Time in Patent Office: Days
• US Class Current: 715/853
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 9/465 Distributed object oriented systems remote method invocation [RMI] G06F9/548
• CPC Class Codes: G09B 5/14 with provision for individual teacher-student communication
• CPC Class Codes: H03J 1/0025 in a remote control unit remote control tuning in general H03J9/00
• CPC Class Codes: H03J 2200/25 Remote control device with display
• CPC Class Codes: H03J 2200/26 Remote control device with touch screen display
• CPC Class Codes: H04B 1/202 by remote control
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/2809 indicating that an appliance service is present in a home automation network monitoring functionality H04L43/0817; discovery or management thereof, e.g. service location protocol [SLP] or web services, H04L67/51
• CPC Class Codes: H04L 12/2818 from a device located outside both the home and the home network access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; arrangements for transmitting signals characterised by the use of a wireless eletrical link G08C17/00; telephonic communication systems adapted for combination with remote control systems H04M11/007
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 2012/2841 Wireless
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 69/329 : in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 : Network security protocols
• CPC Class Codes: H04M 1/72415 : for remote control of appliances
• CPC Class Codes: H04N 21/4113 : PC
• CPC Class Codes: H04N 21/41265 : having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/4131 : home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/42204 : User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/426 : Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4312 : involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 : for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/43615 : Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/43622 : Interfacing an external recording device
• CPC Class Codes: H04N 21/47 : End-user applications
• CPC Class Codes: H04N 21/47214 : for content reservation or setting reminders; for requesting event notification, e.g. of sport results or stock market stock exchange data over packet-switching network H04L12/1804; push services over packet-switching network H04L12/1859; notification of incoming messages in packet switching networks H04L51/224
• CPC Class Codes: H04N 21/4722 : for requesting additional data associated with the content
• CPC Class Codes: H04N 21/478 : Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/482 : End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/4821 : using a grid, e.g. sorted out by channel and broadcast time
• CPC Class Codes: H04N 21/485 : End-user interface for client configuration
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8586 : by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02
• CPC Class Codes: H04N 7/17318 : Direct or substantially direct transmission and handling of requests
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: Y10S 715/977 : Dynamic icon, e.g. animated or live action

Abstract

An intuitive graphical user interface is based upon a geographic map structure, and includes a system for controlling remote external electronic devices. In the defined graphical user interface, each space of the geographic map structure is rendered on a touch screen display as a graphic image of a geographic space. Within each space are colored cartoon-like icons called “objects” which can be selected and manipulated by the user. Certain objects, referred to as portals, transport the user from one space to another space when Selected. Other objects, referred to as buttons, perform associated actions or functions when Selected. The graphical user interface is displayed on a hand-held display device used to control remote devices. Each remote electronic device transmits a user interface program object that defines a graphical user interface to the display device. The hand-held display devices displays the graphical user interface associated with the remote device when a user selects an icon associated with the remote device on the display device'"'"'s touch screen display.

Claims

1. A method of providing a graphical user interface on a computer system, said computer system comprising a CPU, a memory, and graphics display screen, said method comprising the steps of:

providing a plurality of spaces, each of said spaces comprising a graphic background image;

organizing said spaces in a directed graph structure, said spaces connected together with a plurality of portals;

locating said plurality of portals in said spaces, each of said portals having an associated target space such that said portal connects the space said portal is in with said associated target space, each of said portals comprising a graphic icon;

rendering a graphic background image of a first space and portals located in said first space on said graphics display screen, said first space having at least a first portal;

receiving input from a user, said input comprising a portal selected from said portals located in said first space; and

rendering a graphic background image of a target space associated with said selected portal on said graphics display screen;

such that said user can move between spaces in said directed graph structure by selecting said portals within said spaces.

2. The method of providing a graphical user interface on a computer system as claimed in claim 1 further wherein said steps of rendering a background image further comprises the steps of:

rendering a first subset of said background image on said graphics display screen, said first subset from a first location in said background image;

receiving directional input from said user, said directional input comprising a movement direction; and

rendering a second subset of said background image on said graphics display screen, said second subset from a second location in said background image, said second location being said first location plus an offset corresponding to said movement direction;

such that said user can move about within a background image associated with a space.

3. The method of providing a graphical user interface on a computer system as claimed in claim 2 further wherein said step of receiving directional input from said user further comprises detecting a finger of said user sliding across said display screen.

4. The method of providing a graphical user interface on a computer system as claimed in claim 1 further comprising the steps of:

providing a wayback portal, said wayback portal displayed in each of said spaces, said wayback portal comprising a stack of data structures, each of said data structures comprising a pointer, each of said pointers pointing to one of said spaces;

pushing a first data structure on said wayback portal stack when said user moves to said first space, said first data structure containing a pointer to said first space;

popping a second data structure off of said wayback portal stack when said user selects said wayback portal, said second data structure containing a pointer to a second space; and

moving said user to said second space pointed to by said pointer in said second data structure popped off said wayback portal stack;

such that said user can trace back along a path of spaces which said user travels through.

5. The method of providing a graphical user interface on a computer system as claimed in claim 4 wherein said data structures in said wayback portal further comprises a coordinate, said coordinate identifying a location in said first space where said user was when said user left said first space.

6. The method of providing a graphical user interface on a computer system as claimed in claim 4 wherein said wayback portal is rendered as a snapshot of a space pointed to by a pointer in a data structure on top of said wayback portal stack.

7. The method of providing a graphical user interface on a computer system as claimed in claim 1 further comprising the steps of:

providing a plurality of button objects, each of said button objects associated with a function;

rendering said button objects within said spaces, said button objects comprising graphic icons; and

executing the function associated with a button object when said button object is selected by a user.

8. The method of providing a graphical user interface on a computer system as claimed in claim 1 wherein said display screen comprises a touch sensitive display screen.

9. The method of providing a graphical user interface on a computer system as claimed in claim 8 wherein said input from said user comprises a finger touching said touch sensitive display screen.

10. The method of providing a graphical user interface on a computer system as claimed in claim 1, said method further comprising the steps of:

rendering an anthropomorphic graphic character on said display;

attracting said user'"'"'s attention by animating said anthropomorphic graphic character; and

conveying state information to said user by changing a visual appearance of said anthropomorphic character.

11. A method of displaying a plurality of objects for a user to select from on a display screen associated with a computer system, said method comprising the steps of:

organizing a plurality of objects into a circular object list;

rendering a first subset of said objects in said circular object list as a first subset of graphic objects on said display screen, said graphic objects rendered on said display screen aligned along a first axis; and

rotating through said circular object list when said user slides a pointing device across said graphic objects in a direction substantially parallel to said first axis such that said first subset of graphic objects on said display screen is replaced by a second subset graphic objects on said display screen, said second subset graphic objects on said display screen corresponding to a second subset of said objects in said circular object list;

such that said user can choose any of said plurality of objects by rotating through the circular object list until a desired object appears on said display screen.

12. The method of displaying a plurality of graphic objects for a user to select from as claimed in claim 11 further comprising the step of:

selecting a graphic object displayed on said display screen when a user places a pointing device on said graphic object displayed on said display screen for a minimum amount of time.

13. The method of displaying a plurality of graphic objects for a user to select from as claimed in claim 11 further comprising the step of:

moving a graphic object off said circular object list when said user places said pointing device on said graphic object and moves said pointing device in a direction substantially perpendicular to said first axis.

14. The method of displaying a plurality of graphic objects for a user to select from as claimed in claim 13 wherein said display screen comprises a touch screen and said pointing device comprises a finger.

15. The method of displaying a plurality of graphic objects for a user to select from as claimed in claim 11 wherein said step of rotating through said circular object list is enhanced by simulating a physical wheel that is affected by friction.

17. The method of storing a path traveled along said directed graph structure in a graphical user interface for a computer system as claimed in claim 11 wherein said wayback portal comprises a miniature image of the space indicated by the pointer on the top of the wayback portal stack.

16. In a graphical user interface for a computer system, said graphical user interface comprising a plurality of spaces organized in a directed graph structure, each of said spaces comprising a graphic background image, said spaces coupled in said directed graph structure using a plurality of portals, each of said portals providing a connection between a first space and a second space, a method for storing a path traveled along said directed graph structure, said method comprising the steps of:

rendering a graphic background image corresponding to a first space on a display screen, said first space including at least one portal, said portal associated with a target space;

providing a wayback portal, said wayback portal displayed in each of said spaces, said wayback portal comprising a stack of space pointers, each of said space pointers pointing to a previously visited space;

receiving input from a user, said input comprising said wayback portal or a portal selected from said portals located in said first space; and

processing said input received from said user such that if said user selects a portal, said user interface pushing a pointer to a space associated with said selected portal on said wayback stack and rendering a graphic background image of the space associated with said selected portal on said graphics display screen, else if said user selects said wayback portal, said user interface popping a space pointer off the top of said wayback portal stack and rendering said space indicated by said space pointer on said graphics display screen.

18. In a computer system having a graphical user interface, an animated multipurpose icon, said icon comprising the elements of:

an anthropomorphic graphic character rendered on said display;

means for attracting a user'"'"'s attention by animating said anthropomorphic graphic character; and

means for conveying state information by changing a visual appearance of said anthropomorphic character.

19. In a graphical user interface for a computer system, said graphical user interface comprising a plurality of graphic objects rendered on a display screen, said display screen comprising a touch screen which senses a finger placed on the screen and determines a location of said finger on said touch screen, a method of allowing a user to grab a graphic object, said method comprising the steps of:

rendering a graphic object on said touch screen display, said graphic object defining an area on said touch screen display;

placing a finger on said touch screen display near said graphic object rendered on said touch screen display;

circling said graphic object rendered on said touch screen display with said finger;

touching said graphic object rendered on said touch screen display; and

rendering said graphic object on said touch screen display at a location defined by said finger on said display screen as determined by said touch screen display;

such that a user can select said graphic object by circling said graphic object with a finger.

20. An apparatus for controlling at least one remote device, said apparatus comprising the elements of:

a central processing unit (CPU);

a memory, said memory coupled to said central processing unit;

a display screen, said display screen coupled to said CPU, said display screen displaying graphic images;

communications means for receiving and transmitting messages across a communications network, said communications means coupled to said central processing unit;

means for receiving a user interface program object over said communications means, said user interface program object broadcast by a remote device, said user interface program object defining a user interface for said remote device;

means for invoking a user interface method in said user interface program object, said user interface method displaying a graphical user interface associated with said remote device on said display screen;

means for accepting input from a user, said input guided by said graphical user interface associated with said remote device on said display screen; and

means for invoking a device driver method associated with said remote device in response to said input from said user, said device driver method controlling said remote device;

such that said user interface of said remote device is displayed on said display screen and interaction with said user interface invokes a device driver method.

21. The apparatus for controlling at least one remote device as claimed in claim 20, said apparatus further comprising

means for receiving a device driver program object associated with said remote device over said communications means, said device driver program object including said device driver method for controlling said remote device.

22. The apparatus for controlling at least one remote device as claimed in claim 21, said apparatus further wherein said device driver method is broadcast by said remote device across said communications network and copied into said apparatus.

23. The apparatus for controlling at least one remote device as claimed in claim 22 wherein said device driver method translates said input from said user into a plurality of control codes.

24. The apparatus for controlling at least one remote device as claimed in claim 23 wherein said apparatus transmits said plurality of control codes to said remote device.

25. The apparatus for controlling at least one remote device as claimed in claim 20 wherein said device driver method is located within said remote device.

26. The apparatus for controlling at least one remote device as claimed in claim 25 wherein said device driver method is invoked remotely across said communications network by said apparatus.

27. The apparatus for controlling at least one remote device as claimed in claim 26 wherein said device driver method is remotely invoked using a robust remote procedure call manager.

29. The method of obtaining a remote device'"'"'s user interface from said remote device and controlling said remote device with a local display device as claimed in claim 28, wherein said method further comprises the steps of:

interacting with said remote device'"'"'s user interface displayed on said local display device; and

invoking a control method in a device driver program object to handle said interactions with said remote device'"'"'s user interface.

30. The method of obtaining a remote device'"'"'s user interface from said remote device and controlling said remote device with a local display device as claimed in claim 29, wherein said device driver program object is located within said remote device.

31. The method of obtaining a remote device'"'"'s user interface from said remote device and controlling said remote device with a local display device as claimed in claim 30, wherein said control method in said device driver program object is invoked remotely across said communications network.

32. The method of obtaining a remote device'"'"'s user interface from said remote device and controlling said remote device with a local display device as claimed in claim 29, said method further comprising the step of transmitting said device driver program object across said communications network to said local display device.

33. The method of obtaining a remote device'"'"'s user interface from said remote device and controlling said remote device with a local display device as claimed in claim 32, wherein said device driver method within said device driver program object is invoked within said local display device.

34. The method of obtaining a remote device'"'"'s user interface from said remote device and controlling said remote device with a local display device as claimed in claim 33, wherein said device driver method translates said user input into commands codes.

35. The method of obtaining a remote device'"'"'s user interface from said remote device and controlling said remote device with a local display device as claimed in claim 34, wherein said local display device transmits said commands codes to said remote device.

37. The method of copying an object in a graphical user interface as claimed in claim 36, wherein said input comprises a finger touching a touch screen display

39. The method of copying an object in a graphical user interface as claimed in claim 38, wherein said input comprises a finger touching a touch screen display

28. A method of obtaining a graphical user interface from said remote device and controlling said remote device with a local display device, said local display device coupled to a communication network, said method comprising:

connecting said remote device to said communication network;

signaling from said remote device that said remote device has a graphical user interface to export upon being connected to said communication network;

requesting said graphical user interface by said local display device;

transmitting a graphical user interface program object to said local display device, said graphical user interface program object containing a method which display said graphical user interface; and

invoking said method in said user interface program object such that said remote device'"'"'s graphical user interface is displayed on said local display device.

36. A method of copying an object in a graphical user interface, said method comprising the steps of:

rendering a first graphic image of said object at a first position on a graphics display screen;

receiving input from a user, said input comprising a command to copy said object;

rendering a ghost image of said object at said first position on said display screen, said ghost image comprising a copy of said first graphic image of said object drawn in muted colors;

receiving input from a user, said input comprising a second position to locate a copy of said object;

rendering said first graphic image of said object at said second position; and

changing the ghost image of said object at said first position on said display screen such that said ghost image of said object becomes identical to said first graphic image of said object;

such that two graphic images of said object now exist.

38. A method of moving an object in a graphical user interface, said method comprising the steps of:

rendering a first graphic image of said object at a first position on a graphics display screen;

receiving input from a user, said input comprising a command to move said object;

rendering a ghost image of said object at said first position on said display screen, said ghost image comprising a copy of said first graphic image of said object drawn in muted colors;

receiving input from a user, said input comprising a second position to locate a said object;

rendering said second graphic image of said object at said second position; and

moving said ghost image of said object at said first position to said second position such that said ghost image of said object merges into said second graphic image of said object at said second position;

such that one graphic image of said object now exists at said second position.

40. In an object-oriented graphical user interface, said object-oriented graphic user interface illustrating objects as graphical icons, a method of providing information to a program object, said method comprising the steps of:

rendering a graphic image of a data object at a first position on a graphics display screen;

receiving input from a user, said input comprising a command to move said data object;

rendering a ghost image of said object at said first position on said display screen, said ghost image comprising a copy of said first graphic image of said object drawn in muted colors;

receiving input from a user, said input comprising a second position to locate a said data object, said program object located at said second position; and

receiving input from a user, said input a command to provide said data object to said program object;

such that two graphic images of said object now exist.

41. The method of providing information to a program object as claimed in claim 40, wherein said input comprises a finger touching a touch screen display

42. A method of navigating through a graphical user interface, said graphical user interface displayed on a display screen, said graphical user interface comprising a plurality of spaces, each of said spaces comprising a graphic background image, said spaces organized into a map structure, said method comprising the steps of:

displaying a first space on said display screen, said first space representing where a user currently is within said map structure;

displaying a plurality of portals in said first space, each of said portals having an associated target space such that said portal connects the space said portal is in with said associated target space such that each portal provides said user a place to go, each of said portals comprising a graphic icon; and

displaying a wayback portal in said first space, said wayback depicting a image of a space said user just left.

43. The method of navigating through a graphical user interface as claimed in claim 42, wherein said map structure is organized as a directed graph.

44. The method of navigating through a graphical user interface as claimed in claim 43, wherein if a user selects one of said portals then said user is moved to a space associated with said Portal.