Customization of GUI layout based on history of use

• US 9,448,701 B2
• Filed: 09/28/2012
• Issued: 09/20/2016
• Est. Priority Date: 05/21/2009
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Zalewski, Gary M., Nicholson, Charles
• Primary Examiner(s): Nguyen, Maikhanh
• Application Number:
  US13/631,583
• Publication Number:
  US 20130212504A1
• Time in Patent Office: 1,453 Days
• Field of Search: 715/200, 715/201, 715/204, 715/205, 715/210, 715/226, 715/231, 715/234, 715/243, 715/249, 715/255, 715/273, 715/276, 715/700, 715/731, 715/746, 715760-762, 715/767, 715/771
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04855 Interaction with scrollbars
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A hand-held electronic device, method of operation and computer readable medium are disclosed. The device may include a processor is operably coupled to a visual display and touch screen. Instructions executable by the processor may be configured to a) present an image on the visual display containing one or more active elements; b) correlate one or more active portions of the touch interface to one or more corresponding active elements in the image on the visual display; and c) adjust a layout of content shown on the display according to a probability of one or more actions that may be taken with the one or more active elements.

Claims

1. A hand-held electronic device, comprising:

a processor operably coupled to a visual display and a touch interface; and

instructions executable by the processor configured such that, when executed, the instructions cause the device to;

a) present an image on the visual display containing one or more active elements;

a′

) perform a tessellation of the image so that the image is divided into one or more regions that fill the display, wherein each region corresponds to a different active element;

b) correlate one or more active portions of the touch interface to one or more corresponding active elements in the image on the visual display; and

c) adjust a layout of content shown on the display according to a probability of one or more potential actions with the one or more active elements,

wherein the instructions are configured to cause the device to adjust the layout by locating active elements that are likely to be used in sequence closer together than in a default layout.

2. The device of claim 1, wherein the instructions are configured to cause the device to decay the layout to a non-predictive layout over time, when executed.

3. The device of claim 1, further comprising a prediction engine configured to compute a probability for a subsequent action based on a past pattern of user behavior following an action of a given type with the device, when executed.

4. The device of claim 3, wherein the past behavior is correlated to a type of content displayed on the display.

5. The device of claim 3, wherein the prediction engine is configured to update the probability for the subsequent action as the user uses the device, when executed.

6. The device of claim 1, wherein the instructions are configured to cause the device to adjust the layout in a way that facilitates one or more subsequent actions that are most probable when executed.

7. The device of claim 6, wherein the instructions are configured to cause the device to adjust an appearance of the active elements when executed.

8. The device of claim 7, wherein active elements likely to be used are highlighted or more brightly colored than other elements.

9. The device of claim 6, wherein the instructions are configured to cause the device to adjust an operation of one or more of the active elements when executed.

10. The device of claim 9, wherein the visual display is a touch screen that is separate from the touch pad.

11. The device of claim 10, wherein the visual display is located on first surface of a case and the touch pad is located on a second surface of the case that is opposite the first surface.

12. The device of claim 10, wherein the visual display and touch pad are disposed on the same side of the case.

13. The device of claim 10, wherein the visual display and touch pad are disposed on different sides of the case.

14. The device of claim 13, wherein the visual display is disposed on a front side of the case and the touch pad is disposed on a back side of the case.

15. The device of claim 10, wherein the case includes first and second case portions wherein the visual display is disposed on the first case portion and wherein the touch pad is disposed on the second case portion.

16. The device of claim 15, wherein the first and second case portions are slidably connected to each other.

17. The device of claim 15, wherein the first and second case portions are connected to each other in a hinged configuration.

18. The device of claim 17, wherein the visual display and touch pad face inward when the first and second portions are in a closed position.

19. The device of claim 18, wherein one of the visual display and touch pad faces inward and the other of the touch pad and visual display faces outward when the first and second portions are in a closed position.

20. The device of claim 18, wherein the visual display and touch pad face inward when the first and second portions are in a closed position.

21. The device of claim 1, wherein the tessellation divides the image into one or more convex regions.

22. The device of claim 21, wherein the tessellation is a Voronoi decomposition.

23. The device of claim 1, wherein a size of each region depends on a corresponding probability that a user uses the corresponding active element within a given time frame.

24. The device of claim 23, wherein b) includes making a first region with a first corresponding probability larger than a second region with a second corresponding probability when the first corresponding probability is greater than the second corresponding probability.

25. The device of claim 23, wherein the probability that the user uses the corresponding active element depends on a history of use of the device.

26. The device of claim 23, wherein the probability that the user uses the corresponding active element depends on a time elapsed since a previous action.

27. The device of claim 26, wherein the instructions are configured to cause the device to iteratively repeat a), a′

), b) and c) when executed.

28. The device of claim 1, wherein the visual display is a touch screen that includes the touch interface.

29. The device of claim 1, wherein the touch interface is a touch pad.

30. The device of claim 1, wherein the visual display includes a touch screen and the touch interface includes the touch screen.

31. The device of claim 1, wherein the instructions further comprise instructions that cause the device to activate one or more of the elements in response to a distinct mode of touch that is distinct from a normal mode of touch that does not activate the one or more active elements when executed.

32. The device of claim 31, wherein the distinct mode of touch is a two finger touch and wherein the normal mode of touch is a single finger touch.

33. The device of claim 1, wherein the instructions are further configured to cause the device to activate one of the one or more active elements in response to a touch to a corresponding one of the active portions of the touch interface when executed.

34. The device of claim 1, wherein the instructions are further configured to cause the device to present a transformed element on the display in response to a user interaction with the touch interface, wherein the transformed element interacts with the touch screen in a different mode of operation than a mode of operation of the corresponding active element prior to its transformation when executed.

35. The device of claim 34, wherein the transformed element appears magnified on the visual display compared to the active element prior to transformation into the transformed active element.

36. The device of claim 35, wherein the instructions are configured to cause the device to control a degree of magnification of the transformed element according to a mode of touch on the touch interface when executed.

37. The device of claim 34, wherein the instructions further comprise an instruction configured to cause the device to revert the transformed element to a form the active element had prior to being transformed into the transformed element in response to a signal from the touch interface or after an interaction with the transformed element is completed when executed.

38. The device of claim 37, wherein the instructions are configured to cause the device to revert the transformed active element in response to a removal of a touch on the touch interface when executed.

39. The device of claim 37, wherein the instructions further comprise an instruction configured to cause the device to highlight an active element that was most recently transformed when executed.

40. The device of claim 39, wherein the visual display is a touch screen and wherein the instructions are configured to cause the device to interpret a touch anywhere on the touch screen as an activation of the most recently transformed active element when executed.

41. The device of claim 34, wherein the visual display is a touch screen and wherein the instructions further include an instruction configured to cause the device to re-purpose one or more portions of the touch screen outside the transformed element to act as inputs for commands associated with the transformed element when executed.

42. A method for operating a hand-held electronic device having a processor operably coupled to a visual display and a touch interface;

and instructions executable by the processor to implement the method, the method comprising;

a) presenting an image on the visual display containing one or more active elements;

a′

) performing a tessellation of the image so that the image is divided into one or more regions that fill the display, wherein each region corresponds to a different active element;

b) correlating one or more active portions of the touch interface to one or more corresponding active elements in the image on the visual display; and

c) adjusting a layout of content shown on the display according to a probability of one or more potential actions with the one or more active elements,

wherein adjusting the layout includes locating active elements that are likely to be used in sequence closer together than in a default layout.

43. The method of claim 42, wherein the probability of the one or more actions depends on a time elapsed since a previous action.

44. The method of claim 42, wherein the probability of the one or more actions depends on a history of use of the device.

45. The method of claim 42, wherein the probability of the one or more actions includes a probability for a subsequent action based on a past pattern of user behavior following an action of a given type with the device.

46. The method of claim 45, wherein the past behavior is correlated to a type of content displayed on the display.

47. The method of claim 42, further comprising updating the probability for the subsequent action as the user uses the device.

48. The method of claim 42, wherein c) includes adjusting the layout in a way that facilitates one or more subsequent actions that are most probable.

49. The method of claim 48, wherein adjusting the layout includes adjusting an appearance of the active elements.

50. The method of claim 49, wherein one or more active elements that are likely to be used are highlighted or more brightly colored than other elements.

51. The device of claim 48, wherein adjusting the layout includes adjusting an operation of one or more of the active elements.

52. The method of claim 42, wherein the tessellation divides the image into one or more convex regions.

53. The method of claim 52, wherein the tessellation is a Voronoi decomposition.

54. The method of claim 53, wherein a size of each region depends on a corresponding probability that a user uses the corresponding active element within a given time frame.

55. The method of claim 54, wherein the corresponding probabilities for each region decay over time to an unbiased probability whereby the tessellation of the image decays over time to an unbiased tessellation.

56. The method of claim 42, further comprising transforming one or more of the active elements to a transformed element in response to a user interaction with a corresponding one or more of the active portions of the touch interface.

57. The method of claim 56, wherein the transformed element interacts with the touch interface in a different mode of operation than a mode of operation of the corresponding active element prior to its transformation.

58. The method of claim 56, wherein the transformed element appears magnified on the visual display compared to the active element prior to transformation into the transformed active element.

59. The method of claim 58, wherein a mode of touch on the touch interface controls a degree of magnification of the transformed element.

60. The method of claim 56, further comprising reverting the transformed element to a form the active element had prior to being transformed into the transformed element in response to a signal from the touch interface.

61. The method of claim 60, wherein a removal of a touch on the touch interface triggers reverting the transformed element.

62. The method of claim 56, further comprising highlighting an active element that was most recently transformed.

63. The method of claim 62, wherein a touch anywhere on the touch interface activates the most recently transformed active element.

64. The method of claim 42, further comprising, for each active element of the one or more active elements, determining the corresponding probability that a user uses the active element within a given time frame.

65. A non-transitory computer readable medium programmed with computer executable instructions for operating a hand-held electronic device having a processor operably coupled to a visual display and a touch interface, wherein the instructions are executable by the processor to implement a method comprising:

a) presenting an image on the visual display containing one or more active elements;

a′

) performing a tessellation of the image so that the image is divided into one or more regions that fill the display, wherein each region corresponds to a different active element;

b) correlating one or more active portions of the touch interface to one or more corresponding active elements in the image on the visual display; and

c) adjusting a layout of content shown on the display according to a probability of one or more potential actions with the one or more active elements,

wherein adjusting the layout includes locating active elements that are likely to be used in sequence closer together than in a default layout.