GESTURE BASED CONTEXT MENUS

• US 20130086056A1
• Filed: 10/21/2011
• Published: 04/04/2013
• Est. Priority Date: 09/30/2011
• Status: Abandoned Application

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Dyor, Matthew G., Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Huang, Xuedong, Davis, Marc E.
• Application Number:
  US13/278,680
• Publication Number:
  US 20130086056A1
• Time in Patent Office: Days
• US Class Current: 707/730
• CPC Class Codes: G06F 16/337 Profile generation, learning or modification
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

Methods, systems, and techniques for providing context menus based upon gestured input are provided. Example embodiments provide a Gesture Based Context Menu System, which enables a gesture-based user interface to invoke a context menu to present one or more choices of next actions and/or entities based upon the context indicated by the gestured input and a set of criteria. In overview, the GBCMS allows an area of electronically presented content to be dynamically indicated by a gesture and then examines the indicated area in conjunction with a set of criteria to determine and present a context menu of further choices available to the user. The choices may be presented in the form of, for example, a pop-up menu, a pull-down menu, an interest wheel, or a rectangular or non-rectangular menu. In some embodiments the menus dynamically change as the gesture is modified.

Claims

1. A method in a computing system for providing a gesture based context based for presenting content, comprising:

receiving, from an input device capable of providing gesture input, an indication of a user inputted gesture that corresponds to an indicated area of electronic content presented via a presentation device associated with the computing system;

determining, based upon the indicated area and a set of criteria, a plurality of actions and/or entities that may be used with the indicated area to provide auxiliary content;

presenting the determined plurality of actions and/or entities in a context menu; and

upon receiving an indication that one of the presented plurality of actions and/or entities has been selected, using the selected action and/or entity to determine and present the auxiliary content.

2. The method of claim 1 wherein the determining, based upon the indicated area and a set of criteria, a plurality of actions and/or entities that may be used with the indicated area to provide auxiliary content further comprises:

determining a plurality of actions and/or entities based upon a set of rules used to convert one or more nouns that relate to the indicated area into corresponding verbs.

3. The method of claim 2, the determining a plurality of actions and/or entities based upon a set of rules used to convert one or more nouns that relate to the indicated area into corresponding verbs further comprising:

deriving the plurality of actions and/or entities by determining a set of most frequently occurring words in the electronic content and converting the set into corresponding verbs.

4. The method of claim 2, the determining a plurality of actions and/or entities based upon a set of rules used to convert one or more nouns that relate to the indicated area into corresponding verbs further comprising:

deriving the plurality of actions and/or entities by determining a set of most frequently occurring words in proximity to the indicated area and converting the set into corresponding verbs.

5. The method of claim 2, the determining a plurality of actions and/or entities based upon a set of rules used to convert one or more nouns that relate to the indicated area into corresponding verbs further comprising:

deriving the plurality of actions and/or entities by determining a set of common verbs used with one or more entities encompassed by the indicated area.

6. The method of claim 5, the deriving the plurality of actions and/or entities by determining a set of common verbs used with one or more entities encompassed by the indicated area, further comprising:

determining one or more entities located with the indicated area;

searching the electronic content to determine all uses of the one or more entities and for each such entity, a corresponding verb;

determining from the corresponding verbs a set of most frequently occurring verbs; and

using the determined set of most frequently occurring verbs as the set of common verbs.

7. The method of claim 2, the determining a plurality of actions and/or entities based upon a set of rules used to convert one or more nouns that relate to the indicated area into corresponding verbs further comprising:

generating the plurality of actions and/or entities by determining a set of default actions.

8. The method of claim 7 wherein the default actions include actions that specify some form of buying or shopping, sharing, exploring and/or obtaining information.

9. The method of claim 7 wherein the default actions include an action to find a better <

entity>

, where <

entity>

is an entity encompassed by the indicated area.

10. The method of claim 7 wherein the default actions include an action to share a <

entity>

, where <

entity>

is an entity encompassed by or related to the indicated area.

11. The method of claim 7 wherein the default actions include an action to obtain information about a <

entity>

, where <

entity>

is an entity encompassed by or related to the indicated area.

12. The method of claim 7 wherein the default actions include one or more actions that specify comparative actions.

13. The method of claim 12 wherein the comparative actions include an action to obtain an entity sooner.

14. The method of claim 12 wherein the comparative actions include an action to purchase an entity cheaper.

15. The method of claim 12 wherein the comparative actions include an action to find a better deal.

16. The method of claim 1 wherein the determining, based upon the indicated area and a set of criteria, a plurality of actions and/or entities that may be used with the indicated area to provide auxiliary content further comprises:

determining a plurality of actions and/or entities based upon a social network associated with the user.

17. The method of claim 16 wherein determining a plurality of actions and/or entities based upon a social network associated with the user comprises:

predicting a set of actions based upon similar actions taken by other users in the social network associated with the user.

18. The method of claim 1 wherein the set of criteria includes prior history associated with the user and the determining, based upon the indicated area and a set of criteria, a plurality of actions and/or entities that may be used with the indicated area to provide auxiliary content further comprises:

selecting a plurality of actions and/or entities based upon prior history associated with the user.

19. The method of claim 18 wherein the prior history associated with the user includes at least one of prior search history, prior navigation history, prior purchase history, and/or demographic information.

24. The method of claim 1 wherein the set of criteria includes an attribute of the gesture and the determining, based upon the indicated area and a set of criteria, a plurality of actions and/or entities that may be used with the indicated area to provide auxiliary content further comprises:

determining a plurality of actions and/or entities based upon an attribute of the gesture.

25. The method of claim 24 wherein the attribute of the gesture is at least one of a size, a direction, a color of the gesture, and/or a measure of steering of the gesture.

29. The method of claim 1 wherein the set of criteria includes a context of other text, audio, graphics, and/or objects within the presented electronic content and the determining, based upon the indicated area and a set of criteria, a plurality of actions and/or entities that may be used with the indicated area to provide auxiliary content further comprises:

determining a plurality of actions and/or entities based upon the context of other text, audio, graphics, and/or objects within the presented electronic content.

30. The method of claim 1, further comprising:

receiving an indication that the user inputted gesture has been adjusted; and

dynamically modifying the presented plurality of actions and/or entities in the context menu.

31. The method of claim 30, further comprising:

determining and presenting a second auxiliary content based upon the adjusted user inputted gesture.

32. The method of claim 30 wherein the receiving an indication that the user inputted gesture has been adjusted and the dynamically modifying the presented plurality of actions and/or entities in the context menu further comprises:

receiving an indication that the gesture has at least changed in size, changed in direction, changed in emphasis, and/or changed in type of gesture; and

dynamically modifying the presented plurality of actions and/or entities in the context menu based upon the gesture change.

33. The method of claim 30 wherein the modified presented plurality of actions and/or entities are used to determine and present the auxiliary content.

34. The method of claim 1 wherein the context menu is presented as at least one of a drop down menu, a pop-up menu, or an interest wheel.

37. The method of claim 1 wherein the context menu is rectangular shaped.

38. The method of claim 1 wherein the context menu is non-rectangular shaped.

39. The method of claim 1 wherein the auxiliary content is at least one of an advertisement, an opportunity for commercialization, and/or supplemental content.

40. The method of claim 39 wherein the auxiliary content is at least one of a computer-assisted competition, a bidding opportunity, a sale or an offer for sale of a product and/or a service, and/or interactive entertainment.

41. The method of claim 1 wherein the auxiliary content is at least one of a web page, an electronic document, and/or an electronic version of a paper document.

42. The method of claim 1, the using the selected action to determine and present the auxiliary content, further comprising:

determining an auxiliary content based upon the selected action and at least one of the indicated area and/or the set of criteria; and

presenting the determined auxiliary content.

43. The method of claim 1, further comprising:

presenting the determined auxiliary content as an overlay on top of the presented electronic content.

44. The method of claim 43 wherein the determining an auxiliary content based upon the selected action and at least one of the indicated area and/or the set of criteria is made visible using animation techniques and/or by causing a pane to appear as though the pane is caused to slide from one side of the presentation device onto the presented electronic content.

45. The method of claim 1, further comprising:

presenting the determined auxiliary content in an auxiliary window, pane, frame, or other auxiliary display construct of the presented electronic content.

47. The method of claim 1 wherein the user inputted gesture approximates at least one of a circle shape, an oval shape, a closed path, and/or a polygon.

51. The method of claim 1 wherein the user inputted gesture is an audio gesture.

55. The method of claim 1 wherein the indicated area on the presented electronic content includes at least a word or a phrase, a graphical object, an image, and/or an icon.

57. The method of claim 1 wherein the indicated area on the presented electronic content includes an utterance.

58. The method of claim 1 wherein the indicated area comprises either non-contiguous parts or contiguous parts.

59. The method of claim 1 wherein the indicated area is determined using syntactic and/or semantic rules.

60. The method of claim 1 wherein the input device is at least one of a mouse, a touch sensitive display, a wireless device, a human body part, a microphone, a stylus, and/or a pointer.

61. The method of claim 1 wherein the presentation device is at least one of a browser, a mobile device, a hand-held device, embedded as part of the computing system, a remote display associated with the computing system, a speaker, or a Braille printer.

67. The method of claim 1 wherein the electronic content is at least one of code, a web page, an electronic document, an electronic version of a paper document, an image, a video, an audio and/or any combination thereof.

68. The method of claim 1 performed by a client or a server.

20-22. -22. (canceled)

26-28. -28. (canceled)

35-36. -36. (canceled)

46. (canceled)

48-50. -50. (canceled)

52-54. -54. (canceled)

56. (canceled)

62-66. -66. (canceled)

69-210. -210. (canceled)