System and method for issuing commands to applications based on contextual information

• US 9,223,590 B2
• Filed: 05/28/2010
• Issued: 12/29/2015
• Est. Priority Date: 01/06/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Moore, Bradford Allen, Swales, Stephen W.
• Primary Examiner(s): DYER, ANDREW R
• Application Number:
  US12/789,684
• Publication Number:
  US 20110167340A1
• Time in Patent Office: 2,041 Days
• Field of Search: 715/256, 715/257, 715/258, 715/262, 715/271
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0233 Character input methods
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/45512 Command shells
• CPC Class Codes: G06F 9/54 Interprogram communication

Abstract

A system and method for issuing commands to an application based on contextual information. A control application receives an indication that a text manipulation event has occurred in a user interface of a second application. Next, the control application queries the second application to obtain contextual information established by the second application prior to the event, the contextual information providing context to the text manipulation event that occurred in the user interface of the second application. The control application then issues one or more commands to the second application based on the contextual information providing context to the text manipulation event.

Claims

1. A system, comprising:

one or more processors;

memory; and

a control application stored in the memory, the control application comprising instructions to;

retain contextual information for a second application;

receive a notification from the second application that the retained contextual information is outdated;

receive an indication that a text manipulation event has occurred based on a keyboard input that modifies text in a user interface of the second application, wherein the keyboard input is generated based on input with a keyboard that includes keys with corresponding hit zones;

send a query to the second application to obtain updated contextual information established by the second application prior to the event, wherein the updated contextual information is based on internal state information for the second application that is not directly accessible by the control application, and the updated contextual information provides context to the text manipulation event that occurred at the location in the user interface of the second application;

receive the updated contextual information from the second application; and

adjust a size of the hit zone of one or more keys on the keyboard based on the updated contextual information providing context to the text manipulation event; and

wherein the one or more commands cause the second application to execute the one or more commands issued by the control application.

2. The system of claim 1, wherein the contextual information relating to the text manipulation event includes a logical location and a predetermined unit of text relating to the text manipulation event.

3. The system of claim 2, wherein the predetermined unit of text is selected from the group consisting of:

a character, a word, a sentence, a paragraph, a line of text, a section of a document, and a document.

4. The system of claim 2, wherein the logical location of the text manipulation event is selected from the group consisting of:

a point between two characters in the user interface of the second application; and

a range including one or more characters that is selected in the user interface of the second application.

5. The system of claim 4, wherein a respective query from the control application requesting the contextual information of the text manipulation event includes a physical location of the text manipulation event in the user interface of the second application, and the logical location of the text manipulation event corresponds to the physical location.

6. The system of claim 1, wherein the text manipulation event is selected from the group consisting of:

an insertion of one or more characters;

a deletion one or more characters;

a selection of one or more characters; and

a deselection of one or more characters.

7. The system of claim 1, wherein the control application includes instructions to determine the one or more commands, based on the contextual information providing context to the text manipulation event, prior to issuing the one or more commands to the second application.

8. The system of claim 7, wherein the instructions to determine the one or more commands include instructions to:

determine that the contextual information and text manipulation event indicate a sequence of characters that represent a single character;

determine one or more single characters from a plurality of possible single characters based on the contextual information and text manipulation event; and

generate one or more commands for instructing the second application to display, for user selection, the one or more single characters from the plurality of possible single characters.

9. The system of claim 7, wherein the instructions to determine the one or more commands include instructions to:

determine that the contextual information and text manipulation event indicate a sequence of characters that represent a potentially misspelled word;

determine one or more words from a plurality of possible words that represent a correct spelling of the potentially misspelled word; and

generate one or more commands for instructing the second application to display, for user selection, the one or more words from the plurality of possible words.

10. The system of claim 7, wherein the instructions to determine the one or more commands include instructions to:

determine that the contextual information and text manipulation event indicate a sequence of characters that represent a portion of a word;

determine one or more candidate words in accordance with the portion of the word; and

generate one or more commands for instructing the second application to display, for user selection, the one or more candidate words.

11. The system of claim 1, wherein the second application includes instructions to determine the contextual information providing context to the text manipulation event.

12. The system of claim 11, wherein the instructions to determine the contextual information providing context to the text manipulation event include instructions to:

determine a text direction associated with a logical location of the text manipulation event; and

determine boundaries of a predetermined text unit that includes the text associated with the logical location of the text manipulation event based on the text direction.

13. The system of claim 1, wherein the control application includes instructions to receive the text manipulation event in the user interface of the second application.

14. The system of claim 1, wherein the second application includes instructions to notify the control application that contextual information obtained by the control application from the second application can no longer be relied upon by the control application.

15. The system of claim 1, wherein the second application includes instructions to notify the control application that a selection of text in the second application has changed.

16. The system of claim 1, wherein the control application provides enhanced keyboard services to the second application based on the internal state information.

17. The system of claim 16, wherein the enhanced keyboard services comprise one or more services selected from the group consisting of:

spelling correction, autocompletion, grammar correction, and adjusting a hit zone of one or more soft keyboard keys.

18. The system of claim 1, wherein the contextual information that is not directly accessible to the control application comprises text position information for text adjacent to the location.

19. The system of claim 1, wherein the internal state information is not directly accessible to the control application if one or more memory locations of the application internal state information are not known to the control application.

20. The system of claim 1, wherein the internal state information is not directly accessible to the control application if the second application stores the internal state information in a manner unknown to the control application.

21. The system of claim 1, wherein the control application further comprises instructions to, prior to querying the second application to obtain contextual information:

determine whether the control application has sufficient contextual information for providing an enhanced keyboard function to the second application; and

wherein the querying is in accordance with a determination that the control application does not have sufficient contextual information for providing the enhanced keyboard function to the second application.

22. The system of claim 1, wherein the control application further comprises instructions to:

determine the type of text manipulation event that has occurred at the location; and

wherein the one or more commands issued to the second application are based on the contextual information and the type of text manipulation event.

23. The system of claim 1, wherein the control application further comprises instructions to store a usage history for a user for providing the enhanced keyboard services.

24. The system of claim 1, wherein the control application provides an application programming interface (API) to enable enhanced keyboard services on the second application.

25. The system of claim 1, wherein the second application is a third party application executing on the device and the control application is a first party application executing on the device.

26. The system of claim 1, wherein the control application further comprises instructions to retain contextual information for a plurality of additional secondary applications.

27. A computer-implemented method, comprising:

at a computer system including one or more processors and memory storing one or more programs including a control application and a second application, the one or more processors executing the one or more programs to perform the operations of;

retaining contextual information for a second application;

receiving a notification from the second application that the retained contextual information is outdated;

receiving, by the control application, an indication that a text manipulation event has occurred based on a keyboard input that modifies text in a user interface of the second application, wherein the keyboard input is generated based on input with a keyboard that includes keys with corresponding hit zones;

sending, by the control application, a query to the second application to obtain updated contextual information established by the second application prior to the event, wherein the updated contextual information is based on internal state information for the second application that is not directly accessible by the control application, and the updated contextual information provides context to the text manipulation event that occurred at the location in the user interface of the second application;

receiving the updated contextual information from the second application; and

adjusting a size of the hit zone of one or more keys on the keyboard based on the updated contextual information providing context to the text manipulation event.

28. The method of claim 27, wherein in response to the querying by the control application, the method further comprises:

receiving a response from the second application providing the contextual information providing context to the text manipulation event that occurred in the user interface of the second application.

29. The method of claim 27, wherein the contextual information providing context to the text manipulation event comprises information about a text direction associated with a logical location of the text manipulation event and one or more boundaries of a predetermined text unit that includes the text associated with the logical location of the text manipulation event based on the text direction.

30. The method of claim 27, wherein the contextual information relating to the text manipulation event includes a logical location and a predetermined unit of text relating to the text manipulation event.

31. The method of claim 30, wherein the predetermined unit of text is selected from the group consisting of:

a character, a word, a sentence, a paragraph, a line of text, a section of a document, and a document.

32. The method of claim 30, wherein the logical location of the text manipulation event is selected from the group consisting of:

a point between two characters in the user interface of the second application; and

a range including one or more characters that is selected in the user interface of the second application.

33. The method of claim 32, wherein a respective query from the control application requesting the contextual information of the text manipulation event includes a physical location of the text manipulation event in the user interface of the second application, and the logical location of the text manipulation event corresponding to the physical location.

34. The method of claim 27, wherein the text manipulation event is selected from the group consisting of:

an insertion of one or more characters;

a deletion one or more characters;

a selection of one or more characters; and

a deselection of one or more characters.

35. The method of claim 27, including determining, by the control application, the one or more commands, based on the contextual information providing context to the text manipulation event, prior to issuing the one or more commands to the second application.

36. The method of claim 35, further comprising:

determining that the contextual information and text manipulation event indicate a sequence of characters that represent a single character;

determining one or more single characters from a plurality of possible single characters based on the contextual information and text manipulation event; and

generating one or more commands for instructing the second application to display, for user selection, the one or more single characters from the plurality of possible single characters.

37. The method of claim 35, further comprising:

determining that the contextual information and text manipulation event indicate a sequence of characters that represent a potentially misspelled word;

determining one or more words from a plurality of possible words that represent a correct spelling of the potentially misspelled word; and

generating one or more commands for instructing the second application to display, for user selection, the one or more words from the plurality of possible words.

38. The method of claim 35, further comprising:

determining that the contextual information and text manipulation event indicate a sequence of characters that represent a portion of a word;

determining one or more words from a plurality of candidate words, each comprising a complete words that incorporates the portion of the word; and

generating one or more commands for instructing the second application to display, for user selection, the one or more candidate words.

39. The method of claim 27, including receiving, by the control application, the text manipulation event in the user interface of the second application.

40. The method of claim 27, further comprising receiving, from the second application, notification at the control application that contextual information obtained by the control application from the second application can no longer be relied upon by the control application.

41. The method of claim 27, further comprising receiving, from the second application, notification at the control application that a selection of text in the second application has changed.

42. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising:

a control application stored in the memory, the control application comprising instructions to;

retain contextual information for a second application;

receive a notification from the second application that the retained contextual information is outdated;

receive an indication that a text manipulation event has occurred based on a keyboard input that modifies text in a user interface of the second application, wherein the keyboard input is generated based on input with a keyboard that includes keys with corresponding hit zones;

send a query to the second application to obtain updated contextual information established by the second application prior to the event, wherein the updated contextual information is based on internal state information for the second application that is not directly accessible by the control application, and the updated contextual information provides context to the text manipulation event that occurred at the location in the user interface of the second application;

receive the updated contextual information from the second application; and

adjust a size of the hit zone of one or more keys on the keyboard based on the updated contextual information providing context to the text manipulation event; and

wherein the one or more commands cause the second application to execute the one or more commands issued by the control application.

44. The non-transitory computer-readable storage medium of claim 42, wherein the contextual information relating to the text manipulation event includes a logical location and a predetermined unit of text relating to the text manipulation event.

45. The non-transitory computer-readable storage medium of claim 44, wherein the predetermined unit of text is selected from the group consisting of:

a character, a word, a sentence, a paragraph, a line of text, a section of a document, and a document.

46. The non-transitory computer-readable storage medium of claim 44, wherein the logical location of the text manipulation event is selected from the group consisting of:

a point between two characters in the user interface of the second application; and

a range including one or more characters that is selected in the user interface of the second application.

47. The non-transitory computer-readable storage medium of claim 42, wherein the text manipulation event is selected from the group consisting of:

an insertion of one or more characters;

a deletion one or more characters;

a selection of one or more characters; and

a deselection of one or more characters.

48. The non-transitory computer-readable storage medium of claim 42, wherein the control application includes instructions to determine the one or more commands, based on the contextual information providing context to the text manipulation event, prior to issuing the one or more commands to the second application.

49. The non-transitory computer-readable storage medium of claim 42, wherein the second application includes instructions to determine the contextual information providing context to the text manipulation event.

50. The non-transitory computer-readable storage medium of claim 42, wherein the control application includes instructions to receive the text manipulation event in the user interface of the second application.

51. The non-transitory computer-readable storage medium of claim 42, wherein the second application includes instructions to notify the control application that contextual information obtained by the control application from the second application can no longer be relied upon by the control application.

52. The non-transitory computer-readable storage medium of claim 42, wherein the second application includes instructions to notify the control application that a selection of text in the second application has changed.

43. A computer-implemented method, comprising:

at a computer system including one or more processors and memory storing one or more programs including a control application and a second application, the one or more processors executing the one or more programs to perform the operations of;

retaining contextual information for a second application;

receiving a notification from the second application that the retained contextual information is outdated;

receiving, by the control application, an indication that a text manipulation event has occurred based on a keyboard input that modifies text in a user interface of the second application, wherein the keyboard input is generated based on input with a keyboard that includes keys with corresponding hit zones;

sending, by the control application, a query to the second application to obtain updated contextual information established by the second application prior to the event, wherein the updated contextual information is based on internal state information for the second application that is not directly accessible by the control application, and the updated contextual information provides context to the text manipulation event that occurred at the location in the user interface of the second application;

receiving the updated contextual information from the second application;

determining, by the control application, one or more commands based on the updated contextual information providing context to the text manipulation event; and

adjusting a size of the hit zone of one or more keys on the keyboard based on the updated contextual information providing context to the text manipulation event.