Dynamic function row item discovery and context

• US 10,261,667 B2
• Filed: 11/02/2016
• Issued: 04/16/2019
• Est. Priority Date: 09/23/2016
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Ozer, Ali T., Dreessen, Christopher A.
• Primary Examiner(s): Duong, Hien L
• Application Number:
  US15/341,827
• Publication Number:
  US 20180088748A1
• Time in Patent Office: 895 Days
• CPC Class Codes: G06F 1/1671 Special purpose buttons or auxiliary keyboards, e.g. retractable mini keypads, keypads or buttons that remain accessible at closed laptop G06F1/1666 takes precedence
• CPC Class Codes: G06F 3/0219 Special purpose keyboards
• CPC Class Codes: G06F 3/0238 Programmable keyboards key guide holders G06F3/0224
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

A device may include a dynamic function row (DFR) comprising a touchscreen. A device processor may operate a DFR agent controlling the DFR. The DFR agent may generate and/or select images to display on the DFR based on rules and/or contexts, and the DFR may display the images.

Claims

1. A method comprising:

instantiating, by at least one processor of a device, a DFR agent, wherein the device comprises;

a main display configured to display one or more application-generated user interface (UI) elements; and

a keyboard comprising a dynamic function row (DFR) that includes a touchscreen configured to display one or more DFR-specific elements;

receiving, at the DFR agent, a plurality of registrations each specifying at least one registered element for display on the DFR from at least one of a plurality of applications running on the device, the plurality of applications being ordered in a responder chain;

selecting, by the DFR agent, one or more of the registered elements based on a priority of applications defined by the responder chain order;

generating, by the DFR agent, an image to display on the DFR comprising the selected elements; and

displaying, by the DFR, the image.

2. The method of claim 1, wherein a foreground application that is currently active on the main display of the device has a highest priority.

3. The method of claim 2, wherein at least one of the selected one or more of the registered elements are registered by an application other than the foreground application.

4. The method of claim 1, wherein an application that is currently processing an event has a highest priority.

5. The method of claim 4, wherein at least one of the selected one or more of the registered elements are registered by an application other than the application that is currently processing the event.

6. The method of claim 1, wherein selecting the one or more of the registered elements comprises:

determining that a first element registered by a first application in the responder chain cannot be displayed; and

selecting a second element registered by a second application having a lower priority than the first application in response to the determining that the first element registered by the first application in the responder chain cannot be displayed.

7. The method of claim 1, further comprising ordering, by the DFR agent, the plurality of applications in the responder chain based on a current state of the device.

8. The method of claim 7, further comprising:

detecting, by the DFR agent, a change in the current state of the device; and

reordering, by the DFR agent, the plurality of applications in the responder chain in response to the detecting the change in the current state of the device.

9. The method of claim 1, wherein the selecting is further based on a state of the device.

10. The method of claim 9, wherein at least one of the registered elements is registered as being dependent on the state of the device.

11. The method of claim 9, wherein at least one of the selected elements is registered by an application having a lower priority than an application with no registered elements selected due to the state of the device.

12. The method of claim 1, wherein the DFR agent, rather than the at least one application, determines sizing and placement of the selected one or more of the registered elements in generating the image.

13. The method of claim 1, wherein the image is further based on a user preference that dictates which of the registered elements are displayed to the DFR in various situations, and wherein the DFR is configured to only display DFR-specific elements.

14. The method of claim 1, further comprising;

defining, by the DFR agent, at least one region of the image, the at least one region corresponding to an application command;

receiving, by the DFR, a touch input at an area of the touchscreen corresponding to the at least one region;

detecting, by the DFR agent, the application command based on the touch input; and

sending, by the DFR agent, the application command to at least one application to which the application command is directed.

15. A device comprising:

a main display configured to display one or more application-generated user interface (UI) elements;

a keyboard comprising a dynamic function row (DFR) that includes a touchscreen configured to display one or more DFR-specific elements; and

at least one processor configured to;

instantiate a DFR agent;

receive, at the DFR agent, a plurality of registrations each specifying at least one registered element for display on the DFR from at least one of a plurality of applications running on the device, the plurality of applications being ordered in a responder chain;

select, by the DFR agent, one or more of the registered elements based on a priority of applications defined by the responder chain order;

generate, by the DFR agent, an image to display on the DFR comprising the selected elements; and

display the image to the DFR.

16. The device of claim 15, wherein a foreground application that is currently active on the main display of the device has a highest priority.

17. The device of claim 16, wherein at least one of the selected one or more of the registered elements are registered by an application other than the foreground application.

18. The device of claim 15, wherein an application that is currently processing an event has a highest priority.

19. The device of claim 18, wherein at least one of the selected one or more of the registered elements are registered by an application other than the application that is currently processing the event.

20. The device of claim 15, wherein the at least one processor is further configured to select, by the DFR agent, the one or more of the registered elements by:

determining that a first element registered by a first application in the responder chain cannot be displayed; and

selecting a second element registered by a second application having a lower priority than the first application in response to the determining that the first element registered by the first application in the responder chain cannot be displayed.

21. The device of claim 15, wherein the at least one processor is further configured to order, by the DFR agent, the plurality of applications in the responder chain based on a current state of the device.

22. The device of claim 21, wherein the at least one processor is further configured to:

detect, by the DFR agent, a change in the current state of the device; and

reorder, by the DFR agent, the plurality of applications in the responder chain in response to the detecting the change in the current state of the device.

23. The device of claim 15, wherein at least one application has a plurality of sets of context-specific elements registered for display on the DFR, and wherein the at least one processor is further configured to:

determine, by the DFR agent, a context of the device; and

determine, by the DFR agent, that a first set of context-specific elements registered by the at least one application are relevant to the context, and

wherein the first set of context-specific elements are selected by the DFR agent for display on the DFR.

24. The device of claim 15, wherein the selecting is further based on a state of the device, and wherein at least one of the registered elements is registered as being dependent on the state of the device.

25. The device of claim 15, wherein the selecting is further based on a state of the device, and wherein at least one of the selected elements is registered by an application having a lower priority than an application with no registered elements selected due to the state of the device.

26. The device of claim 15, wherein the DFR agent, rather than the at least one application, determines sizing and placement of the selected one or more of the registered elements in generating the image.

27. The device of claim 15, wherein the image is further based on a user preference that dictates which of the registered elements are displayed to the DFR in various situations.

28. The device of claim 15, wherein the at least one processor is further configured to:

define, by the DFR agent, at least one region of the image, the at least one region corresponding to an application command;

detect, by the DFR agent, the application command based on a touch input at an area of the touchscreen corresponding to the at least one region; and

send, by the DFR agent, the application command to at least one application to which the application command is directed.