Method and system for detection and resolution of frustration with a device user interface

• US 10,416,861 B2
• Filed: 04/06/2016
• Issued: 09/17/2019
• Est. Priority Date: 04/06/2016
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Blackberry Limited
• Inventors: Cietwierkowski, Marcin, Almalki, Nazih, Oliver, Robert George, Chen, Zhe, Inwood, Andrew Michael
• Primary Examiner(s): Sun, Li P
• Application Number:
  US15/092,259
• Publication Number:
  US 20170293417A1
• Time in Patent Office: 1,259 Days
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

A method for user interface interaction on a computing device, the method including detecting frustration with a user interface interaction on the computing device; determining a location of input events on a display of the computing device; and magnifying an area proximate to the location of input events. Further, a computing device includes a processor; and a user interface, wherein the computing device, using the processor and user interface, is configured to detect frustration with a user interface interaction on the computing device; determine a location of input events on a user interface of the computing device; and magnify an area proximate to the location of input events.

Claims

1. A method for providing a user interface on a computing device, the method comprising:

detecting input indicative of user frustration with the computing device, wherein detecting the input indicative of user frustration comprises detecting repeated inputs at a displayed portion of the user interface, such repeated inputs being near a displayed user interface element and not resulting in selection of any displayed user interface element of the user interface; and

in response to the detected input indicative of user frustration;

determining a display location associated with input events relating to the detected input, the determined display location corresponding to the displayed user interface element; and

enlarging only the displayed user interface element, wherein a selection of the displayed user interface element triggers performing an associated computing function.

2. The method of claim 1, wherein the detecting input indicative of user frustration comprises detecting audio cues utilizing a microphone of the computing device.

3. The method of claim 1, wherein the detecting input indicative of user frustration comprises detecting visual cues utilizing a camera of the computing device.

4. The method of claim 1, wherein the enlarging is performed by an operating system using a frame buffer of the computing device.

5. The method of claim 4, wherein the enlarging is performed for all applications executing on the computing device.

6. The method of claim 4, wherein the enlarging is performed based on foreground application context.

7. The method of claim 1, wherein the enlarging is performed based on an alert sent from an operating system to an application on the device.

8. The method of claim 7, wherein the application subscribes for the alert.

9. The method of claim 7, wherein the application further invokes frustration reduction techniques comprising at least one of:

accepting voice commands;

connecting to a help desk for the application; and

changing an output user interface for the application.

10. A computing device comprising:

a processor; and

a user interface,

wherein the computing device, using the processor and user interface, is configured to;

detect input indicative of user frustration with the computing device, wherein detecting input indicative of user frustration comprises detecting repeated inputs at a displayed portion of the user interface, such repeated inputs near a displayed user interface element and not resulting in selection of any displayed user interface element of the user interface; and

in response to the detected input indicative of user frustration;

determine a display location associated with input events relating to the detected input, the determined display location corresponding to the displayed user interface element; and

enlarge only the displayed user interface element, wherein a selection of the displayed user interface element triggers performing an associated computing function.

11. The computing device of claim 10, wherein the computing device is configured to detect input indicative of user frustration by detecting audio cues utilizing a microphone of the computing device.

12. The computing device of claim 10, wherein the computing device is configured to detect input indicative of user frustration by detecting visual cues utilizing a camera of the computing device.

13. The computing device of claim 10, wherein the computing device is configured to enlarge through an operating system using a frame buffer of the computing device.

14. The computing device of claim 13, wherein the computing device is configured to enlarge all applications executing on the computing device.

15. The method of claim 13, wherein the computing device is configured to enlarge based on foreground application context.

16. The computing device of claim 10, wherein the computing device is configured to enlarge based on an alert sent from an operating system to an application on the device.

17. The computing device of claim 16, wherein the application subscribes for the alert.

18. The computing device of claim 16, wherein the application further invokes frustration reduction techniques comprising at least one of:

accepting voice commands;

connecting to a help desk for the application; and

changing an output user interface for the application.

19. A non-transitory computer readable medium containing program code, which when executed by a processor of a computing device, cause the computing device to:

detect input indicative of user frustration with the computing device, wherein detecting input indicative of user frustration comprises detecting repeated inputs at a displayed portion of the user interface, such repeated inputs near a displayed user interface element and not resulting in selection of any displayed user interface element of the user interface; and

in response to the detected input indicative of user frustration;

determine a display location associated with input events relating to the detected input, the determined display location corresponding to the displayed user interface element; and

enlarge only the displayed user interface element, wherein a selection of the displayed user interface element triggers performing an associated computing function.