System having user interface using motion based object selection and mouse movement

• US 8,497,842 B2
• Filed: 02/28/2008
• Issued: 07/30/2013
• Est. Priority Date: 11/02/2007
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: QNX Software Systems Limited (Canada) (Blackberry Limited)
• Inventors: Tomkins, Steve, Dodge, Dan, Van Der Veen, Peter, Tang, Xiaodan, Burgess, Colin
• Primary Examiner(s): Shalwala, Bipin
• Assistant Examiner(s): CHOWDHURY, AFROZA Y
• Application Number:
  US12/039,676
• Publication Number:
  US 20090115736A1
• Time in Patent Office: 1,979 Days
• Field of Search: 345/156, 345/173
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A system includes a touchscreen display and an interface application. The interface application controls the touchscreen display to provide a container and a user interface control. The user interface control may be selected through manipulation of the touchscreen display in an area of the container proximate the user interface control. The user interface application identifies a user manipulation as a mouse movement in response to manipulation of the touchscreen display in an area of the container when the manipulation has a motion magnitude exceeding a threshold value.

Claims

1. A system comprising:

a processor;

a touchscreen display interacting with the processor;

an interface application executable by the processor to control the touchscreen display, the interface application controlling the touchscreen display to provide a container and a user interface control, the interface application adapted to compare a magnitude of motion of a gesture across a portion of the touchscreen display in the area of the container with a threshold value, the interface application adapted to determine whether the gesture is in an area of the container proximate the user interface control when the comparison indicates that the magnitude of motion of the gesture does not exceed the threshold value, the interface application including a mouse gesture handler that interprets the gesture as a mouse movement when the comparison indicates that the magnitude of motion of the gesture exceeds the threshold value; and

a mouse gesture component adapted to communicate with the mouse gesture handler of the interface application and execute mouse gestures identified by the mouse gesture handler.

2. The system of claim 1, where the interface application comprises a movie clip based application.

3. The system of claim 2, where the interface application comprises:

a movie clip based application server;

an object selection handler in communication with the movie clip based application server; and

wherein the mouse gesture handler is in communication with the movie clip application server.

4. The system of claim 3, further comprising an operating system executable by the processor, where the operating system comprises:

an operating system application server in communication with movie clip based application server;

an object selection component in communication with the operating system application server and adapted to communicate with the object selection handler of the interface application through communications between the operating system application server and the movie clip application server to handle object selections identified by the object selection handler; and

wherein the mouse gesture component is in communication with the operating system application server and communicates with the mouse gesture handler of the interface application through communications between the operating system application server and the movie clip application server.

5. The system of claim 4, where the movie clip based application server and the operating system application server communicate with one another using an XML communication format.

6. The system of claim 1, where the container and user interface control are disposed on different layers of a multilayer display arrangement.

7. The system of claim 6, where the user interface control comprises a plurality of user interface controls including keyboard characters.

8. The system of claim 7, and further comprising an operating system executable by the processor, where the operating system is responsive to selection of the keyboard characters as identified by the interface application to execute text entry functions corresponding to the selection of the keyboard characters, and where the operating system is responsive to mouse movement interpretation by the interface application to execute a mouse movement function.

9. The system of claim 6, where the user interface control comprises a plurality of user interface controls disposed on different layers of a multilayer display arrangement, and where the interface application executes a hierarchical search of the layers to determine which of the plurality of user controls is selected through manipulation of the touchscreen display in a given area of the container.

10. A system comprising:

an interface application executable to control a touchscreen display to provide a container and a user interface control, where the interface application detects a gesture across a portion of the touchscreen display and compares a magnitude of motion of the gesture with a threshold value, where the interface application determines whether the gesture is a selection of the user interface control when the comparison indicates that the magnitude of motion of the gesture does not exceed the threshold value, and where the interface application identifies the gesture as a mouse movement gesture when the comparison indicates that the magnitude of motion of the gesture exceeds the threshold value;

a first application responsive to the selection of the user interface control as identified by the interface application to execute a function corresponding to selection of the user interface control; and

a second application responsive to the mouse movement gesture as identified by the interface application to execute a mouse motion corresponding to the identified mouse movement gesture;

where the container and user control are disposed on different layers of a multiple layer display arrangement, the multiple layers displayed simultaneously.

11. The system of claim 10, where the interface application comprises a movie clip based application.

12. The system of claim 11, where the interface application comprises:

a movie clip based application server;

an object selection handler in communication with the movie clip application server; and

a mouse gesture handler in communication with the movie clip application server.

13. The system of claim 12, further comprising an operating system executable by the processor, where the first and second applications are implemented in the operating system and comprise:

an operating system application server in communication with movie clip based application server;

an object selection component in communication with the operating system application server and adapted to communicate with the object selection handler of the interface application through communications between the operating system application server and the movie clip application server to handle object selections identified by the object selection handler; and

a mouse gesture component in communication with the operating system application server and adapted to communicate with the mouse gesture handler of the interface application through communications between the operating system application server and the movie clip application server to handle mouse gestures identified by the mouse gesture handler.

14. The system of claim 13, where the movie clip based application server and the operating system application server communicate with one another using an XML communication format.

15. The system of claim 10, where the first and second applications are included in an operating system.

16. The system of claim 10, where the first and second applications are included in a single user application.

17. The system of claim 10, where the user control comprises a plurality of user controls, and where the plurality of user controls includes keyboard characters.

18. The system of claim 17, and further comprising an operating system executable by the processor, where the operating system is responsive to selection of the keyboard characters to execute text entry functions corresponding to the selection of the keyboard characters, and where the operating system is responsive to mouse movement identification to execute a mouse movement function corresponding to the gesture.

19. The system of claim 10, where the user control comprises a plurality of user controls disposed on different layers of the multiple layer display arrangement, and where the interface application executes a hierarchical search of the multiple layers to determine which of the plurality of user controls is selected through manipulation of the touchscreen display in a given area of the container.

20. A method of providing a user interface comprising:

generating a container and one or more selectable objects on a touchscreen display;

detecting a gesture across a portion of the touchscreen display;

determining whether a magnitude of motion of the gesture exceeds a non-zero movement threshold value;

executing a mouse movement on the touchscreen display corresponding to the gesture if the magnitude of motion of the gesture exceeds the movement threshold value;

determining whether the gesture is in an active area proximate at least one of the one or more selectable objects if the magnitude of motion of the gesture does not exceed the movement threshold value; and

executing a function associated with the selection of the at least one of the one or more selectable objects if the gesture is in the area of the at least one of the one or more selectable objects and the magnitude of motion of the gesture does not exceed the movement threshold value.

21. The method of claim 20, where the generating comprises:

generating the container on a first display layer of a multilayer display arrangement; and

generating a selectable object on a second display layer of the multilayer display arrangement.

22. The method of claim 20, where the generating comprises:

generating the container on a display layer of a multilayer display arrangement; and

generating a plurality of selectable objects on one or more further display layers of the multilayer display arrangement, where the plurality of selectable objects includes a plurality of keyboard characters.

23. The method of claim 22, further comprising:

generating a text area on the touchscreen display; and

displaying selected keyboard characters in the text area of the touchscreen display.

24. The method of claim 20, where the generating comprises generating a container and one or more selectable objects on the touchscreen display using a movie clip based application.

25. A method of providing a user interface comprising:

detecting a gesture across a portion of a touchscreen display;

comparing a magnitude of movement of the gesture to a movement threshold value to determine whether the magnitude of movement of the gesture exceeds the movement threshold value;

when the comparison indicates that the magnitude of movement of the gesture exceeds the threshold value, executing a mouse movement on the touchscreen display corresponding to the gesture;

when the comparison indicates that the magnitude of movement of the gesture does not exceed the threshold value,

identifying a selectable object;

determining whether the gesture is in an area proximate the selectable object; and

executing a function associated with the selectable object if the determination indicates the gesture is in the area proximate the selectable object.

26. A method of providing a user interface comprising:

detecting a movement across a portion of a touchscreen display in which the movement on the touchscreen display is linked to the movement of a pointing device, but not to the location of the pointing device;

comparing a magnitude of the movement of the pointing device to a movement threshold level to determine whether the magnitude of the movement across the portion of the touchscreen display exceeds the movement threshold level;

moving an indicator about the touchscreen display that corresponds to the movement of the pointing device when the comparison of the magnitude of the movement of the pointing device exceeds the movement threshold level;

determining whether the movement across a portion of the touchscreen display is within a proximate area of a selectable touchscreen object automatically when the magnitude of the movement of the pointing device does not exceed the movement threshold level; and

executing a function associated with the selectable touchscreen object when the magnitude of the movement of the pointing device does not exceed the movement threshold level and when the movement across a portion of a touchscreen display is within the proximate area of the selectable touchscreen object;

where the execution of the function associated with the touchscreen object occurs automatically in response to the determination that the movement across the portion of the touchscreen display is within the proximate area of the selectable touchscreen object.