Touch screen display user interface and method for improving touch interface utility on the same employing a rules-based masking system

• US 9,733,707 B2
• Filed: 03/22/2012
• Issued: 08/15/2017
• Est. Priority Date: 03/22/2012
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Roth, Hans, Kawalkar, Amit Nishikant
• Primary Examiner(s): Edwards, Carolyn R
• Assistant Examiner(s): Gyawali, Bipin
• Application Number:
  US13/427,336
• Publication Number:
  US 20130249809A1
• Time in Patent Office: 1,972 Days
• Field of Search: None
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Disclosed are methods and systems for improving touch utility on a touch screen display user interface providing a rule-based mask that enables or limits a function in a way that allows a user or developer to enhance precision or efficiency of input, or to create a unique modality of input. A method of touch-based user interaction with a touch screen device includes providing a virtual mask overlaying one or more of a plurality of virtual objects on the touch screen device and defining a mask area, sensing a touch within the mask area, the touch generating a touch data signal, and applying one or more combination of interaction, graphical rendering and/or visual feedback rules to the touch data signal and graphical objects. The interaction, graphical rendering and visual feedback rules can exist individually or in combination.

Claims

1. A method of touch-based user interaction with a touch screen device implemented as an aircraft cockpit display in an aircraft, comprising:

receiving, from one or more of a terrain database of the aircraft, a navigation database of the aircraft, or sensors implemented on the aircraft, data representing one or more of terrain information, navigation information, and aircraft-sensed information to be displayed on the touch screen device;

in response to the step of receiving, displaying, on the touch screen device as a plurality of virtual objects, the one or more of terrain information, navigation information, and aircraft-sensed information, wherein the displayed one or more of terrain information, navigation information, and aircraft information is displayed as a virtual object that represents the received one or more of terrain information, navigation information, and aircraft information;

by touching the touch screen device, defining a region of the touch screen device over which to provide a virtual mask, wherein the step of defining the region comprises initiating touching the touch screen device at a first location and continuing touching the touch screen device along a path from the first location to a second location that is a distance from the first location, the path being a geometric shape along the touch screen device that includes both the first location and the second location;

based on the step of touching, providing the virtual mask overlaying an entirety of two or more of the plurality of virtual objects on the touch screen device and defining a mask area in accordance with the defined region, wherein the mask area corresponds with an enclosed area of the geometric shape and is bounded by at least the first location and the second location, and wherein the two or more of the plurality of virtual objects remain visible on the touch screen device through the overlying virtual mask, wherein the mask area encompasses only a portion, but not an entirety of the touch screen device, wherein the two or more of the plurality of virtual objects comprise a first object of a first type and a second object of a second type, wherein on the touch screen device outside of the mask area, a third object of the second type is displayed;

removing from the touch screen device at least one of the two or more of the plurality of virtual objects, the removed at least one object having been within the mask area, the removed object being the second object of the second type, wherein at least one object remains within the mask area after removing the at least one of the two or more of the plurality of virtual objects, the remaining object being the first object of the first type, wherein the third object of the second type outside of the mask area is not removed;

sensing a touch within the mask area, the touch generating a touch data signal; and

applying one or more interaction rules to the touch data signal.

2. The method of claim 1, further comprising providing a feedback data signal to the touch screen device.

3. The method of claim 2, further comprising applying one or more interaction rules to generate the feedback data signal, and wherein the feedback data signal causes the touch screen device to generate one or more of a visual, a tactile, and an aural feedback response.

4. The method of claim 1, wherein the interaction rules are not applied when the touch occurs outside of the mask area.

5. The method of claim 1, wherein the one or more rules are user defined or developer defined.

12. The method of claim 1, wherein user gestures upon touching the touch screen device dynamically draw the region to a shape corresponding with the user gestures.

14. The method of claim 1, further comprising, prior to the step of defining the region, configuring a de-cluttering rule or a de-cluttering filter in a dynamic manner using a user interface, wherein the de-cluttering rule or the de-cluttering filter defines the manner in which the removing step is performed.

6. A touch screen device implemented as an aircraft display in the cockpit of an aircraft, comprising:

a display screen;

first circuitry configured to receive, from one or more of a terrain database of the aircraft, a navigation database of the aircraft, or sensors implemented on the aircraft, data representing one or more of terrain information, navigation information, and aircraft-sensed information to be displayed on the touch screen device;

second circuitry configured to display the one or more of terrain information, navigation information, and aircraft-sensed information as at least two virtual objects through the display screen, wherein the displayed one or more of terrain information, navigation information, and aircraft information is displayed as a virtual object that represents the received one or more of terrain information, navigation information, and aircraft information; and

third circuitry configured to sense touches applied to the at least two virtual objects; and

a processor configured to;

by receiving a touch to the display screen, define a region of the display screen over which to provide a virtual mask, wherein defining the region comprises initiating touching the touch screen device at a first location and continuing touching the touch screen device along a path from the first location to a second location that is a distance from the first location, the path being a geometric shape along the touch screen device that includes both the first location and the second location;

provide a virtual mask overlaying an entirety of the at least two virtual objects and defining a mask area, wherein the mask area corresponds with an enclosed area of the geometric shape and is bounded by at least the first location and the second location, and wherein the two or more of the plurality of virtual objects remain visible on the touch screen device through the overlying virtual mask, wherein the mask area encompasses only a portion, but not an entirety of the touch screen device, wherein the two or more of the plurality of virtual objects comprise a first object of a first type and a second object of a second type, wherein on the touch screen device outside of the mask area, a third object of the second type is displayed;

remove from the touch screen device at least one of the at least two virtual objects, the removed at least one object having been within the mask area, wherein at least one object remains within the mask area after removing the at least one of the at least two virtual objects, the remaining object being the first object of the first type, wherein the third object of the second type outside of the mask area is not removed;

sense a touch within the mask area, the touch generating a touch data signal; and

apply one or more interaction rules to the touch data signal.

7. The device of claim 6, wherein the processor is further configured to provide a feedback data signal to the touch screen device.

8. The device of claim 7, wherein the feedback data signal causes the touch screen device to generate one or more of a visual, a tactile, and an aural feedback response.

9. The device of claim 7, wherein the interaction rules are not applied when the touch occurs outside of the mask area.

10. The device of claim 7, wherein the one or more rules are user defined or developer defined.

13. The device of claim 6, wherein user gestures upon touching the touch screen device dynamically draw the region to a shape corresponding with the user gestures.

11. A method of touch-based user interaction with a touch screen device implemented as an aircraft cockpit display in an aircraft, comprising:

receiving, from one or more of a terrain database of the aircraft, a navigation database of the aircraft, or sensors implemented on the aircraft, data representing one or more of terrain information, navigation information, and aircraft-sensed information to be displayed on the touch screen device;

in response to the step of receiving, displaying, on the touch screen device as a plurality of virtual objects, the one or more of terrain information, navigation information, and aircraft-sensed information, wherein the displayed one or more of terrain information, navigation information, and aircraft information is displayed as a virtual object that represents the received one or more of terrain information, navigation information, and aircraft information;

by touching the touch screen device, defining a region of the touch screen device over which to provide a virtual mask, wherein user gestures upon touching the touch screen device dynamically draw the region to a shape corresponding with the user gestures, and wherein the step of defining the region comprises initiating touching the touch screen device at a first location and continuing touching the touch screen device along a path from the first location to a second location that is a distance from the first location, the path being a geometric shape along the touch screen device that includes both the first location and the second location;

based on the step of touching, providing the virtual mask overlaying an entirety of a first one of the plurality of virtual objects on the touch screen device and defining a mask area in accordance with the defined region, wherein the mask area corresponds with an enclosed area of the geometric shape and is bounded by at least the first location and the second location, and wherein the first one of the plurality of virtual objects remains visible on the touch screen device through the overlying virtual mask, and wherein a second one of the plurality of virtual objects on the touch screen device is not within the mask area, and wherein the first one and the second one of the plurality of virtual objects are the same type of virtual object;

sensing a first touch within the mask area corresponding to a touch of the first one of the plurality of virtual objects, the first touch generating a first touch data signal;

applying a first touch interaction rule to the first touch data signal that causes a first type of action to be performed with regard to the first one of the plurality of virtual objects;

sensing a second touch outside of the mask area corresponding to a touch of the second one of the plurality of virtual objection, the second touch generating a second touch data signal that is different than the first touch data signal; and

applying a second touch interaction rule to the second touch data signal that causes a second type of action to be performed with regard to the second one of the plurality of virtual objects, wherein the second type of action is different than the first type of action even though both the first and second ones of the plurality of virtual objects are of the same type of virtual object, wherein after applying both the first and second touch interaction rules, both of the first and second ones of the plurality of virtual objects remain visible on the touch screen device.