Intelligent gesture-based user's instantaneous interaction and task requirements recognition system and method

• US 9,052,819 B2
• Filed: 01/25/2012
• Issued: 06/09/2015
• Est. Priority Date: 01/25/2012
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Kawalkar, Amit Nishikant
• Primary Examiner(s): Bashore, William
• Assistant Examiner(s): CHOWDHURY, RAYEEZ R
• Application Number:
  US13/358,193
• Publication Number:
  US 20130191790A1
• Time in Patent Office: 1,231 Days
• Field of Search: 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Methods and apparatus for determining an intended gesture-based input command from an incomplete gesture-based input command that is supplied to a gesture-based touch screen display that includes at least a touch sensitive region includes receiving an incomplete gesture-based input command on the touch sensitive region of the gesture-based touch screen device, the incomplete gesture-based input command including a gesture profile and a gesture direction. Gesture signals that include data representative of the gesture profile and the gesture direction are generated in response to the input command. The gesture signals are processed in a processor to predict the intended gesture-based input command. The intended gesture-based command is retrieved, with the processor, from an electronically stored standard gesture library.

Claims

1. A method for determining an intended gesture-based input command from an incomplete gesture-based input command that is supplied to a gesture-based touch screen display that includes at least a touch sensitive region, the method comprising the steps of:

receiving an incomplete gesture-based input command on the touch sensitive region of the gesture-based touch screen device, the incomplete gesture-based input command including a gesture profile and a gesture direction;

generating gesture signals in response to the input command, the gesture signals including data representative of the gesture profile and the gesture direction;

processing the gesture signals, in a processor, to predict the intended gesture-based input command;

retrieving, with the processor, the intended gesture-based command from an electronically stored standard gesture library;

implementing, with the processor, the intended gesture-based command;

retrieving, with the processor and based on the intended gesture-based command, one or more task requirements from an electronically stored task requirements library, the one or more task requirements comprising one or more tasks that are related to the intended gesture-based command; and

continuously rendering, on the touch screen display, one or more graphical user interface elements representative of the one or more tasks that are related to the intended gesture-based command until a user does not interact with the one or more graphical user interface elements within a predetermined time period.

2. The method of claim 1, further comprising:

selectively retrieving, with the processor, image data representative of the one or more graphical user interface elements from an electronically stored graphical user interface definition library.

3. The method of claim 1, wherein the step of processing the gesture signals, in a processor, to predict the intended gesture-based input command comprises:

extrapolating the gesture profile and the gesture direction to generate an extrapolated input command; and

comparing the extrapolated input command to standard gesture-based commands electronically stored in the standard gesture library.

4. The method of claim 3, further comprising retrieving the intended gesture-based command based on the comparison.

5. The method of claim 1, wherein the step of processing the gesture signals, in a processor, to predict the intended gesture-based input command comprises:

comparing the gesture profile and gesture direction to standard gesture-based commands electronically stored in the standard gesture library.

6. The method of claim 5, further comprising retrieving the intended gesture-based command based on the comparison.

7. A gesture-based touch screen display system, comprising:

a touch screen display including at least a touch sensitive region, the touch sensitive region adapted to receive incomplete gesture-based input commands from a user, the touch sensitive region configured, upon receipt of the incomplete gesture-based input commands, to generate gesture signals that include data representative of a gesture profile and a gesture direction; and

a processor coupled to receive the gesture signals and configured, upon receipt thereof, to;

(i) predict an intended gesture-based input command,

(ii) retrieve the intended gesture-based command from an electronically stored standard gesture library,

(iii) implement the intended gesture-based command,

(iv) retrieve, based on the intended gesture-based command, one or more task requirements from an electronically stored task requirements library, the one or more task requirements comprising one or more tasks that are related to the intended gesture-based command, and

(iv) supply image rendering display commands to the touch screen display that cause the touch screen display to continuously render one or more graphical user interface elements representative of at least a portion of the one or more task requirements until a user does not interact with the one or more graphical user interface elements within a predetermined time period.

8. The system of claim 7, wherein the processor is further configured to selectively retrieve image data representative of the one or more graphical user interface elements from an electronically stored graphical user interface definition library.

9. The system of claim 7, wherein the processor is configured to predict the intended gesture-based input command by:

extrapolating the gesture profile and the gesture direction to generate an extrapolated input command; and

comparing the extrapolated input command to standard gesture-based commands electronically stored in the standard gesture library.

10. The system of claim 9, wherein the processor is further configured to retrieve the intended gesture-based command based on the comparison.

11. The system of claim 7, wherein the processor is configured to predict the intended gesture-based input command by:

comparing the gesture profile and gesture direction to standard gesture-based commands electronically stored in the standard gesture library.

12. The system of claim 11, wherein the processor is further configured to retrieve the intended gesture-based command based on the comparison.