Method of controlling function execution in a mobile terminal by recognizing writing gesture and apparatus for performing the same

• US 10,282,084 B2
• Filed: 08/20/2013
• Issued: 05/07/2019
• Est. Priority Date: 08/20/2012
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kwon, Musik
• Primary Examiner(s): Yi, Rinna
• Application Number:
  US13/971,295
• Publication Number:
  US 20140053114A1
• Time in Patent Office: 2,086 Days
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Methods and apparatus are provided for executing a function of a mobile terminal by recognizing a writing gesture. The writing gesture that is inputted on a touchscreen of the mobile terminal is detected. At least one target item to which the writing gesture applies is determined. A preset writing gesture of the at least one target item is compared with the detected writing gesture to determine whether the preset writing gesture is at least similar to the detected writing gesture. An execution command corresponding to the preset writing gesture is extracted, when it is determined that the detected writing gesture is at least similar to the preset writing gesture. The function of the at least one target item is executed by the execution command.

Claims

1. A method of controlling a mobile terminal, the method comprising:

displaying, on a touchscreen of the mobile terminal, content of a first application selected among a plurality of icons displayed on the touchscreen, wherein each icon represents a respective application;

detecting a user selection of text of the displayed content of the first application;

detecting a writing gesture directed from the selected text of the first application to a target application displayed on the touchscreen, wherein the writing gesture is inputted by the user in contact with the touchscreen by writing at least one of a character, a number, a symbol, a figure, a line, and a combination thereof, to point from the selected text of the first application as a start point to the target application as an end point;

comparing a preset writing gesture of the target application with the detected writing gesture to determine whether the detected writing gesture is similar to, or identical to, one of a plurality of preset writing gestures stored in a writing gesture database;

when the detected writing gesture is determined that there is a similar or identical preset writing gesture in the writing gesture database, extracting an execution command of the target application mapped to the preset writing gesture corresponding to the detected writing gesture, wherein the execution command comprises various functions of the target application;

obtaining representative data of the content of the first application for controlling of execution of the various functions of the target application by recognizing the writing gesture, wherein the representative data is the text of the content of the first application displayed by execution of the first application; and

executing a function of the target application corresponding to the preset writing gesture using the representative data obtained from the first application.

2. The method of claim 1, wherein the representative data comprises latest data generated by performing of the function of the first application.

3. The method of claim 1, further comprising, before detecting the writing gesture, selecting one of a normal touch input mode and a writing gesture input mode for execution of the mobile terminal.

4. The method of claim 3, wherein the writing gesture input mode is selected in the mobile terminal by a finger of user, or with an exclusive pen.

5. The method of claim 1, wherein performing of the function of the target application is controlled by at least one of a number, a size and shape of a figure, and a length, direction, shape, and thickness of the line, of the input writing gesture.

6. The method of claim 1, wherein the target application comprises at least one of an icon, a widget, a list, text, and a Graphical User Interface (GUI).

7. The method of claim 1, further comprising:

storing information about at least one of the first application, the target application and information about the detected writing gesture.

8. The method of claim 1, further comprising:

making, by the mobile terminal, a request to repeat input of the writing gesture or a request for input of another writing gesture, when the detected writing gesture is determined to not match the preset writing gesture.

9. A mobile terminal comprising:

a touchscreen;

a memory storing instructions; and

a processor configured to execute the stored instructions to at least;

display on the touchscreen details of a first application selected among a plurality of icons displayed on the touchscreen, wherein each icon represents a respective application;

detect a user selection of text of the displayed content of the first application;

detect a writing gesture directed from the selected text of the first application to a target application displayed on the touchscreen, wherein the writing gesture is inputted by the user in contact with the touchscreen by writing at least one of a character, a number, a symbol, a figure, a line, and a combination thereof, to point from the selected text of the first application as a start point to the target application as an end point;

compare a preset writing gesture of the target application with the detected writing gesture to determine whether the detected writing gesture is similar to, or identical to, one of a plurality of preset writing gestures stored in a writing gesture database

when the detected writing gesture is determined that there is a similar or identical preset writing gesture in the writing gesture database, extracting an execution command of the target application mapped to the preset writing gesture corresponding to the detected writing gesture, wherein the execution command comprises various functions of the target application;

obtain representative data of the content of the first application for controlling of execution of the various functions of the target application by recognizing the writing gesture, wherein the representative data is the text of the content of the first application displayed by execution of the first application; and

execute a function of the target application corresponding to the preset writing gesture using the representative data obtained from the first i application.

10. The mobile terminal of claim 9, wherein the memory comprises:

the writing gesture database in which the preset writing gesture mapped to at least one of the first target application and the second target application is stored; and

the execution command database in which a command mapped to the preset writing gesture is stored,

wherein the processor is further configured to receive a selecting signal to select one of a normal touch input mode and a writing gesture input mode of the mobile terminal.

11. The mobile terminal of claim 10, wherein the processor is further configured to receive one of a selecting signal to select one of the normal touch input mode and the writing gesture input mode of the mobile terminal from a user of the mobile terminal, and a selecting signal to select the writing gesture input mode from an exclusive pen associated with the mobile terminal to switch the input mode to the writing gesture input mode.

12. The mobile terminal of claim 9, wherein the extracted representative data is converted into an image and displayed on the target application.

13. The mobile terminal of claim 12, wherein the representative data comprises latest data generated by the first application and information to identify content of the first application.

14. The mobile terminal of claim 9, wherein the touchscreen comprises:

a touch panel to which the writing gesture is inputted;

a screen activated according to performing of the function of at least one of the first application and the target application; and

a display panel outputting generated data.

15. The mobile terminal of claim 9, wherein performing of the function of the target application is controlled by at least one of a kind, a shape, and a direction of the writing gesture.