User interface apparatus in a user terminal and method for supporting a memo function

• US 10,877,642 B2
• Filed: 08/29/2013
• Issued: 12/29/2020
• Est. Priority Date: 08/30/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Bae, Joo-Yoon, Kim, Sung-Soo, Kim, Hwa-Kyung, Jun, Jin-Ha
• Primary Examiner(s): Chuang, Jung-Mu T
• Application Number:
  US14/013,542
• Publication Number:
  US 20140068517A1
• Time in Patent Office: 2,679 Days
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A hybrid User Interface (UI) apparatus for supporting both handwriting and speech in a user terminal and a method for supporting the same are provided. The apparatus is configured to display, upon user selection of a whole or of part of the contents of a note on a screen, menu icons for processing the selected contents, and process, upon user selection of one of the displayed menu icons, the selected contents by a function corresponding to the selected menu icon.

Claims

1. A method for operating a user interface (UI) in a user terminal, the method comprising:

receiving, from a touch screen of the user terminal, a first input associated with handwritten characters within a memo-based application and displaying the handwritten characters in the memo-based application on the touch screen of the user terminal;

receiving, from the touch screen of the user terminal, a second input for selecting a portion of the handwritten characters within the memo-based application when at least the portion of the handwritten characters is displayed on the touch screen;

receiving, from the touch screen of the user terminal, a third input associated with the portion of the handwritten characters, wherein the third input includes a combination of at least one handwritten symbol and at least one handwritten character;

identifying, by at least one processor of the user terminal, in response to the receiving of the third input, a plurality of applications that are capable of performing one or more functions to process the portion of the handwritten characters based on content of the second input associated with the portion of the handwritten characters;

displaying a menu icon corresponding to each application identified as capable of performing the one or more functions associated with processing the portion of the handwritten characters, each menu icon being displayed within the memo-based application on the touch screen of the user terminal;

in response to receiving a fourth input associated with a first menu icon, determining, by using a natural language interaction, whether additional information is required for a first application associated with the first menu icon to perform the one or more functions associated with processing the portion of the handwritten characters; and

in response to determining that no additional information is required for the first application to perform the one or more functions associated with processing the portion of the handwritten characters, executing the first application to perform the one or more functions associated with processing the portion of the handwritten characters.

2. The method of claim 1, wherein the handwritten characters are inputted using a finger or a touch electronic pen on the touch screen.

3. The method of claim 1, wherein the handwritten characters are inputted on the touch screen provided by a memo application.

4. The method of claim 1, wherein the handwritten characters include at least one handwritten command for processing the content of the second input.

5. The method of claim 1, wherein the displaying of the menu icon corresponding to each application identified as capable of performing the one or more functions associated with processing the portion of the handwritten characters comprises displaying a menu icon associated with at least one of a mail transmission application, a call application, a search application, or a message transmission application.

6. The method of claim 1, further comprising:

in response to determining that additional information is required for the first application to perform the one or more functions associated with processing the portion of the handwritten characters;

determining a plurality of different meanings based on the portion of the handwritten characters and the first menu icon,

displaying, on the touch screen, at least one question generated based on the plurality of different meanings,

receiving, from the touch screen, a fifth input including additional information provided in response to the at least one question, and

executing the first application to perform the one or more functions associated with processing the portion of the handwritten characters based on the first menu icon, the portion of the handwritten characters, and the additional information.

13. A non-transitory computer readable storage medium with a computer program stored thereon, the computer program executable by at least one processor to perform the method of claim 1.

7. A user terminal for operating a user interface (UI), the user terminal comprising:

a touch screen; and

at least one processor configured to;

receive, from the touch screen, a first input associated with handwritten characters within a memo-based application and display the handwritten characters in the memo-based application on the touch screen,

receive, from the touch screen, a second input for selecting a portion of the handwritten characters within the memo-based application when at least the portion of the handwritten characters is displayed on the touch screen,

receive, from the touch screen, a third input associated with the portion of the handwritten characters, wherein the third input includes a combination of at least one ef-a handwritten symbol and at least one handwritten character,

identify, by at least one processor of the user terminal, in response to the receiving of the third input, a plurality of applications that are capable of performing one or more functions to process the portion of the handwritten characters based on a content of the second input associated with the portion of the handwritten characters,

display a menu icon corresponding to each application identified as capable of performing the one or more functions associated with processing the portion of the handwritten characters, each menu icon being displayed within the memo-based application on the touch screen,

in response to receiving a fourth input associated with a first menu icon, determine, by using a natural language interaction, whether additional information is required for a first application associated with the first menu icon to perform the one or more functions associated with processing the portion of the handwritten characters, and

in response to determining that no additional information is required for the first application to perform the one or more functions associated with processing the portion of the handwritten characters, execute the first application to perform the one or more functions associated with processing the portion of the handwritten characters.

8. The user terminal of claim 7, wherein the handwritten characters are inputted using a finger or a touch electronic pen on the touch screen.

9. The user terminal of claim 7, wherein the handwritten characters are inputted on the touch screen provided by a memo application.

10. The user terminal of claim 7, wherein the handwritten characters include at least one handwritten command for processing the content of the second input.

11. The user terminal of claim 7, wherein, to display the menu icon corresponding to each application identified as capable of performing the one or more functions associated with processing the portion of the handwritten characters, the at least one processor is further configured to display at least one menu icon associated with at least one of a mail transmission application, a call application, a search application, or a message transmission application.

12. The user terminal of claim 7, wherein the at least one processor is further configured to:

in response to determining that additional information is required for the first application to perform the one or more functions associated with processing the portion of the handwritten characters;

determine a plurality of different meanings based on the portion of the handwritten characters and the first menu icon,

display, on the touch screen, at least one question generated based on the plurality of different meanings,

receive, from the touch screen, a fifth input including additional information provided in response to the at least one question, and

execute the first application to perform the one or more functions associated with processing the portion of the handwritten characters based on the first menu icon, the portion of the handwritten characters, and the additional information.