Capturing pen input by a pen-aware shell

• US 10,254,858 B2
• Filed: 01/25/2017
• Issued: 04/09/2019
• Est. Priority Date: 01/25/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Verma, Arunabh P., Beland, Sophie A., Oke, Oluwadara, Geraci, II, William M.
• Primary Examiner(s): Polo, Gustavo D
• Application Number:
  US15/415,862
• Publication Number:
  US 20180210566A1
• Time in Patent Office: 804 Days
• Field of Search: None
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/171 by use of digital ink

Abstract

The electronic devices described herein are configured to enhance user experience associated with using a pen on the touchscreens of the electronic devices. Contact of a pen on the touch screen is detected by a persistent, pen-aware shell that occupies some or all of the touch screen user interface throughout operation of the electronic device. Detected pen contact with the touch screen is captured/collected as pen input and used to perform a function of the electronic device or shared with an application such that the application may perform a function based on the pen input. A performed function or application with which to share the pen in put is selected by a user, automatically by an operating system, or a combination of the two. Automatic capture of pen input by the persistent, pen-aware shell provides an intuitive way of making use of pen/touch screen capability of an electronic device.

Claims

1. An electronic device comprising:

a user interface;

at least one processor; and

at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the electronic device to at least;

execute a pen-aware shell that occupies at least a portion of the user interface;

upon detecting contact from a pen device on the user interface, capture, by the pen-aware shell, input from the pen device;

upon the pen-aware shell determining the capture of the input from the pen device is complete, convert the captured input to an ink data structure; and

share the ink data structure with an application at least by selecting an application with which to share the ink data structure, wherein the selecting is based at least on a context associated with the ink data structure.

2. The electronic device of claim 1, wherein selecting an application includes providing a plurality of applications for user selection.

3. The electronic device of claim 2, wherein the plurality of applications is based on a defined set of applications.

4. The electronic device of claim 1, wherein the user interface is partially occupied by a pen-compatible application;

and

wherein the pen-aware shell occupies a remainder of the user interface not occupied by the application.

5. The electronic device of claim 1, wherein the user interface is partially occupied by a non-pen-compatible application;

and

wherein the portion of the user interface occupied by the pen-aware shell includes the non-pen-compatible application.

6. An electronic device comprising:

a user interface;

at least one processor; and

at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the electronic device to at least;

execute a pen-aware shell that occupies at least a portion of the user interface;

upon detecting contact from a pen device on the user interface, capture, by the pen-aware shell, input from the pen device;

upon the pen-aware shell determining the capture of the input from the pen device is complete, convert the captured input to an ink data structure; and

share the ink data structure with an application, wherein the electronic device at least;

provides a plurality of applications for user selection, wherein the plurality of applications is based on context associated with the ink data structure; and

selects an application with which to share the ink data structure.

7. The electronic device of claim 6, wherein context associated with the ink data structure includes at least one of a keyword or a symbol associated with at least one application.

8. The electronic device of claim 6, wherein the user interface is partially occupied by a pen-compatible application;

and

wherein the pen-aware shell occupies a remainder of the user interface not occupied by the pen-compatible application.

9. A method comprising:

executing a persistent shell across a background of a desktop;

upon detecting when a pen touches the background, capturing, by the persistent shell, input from the pen;

converting the captured input to an ink data structure;

or

sharing the ink data structure with an application, wherein sharing the ink data structure with an application includes selecting an application with which to share the ink data based on a context associated with the ink data structure.

10. The method of claim 9, wherein performing a function based on the ink data structure includes determining a function command based on the ink data structure;

and

executing the determined function command.

11. The method of claim 9, wherein performing a function based on the ink data structure includes at least one of running an application, opening a file, searching a directory, searching using an Internet search engine, or adjusting a setting of the electronic device.

12. The method of claim 9, wherein sharing the ink data structure with an application includes dragging a representation of the ink data structure onto an icon associated with the application.

13. The method of claim 9, wherein selecting an application includes providing a plurality of applications for user selection.

14. The method of claim 9, wherein the background is partially occupied by a pen-compatible application;

and

wherein the pen-aware shell occupies a remainder of the background not occupied by the pen-compatible application.

15. The method of claim 9, wherein the background is partially occupied by a non-pen-compatible application;

and

wherein the portion of the background occupied by the pen-aware shell includes the non-pen-compatible application.

16. One or more computer storage media having computer-executable instructions that, upon execution by a processor, cause the processor to at least:

execute a pen-aware shell across at least a portion of a user interface;

upon detecting pen input on the pen-aware shell, capture, by the pen-aware shell, the pen input;

convert the pen input to an ink data structure;

and

share the ink data structure with with an application, wherein sharing the ink data structure with an application includes selecting an application with which to share the ink data based on a context associated with the ink data structure.

17. The one or more computer storage media of claim 16, wherein selecting an application with which to share the ink data based on a context associated with the ink data structure includes selecting an application based on at least one of a keyword or symbol in the ink data structure.

18. The one or more computer storage media of claim 16, wherein selecting an application with which to share the ink data based on a context associated with the ink data structure includes providing a plurality of applications for user selection.

19. The electronic device of claim 16, wherein the user interface is partially occupied by a pen-compatible application;

and

wherein the pen-aware shell occupies a remainder of the user interface not occupied by the pen-compatible application.

20. The electronic device of claim 16, wherein the user interface is partially occupied by a non-pen-compatible application;

and

wherein the portion of the user interface occupied by the pen-aware shell includes the non-pen-compatible application.