INFORMATION TRIAGE USING SCREEN-CONTACTING GESTURES

• US 20130328786A1
• Filed: 06/07/2012
• Published: 12/12/2013
• Est. Priority Date: 06/07/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hinckley, Ken
• Granted Patent: US 9,229,539 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A user is able to triage information on a touch-enabled computing device. Information objects are displayed on a touch-sensitive screen of the device. Whenever the user makes a first gesture on the screen using a first user input modality, the first gesture and that it was made using this first modality are identified, a first information management operation specifically associated with the first gesture being made with this first modality is also identified, and this first operation is implemented on the objects. Whenever the user subsequently makes a second gesture on the screen using a second user input modality which is different than the first modality, the second gesture and that it was made using this second modality are identified, a second information management operation specifically associated with the second gesture being made with this second modality is also identified, and this second operation is implemented on the objects.

Claims

1. A computer-implemented process for enabling a user to quickly triage information on a touch-enabled computing device, comprising:

using the computing device to perform the following process actions;

displaying one or more information objects on a touch-sensitive display screen of the computing device;

whenever the user makes a first screen-contacting gesture on the screen using a first user input modality,

identifying the first screen-contacting gesture and identifying that said gesture was made using the first user input modality,

identifying a first information management operation that is specifically associated with the first screen-contacting gesture being made with the first user input modality, and

implementing the first information management operation on the information objects; and

whenever the user subsequently makes a second screen-contacting gesture on the screen using a second user input modality which is different than the first user input modality,

identifying the second screen-contacting gesture and identifying that said gesture was made using the second user input modality,

identifying a second information management operation that is specifically associated with the second screen-contacting gesture being made with the second user input modality, and

implementing the second information management operation on the information objects.

2. The process of claim 1, wherein the first user input modality comprises a stylus which is held by the user, and the second user input modality comprises one or more fingers of the user.

3. The process of claim 1, wherein the first user input modality comprises one or more fingers of the user, and the second user input modality comprises a stylus which is held by the user.

4. The process of claim 1, wherein,

the process action of displaying one or more information objects on a touch-sensitive display screen of the computing device comprises an action of displaying a list of email messages within a first sector on the screen,

the first screen-contacting gesture comprises a reply gesture comprising a stroke which starts on top of a desired one of the email messages in said list and ends within a second sector on the screen,

the process action of implementing the first information management operation on the information objects comprises the actions of displaying the contents of the desired one of the email messages within the second sector, and then displaying a digital inking sector on top of a portion of said contents,

the second screen-contacting gesture comprises a sector resizing gesture which specifies how to modify the size of the digital inking sector, and

the process action of implementing the second information management operation on the information objects comprises an action of modifying the size of the digital inking sector according to the sector resizing gesture.

5. The process of claim 4, further comprising an action of, whenever the user makes one or more strokes on the screen within the digital inking sector using the first user input modality,

identifying that said strokes were made within the digital inking sector using the first user input modality,

digitizing each of said strokes thus creating an electronic ink representation thereof, and

displaying said digitization within the digital inking sector.

6. The process of claim 5, further comprising an action of automatically increasing the size of the digital inking sector as necessary to accommodate the one or more strokes the user makes on the screen within said sector.

7. The process of claim 1, wherein the process action of displaying one or more information objects on a touch-sensitive display screen of the computing device comprises an action of displaying a plurality of information objects within a first sector on the screen, further comprising an action of displaying one or more graphical elements within a second sector on the screen, each of the graphical elements corresponding to a different folder, wherein,

the first screen-contacting gesture comprises a plurality of single-object selection gestures each of which taps on a different one of the information objects,

the process action of implementing the first information management operation on the information objects comprises an action of selecting said tapped on information objects,

the second screen-contacting gesture comprises an object movement gesture comprising a stroke which starts on top of any one of the selected information objects and ends on top of a desired one of the graphical elements within the second sector, and

the process action of implementing the second information management operation on the information objects comprises an action of moving the selected information objects into the folder that corresponds to the desired one of the graphical elements.

8. The process of claim 1, wherein the process action of displaying one or more information objects on a touch-sensitive display screen of the computing device comprises an action of displaying a plurality of information objects within a first sector on the screen, further comprising an action of displaying one or more graphical elements within a second sector on the screen, each of the graphical elements corresponding to a different folder, wherein,

the first screen-contacting gesture comprises a plurality of multi-object selection gestures each of which comprises a stroke which passes over a different two or more of the information objects,

the process action of implementing the first information management operation on the information objects comprises an action of selecting said passed over information objects,

the second screen-contacting gesture comprises an object movement gesture comprising a stroke which starts on top of any one of the selected information objects and ends on top of a desired one of the graphical elements within the second sector, and

the process action of implementing the second information management operation on the information objects comprises an action of moving the selected information objects into the folder that corresponds to the desired one of the graphical elements.

9. The process of claim 1, wherein the process action of displaying one or more information objects on a touch-sensitive display screen of the computing device comprises an action of displaying a plurality of information objects within a first sector on the screen, further comprising an action of displaying one or more graphical elements within a second sector on the screen, each of the graphical elements corresponding to a different folder, wherein,

the first screen-contacting gesture comprises a combination of one or more single-object selection gestures and one or more multi-object selection gestures,

each of the single-object selection gestures taps on a different one of the information objects,

each of the multi-object selection gestures comprises a stroke which passes over a different two or more of the information objects,

the process action of implementing the first information management operation on the information objects comprises an action of selecting said tapped on and passed over information objects,

the second screen-contacting gesture comprises an object movement gesture comprising a stroke which starts on top of any one of the selected information objects and ends on top of a desired one of the graphical elements within the second sector, and

the process action of implementing the second information management operation on the information objects comprises an action of moving the selected information objects into the folder that corresponds to the desired one of the graphical elements.

10. The process of claim 9, wherein the process action of implementing the first information management operation on the information objects further comprises an action of visually highlighting the information objects that are tapped on by the single-object selection gestures and passed over by the multi-object selection gestures.

11. The process of claim 9, wherein the process action of implementing the second information management operation on the information objects further comprises an action of displaying a callout element adjacent to the desired one of the graphical elements, wherein the callout element comprises a name of the folder that corresponds to the desired one of the graphical elements, thus allowing the user to read said name whenever the second user input modality is obstructing the desired one of the graphical elements.

12. The process of claim 1, wherein,

the process action of displaying one or more information objects on a touch-sensitive display screen of the computing device comprises an action of displaying a plurality of information objects within a given sector on the screen,

the first screen-contacting gesture comprises a combination of one or more single-object selection gestures and one or more multi-object selection gestures,

each of the single-object selection gestures taps on a different one of the information objects,

each of the multi-object selection gestures comprises a stroke which passes over a different two or more of the information objects,

the process action of implementing the first information management operation on the information objects comprises an action of selecting said tapped on and passed over information objects,

the second screen-contacting gesture comprises a scrolling gesture which specifies how the information objects being displayed within the sector are to be scrolled there-within, and

the process action of implementing the second information management operation on the information objects comprises an action of scrolling the information objects being displayed within the sector according to the scrolling gesture while maintaining the selection of said tapped on and passed over information objects.

13. The process of claim 1, wherein,

the one or more information objects are displayed within a first sector on the screen,

the first screen-contacting gesture is made within a first one or more sectors on the screen, one of which is the first sector,

the process action of identifying a first information management operation that is specifically associated with the first screen-contacting gesture being made with the first user input modality comprises the actions of,

identifying said first one or more sectors, and

identifying a first operating context of the computing device at the time the first screen-contacting gesture is made, wherein the first information management operation is specific to said first one or more sectors and said first operating context,

the second screen-contacting gesture is made within a second one or more sectors on the screen, one of which is the same as one of said first one or more sectors, and

the process action of identifying a second information management operation that is specifically associated with the second screen-contacting gesture being made with the second user input modality comprises the actions of,

identifying said second one or more sectors, and

identifying a second operating context of the computing device at the time the second screen-contacting gesture is made, wherein the second information management operation is specific to said second one or more sectors and said second operating context.

14. The process of claim 1, wherein,

the one or more information objects is displayed within a given sector on the screen,

the first screen-contacting gesture comprises an open in new sector gesture comprising an initial stroke segment and a final stroke segment which interconnect at an inflection point, wherein the initial stroke segment passes over a desired one of the information objects, and

the process action of implementing the first information management operation on the information objects comprises an action of displaying the contents of the desired one of the information objects within a new sector on the screen.

15. The process of claim 1, wherein,

the first screen-contacting gesture comprises an object deletion gesture comprising an initial stroke segment and a final stroke segment which interconnect at an inflection point, wherein the initial stroke segment passes over a desired one or more of the information objects, and

the process action of implementing the first information management operation on the information objects comprises an action of deleting the desired one or more of the information objects.

16. The process of claim 1, wherein,

the process action of displaying one or more information objects on a touch-sensitive display screen of the computing device comprises an action of displaying a plurality of information objects on the screen,

the first screen-contacting gesture comprises one or more multi-object selection gestures each of which comprises a stroke which passes over a different two or more of the information objects,

the process action of implementing the first information management operation on the information objects comprises the actions of selecting said passed over information objects and displaying a command selection widget on the screen,

the command selection widget comprises one or more radial direction indicators each of which is associated with a different information triage command, and

the number of said indicators and the different information triage command that is associated with each of said indicators are dependent on a current operating context of the computing device,

further comprising the actions of, whenever the user subsequently makes another screen-contacting gesture on the screen using any user input modality and said other screen-contacting gesture comprises a stroke which substantially passes over and directionally follows a given one of the radial direction indicators,

identifying said given one of the radial direction indicators and identifying a particular information triage command that is associated therewith, and

implementing said particular information triage command on the selected information objects.

17. A computer-implemented process for enabling a user to quickly triage information on a touch-enabled computing device, comprising:

using the computing device to perform the following process actions;

displaying a list of information objects on a touch-sensitive display screen of the computing device;

whenever the user makes a multi-object framing gesture on the screen using two fingers, wherein one of the fingers contacts the screen on top of a first information object in the list and the other of the fingers contacts the screen on top of a second information object in the list,

identifying said framing gesture and identifying that said framing gesture was made using two fingers, and

selecting the first and second information objects and any other information objects there-between in the list; and

whenever, concurrent with said framing gesture being made on the screen, the user also makes an object manipulation gesture on the screen using a stylus which is held by the user, wherein the object manipulation gesture comprises one or more strokes each of which passes over one or more of the selected information objects,

identifying the object manipulation gesture and identifying that said gesture was made using the stylus,

identifying an information management operation that is specifically associated with the object manipulation gesture being made with the stylus, and

implementing the information management operation on the selected information objects.

18. The process of claim 17, wherein,

the object manipulation gesture comprises an object deletion gesture,

the one or more strokes comprise either one or more vertical strokes, or one or more horizontal strokes, or a pair of strokes which intersect one another, and

the process action of implementing the information management operation on the selected information objects comprises an action of deleting the selected information objects.

19. The process of claim 17, wherein the list of information objects is displayed within a first sector on the screen, further comprising an action of displaying one or more graphical elements within a second sector on the screen, each of the graphical elements corresponding to a different folder, wherein,

the object manipulation gesture comprises an object movement gesture comprising a stroke which starts on top of any one of the selected information objects and ends on top of a desired one of the graphical elements within the second sector, and

the process action of implementing the information management operation on the selected information objects comprises an action of moving the selected information objects into the folder that corresponds to the desired one of the graphical elements.

20. A computer-implemented process for enabling a user to quickly triage information on a touch-enabled computing device, comprising:

using the computing device to perform the following process actions;

displaying a plurality of information objects within a first sector on a touch-sensitive display screen of the computing device;

displaying one or more graphical elements within a second sector on the screen, each of the graphical elements corresponding to a different folder;

whenever the user makes a combination of one or more single-object selection gestures and one or more multi-object selection gestures on the screen using a stylus which is held by the user, wherein each of the single-object selection gestures taps on a different one of the information objects, and each of the multi-object selection gestures comprises a stroke which passes over a different two or more of the information objects,

identifying said combination of gestures and identifying that said single-object selection gestures and said multi-object selection gestures were made using the stylus, and

selecting and visually highlighting said tapped on and passed over information objects; and

whenever the user subsequently makes an object movement gesture on the screen using one or more of their fingers, wherein the object movement gesture comprises a stroke which starts on top of any one of the selected information objects and ends on top of a desired one of the graphical elements within the second sector,

identifying the object movement gesture and identifying that said gesture was made using said fingers,

displaying a callout element adjacent to the desired one of the graphical elements, wherein the callout element comprises a name of the folder that corresponds to the desired one of the graphical elements, and

moving the selected information objects into the folder that corresponds to the desired one of the graphical elements.