Natural input for spreadsheet actions

• US 9,747,270 B2
• Filed: 01/07/2011
• Issued: 08/29/2017
• Est. Priority Date: 01/07/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Campbell, John, Lin, Amy, Waldman, Lawrence, Der, Sherman, Cheng, Karen, Doan, Christopher, Constantine, Thomas, Rothschiller, Chad, Ma, Yun
• Primary Examiner(s): Augustine, Nicholas
• Application Number:
  US12/986,473
• Publication Number:
  US 20120180002A1
• Time in Patent Office: 2,426 Days
• Field of Search: 715745, 715810, 715834, 715841, 715863
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/18 of spreadsheets form-filling G06F40/174

Abstract

Different gestures and actions are used to interact with spreadsheets. The gestures are used in manipulating the spreadsheet and performing other actions in the spreadsheet. For example, gestures may be used to move within the spreadsheet, select data, filter, sort, drill down/up, zoom, split rows/columns, perform undo/redo actions, and the like. Sensors that are associated with a device may also be used in interacting with spreadsheets. For example, an accelerometer may be used for moving and performing operations within the spreadsheet.

Claims

1. A method for interacting with a spreadsheet, comprising:

receiving a gesture on a touch input device, wherein the received gesture is used to interact with the spreadsheet;

receiving sensor information, wherein the sensor information is used to interact with the spreadsheet, wherein the sensor information comprises accelerometer data that is used in changing a view of the spreadsheet, wherein the accelerometer data is used to perform one of;

a zoom out operation with respect to the spreadsheet in response to moving the touch input device in a first direction, and

a zoom in operation with respect to the spreadsheet in response to moving the touch input device in a second direction, and

wherein the zoom out operation and the zoom in operation slow at one or more predetermined locations within the spreadsheet, wherein the one or more predetermined locations are prepositioned within the spreadsheet based at least in part on an arrangement of data within the spreadsheet;

automatically determining a type of the received gesture with respect to the spreadsheet;

performing an action in response to at least one of the determined type of the received gesture with respect to the spreadsheet and the sensor information, wherein performing the action in response to the determined type of the received gesture comprises one or more of;

performing a first action in response to the spreadsheet receiving a karate chop gesture, performing a second action in response to the spreadsheet receiving a chart gesture, performing a third action in response to the spreadsheet receiving a comment gesture, performing a fourth action in response to the spreadsheet receiving a grip gesture, performing a fifth action in response to the spreadsheet receiving a gesture within a gesture box, and performing a sixth action in response to the spreadsheet receiving a sort gesture; and

updating a display of the spreadsheet.

2. The method of claim 1, further comprising:

in response to receiving the karate chop gesture on the spreadsheet, inserting at least one of a row and a column in the spreadsheet near a location of the karate chop gesture.

3. The method of claim 1, further comprising:

in response to receiving the chart gesture, inserting at least one of;

a trend line on a displayed chart in the spreadsheet and a chart in the spreadsheet.

4. The method of claim 1, further comprising:

in response to receiving the comment gesture, displaying a comment box to receive a comment at a location near a received gesture location.

5. The method of claim 1, further comprising:

displaying a grip user interface element on the display of the spreadsheet;

in response to receiving the grip gesture when the grip user interface element is pressed, selecting a portion of the spreadsheet in response to a finger being moved across the spreadsheet; and

in response to receiving the grip gesture when the grip user interface element is not pressed then changing the display of the spreadsheet.

6. The method of claim 1, further comprising:

in response to receiving a zoom gesture, performing at least one of;

displaying a thumbnail for each sheet within a workbook that is associated with the spreadsheet when a zoom level exceeds a predetermined threshold; and

displaying a drilled down view of the data of the spreadsheet.

7. The method of claim 1, further comprising:

displaying a gesture box for receiving gesture commands;

when a gesture is received within the gesture box, a gesture command from the gesture; and

performing the gesture command on data in the spreadsheet.

8. The method of claim 1, further comprising:

displaying a vortex animation in response to a cut command and in response to a paste command based on a location of the received gesture.

9. The method of claim 1, wherein automatically determining the type of the received gesture comprises determining when the gesture is the sort gesture, and wherein performing the action comprises sorting at least a portion of the data within the spreadsheet.

10. The method of claim 1, further comprising:

determining when the spreadsheet is tilted;

determining a speed of the tilting; and

wherein performing an action comprises at least one of;

scrolling a display of the spreadsheet based on at least one of the tilt and the speed of the tilting; and

displaying spreadsheet objects in the spreadsheet spilling to a top of the spreadsheet.

11. The method of claim 1, further comprising:

automatically slowing the scrolling of the spreadsheet when a speed bump that is associated with a position in the spreadsheet is within a predetermined position.

12. The method of claim 1, further comprising:

determining when the spreadsheet is shaken and performing at least one of;

clearing a filter on a spreadsheet object; and

performing a gravity sort that sorts spreadsheet data based on a weight of the spreadsheet objects.

13. The method of claim 1, further comprising:

determining when the spreadsheet is rotated; and

in response, changing a type of view of the spreadsheet.

14. A computer-readable storage medium storing computer-executable instructions for interacting with a spreadsheet comprising:

determining when a gesture on a touch input device is received, wherein the received gesture is used to interact with the spreadsheet;

using accelerometer data as part of changing a view of the spreadsheet including using the accelerometer data to perform one of;

a zoom out operation with respect to the spreadsheet in response to moving the touch input device in a first direction, and

a zoom in operation with respect to the spreadsheet in response to moving the touch input device in a second direction,

wherein the zoom out operation and the zoom in operation slow at one or more predetermined locations within the spreadsheet, wherein the one or more predetermined locations are prepositioned within the spreadsheet based at least in part on an arrangement of data within the spreadsheet;

automatically determining a type of the received gesture with respect to the spreadsheet;

performing an action in response to the determined type of the received gesture with respect to the spreadsheet, wherein performing the action in response to the determined type of the received gesture comprises one or more of;

performing a first action in response to the spreadsheet receiving a karate chop gesture, performing a second action in response to the spreadsheet receiving a chart gesture, performing a third action in response to the spreadsheet receiving a comment gesture, performing a fourth action in response to the spreadsheet receiving a grip gesture, performing a fifth action in response to the spreadsheet receiving a gesture in a gesture box, and performing a sixth action in response to the spreadsheet receiving a sort gesture; and

updating a display of the spreadsheet.

15. The computer-readable storage medium of claim 14, further comprising:

using the accelerometer data to reset a state of the spreadsheet.

16. The computer-readable storage medium of claim 15, further comprising:

determining when the spreadsheet is tilted; and

wherein performing the action comprises displaying spreadsheet objects in the spreadsheet spilling to a top of the spreadsheet.

17. The computer-readable storage medium of claim 15, further comprising:

in response to a movement of the spreadsheet, scrolling the spreadsheet; and

automatically slowing the scrolling of the spreadsheet when a speed bump that is associated with a position in the spreadsheet is within a predetermined position.

18. The computer-readable storage medium of claim 15, further comprising:

determining when the spreadsheet is shaken and performing at least one of;

clearing a filter on a spreadsheet object; and

performing a gravity sort that sorts spreadsheet data based on a weight of the spreadsheet objects.

19. A system for interacting with a spreadsheet, comprising:

a processor, memory, and a computer-readable storage medium;

an operating environment stored on the computer-readable storage medium and executing on the processor;

a spreadsheet application;

a spreadsheet, wherein the spreadsheet comprises spreadsheet objects; and

a gesture manager operating in conjunction with the spreadsheet application that is configured to perform actions comprising;

determining when a gesture on a touch input device is received, wherein the received gesture is used to interact with the spreadsheet;

determining when sensor information is received that relates to a movement of the spreadsheet, wherein the sensor information comprises accelerometer data;

using the accelerometer data to perform one of;

a zoom out operation with respect to the spreadsheet in response to moving the touch input device in a first direction, and

a zoom in operation with respect to the spreadsheet in response to moving the touch input device in a second direction,

wherein the zoom out operation and the zoom in operation slow at one or more predetermined locations within the spreadsheet, wherein the one or more predetermined locations are prepositioned within the spreadsheet based at least in part on an arrangement of data within the spreadsheet;

automatically determining a type of the received gesture when received with respect to the spreadsheet;

performing an action in response to the determined type of the received gesture with respect to the spreadsheet, wherein performing the action in response to the determined type of the received gesture comprises performing actions selected from the group of;

performing a first action in response to the spreadsheet receiving a karate chop gesture, performing a second action in response to the spreadsheet receiving a chart gesture, performing a third action in response to the spreadsheet receiving a comment gesture, performing a fourth action in response to the spreadsheet receiving a grip gesture, performing a fifth action in response to the spreadsheet receiving a gesture in a gesture box, and performing a sixth action in response to the spreadsheet receiving a sort gesture; and

updating a display of the spreadsheet.

20. The system of claim 19, further comprising:

determining when the spreadsheet is tilted;

determining when the spreadsheet is rotated;

determining when the spreadsheet is shaken; and

wherein performing the action comprises at least one of;

displaying spreadsheet objects in the spreadsheet spilling to a top of the spreadsheet;

scrolling the spreadsheet;

reversing a direction of displayed data;

changing a sorting of data; and

changing a view type of the spreadsheet.