Layout constraint manipulation via user gesture recognition

• US 9,405,449 B2
• Filed: 01/14/2010
• Issued: 08/02/2016
• Est. Priority Date: 01/14/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Winternitz, Fabian O., Rowan, Kael R., Dwyer, Timothy G., Kent, Stuart
• Primary Examiner(s): Patel, Manglesh M
• Application Number:
  US12/687,388
• Publication Number:
  US 20110173530A1
• Time in Patent Office: 2,392 Days
• Field of Search: 715/200, 715/246, 715/243, 715/788, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

Systems and methods are described herein for enabling a user of a diagramming system to select elements of a diagram and apply a layout constraint thereto. In accordance with certain embodiments described herein, the systems and methods provide the user with a means to perform these tasks in a manner that is faster, easier and more intuitive than interfaces provided by conventional diagramming systems. For example, in accordance with at least one embodiment described herein, a single user gesture is obtained and the steps of selecting elements of a diagram to which a layout constraint is to be applied and selecting the layout constraint are both performed based on the single user gesture.

Claims

1. A method implemented in a processor-based computing system, comprising:

rendering at least a portion of a diagram to a display device coupled to the computing system, the diagram comprising a plurality of elements;

identifying a user gesture based on input received via at least one user input device coupled to the computing system;

selecting, subsequent to said identifying, a subset of elements from among a plurality of different elements of the diagram based on the identified user gesture;

selecting, automatically, a layout constraint from among a plurality of layout constraints based on a path or a shape of the identified user gesture, each layout constraint controlling a manner in which elements of a diagram are arranged when rendered to a display device;

applying the selected layout constraint to the selected subset of elements; and

rendering at least a portion of the diagram to the display device after application of the selected layout constraint to the selected subset of elements.

2. The method of claim 1, wherein identifying the user gesture comprises:

determining if the input corresponds to a user gesture in a library of predefined user gestures.

3. The method of claim 1, wherein identifying the user gesture based on input received via the at least one user input device comprises identifying the user gesture based on input received via at least one of a mouse, a touch screen, or a video capture device.

4. The method of claim 1, wherein selecting the subset of elements from among the plurality of different elements of the diagram based on the identified user gesture comprises:

selecting elements of the diagram that are intersected by a path defined by the user gesture.

5. The method of claim 4, wherein selecting the layout constraint based on the identified user gesture comprises:

selecting a vertical alignment constraint if the path defined by the user gesture corresponds to a vertical line.

6. The method of claim 4, wherein selecting the layout constraint based on the identified user gesture comprises:

selecting a horizontal alignment constraint if the path defined by the user gesture corresponds to a horizontal line.

7. The method of claim 4, wherein selecting the layout constraint based on the identified user gesture comprises:

selecting a circular alignment constraint if the path defined by the user gesture begins and ends inside the same element.

8. The method of claim 4, wherein selecting the layout constraint based on the identified user gesture comprises:

identifying a first element intersected by the path, the first element comprising a plurality of sides;

identifying a side of the first element that was first intersected by the path;

identifying a last element intersected by the path, the last element comprising a plurality of sides;

identifying a side of the last element that was last intersected by the path;

selecting an alignment constraint based on the identified sides of the first element and the last element.

9. The method of claim 1, wherein selecting the subset of elements from among the plurality of different elements of the diagram based on the identified user gesture comprises selecting elements of the diagram that are enclosed within a shape defined by the user gesture;

and

wherein selecting the layout constraint based on the identified user gesture comprises selecting a grouping constraint.

10. The method of claim 1, wherein identifying the user gesture based on input received via at least one user input device coupled to the computing system comprises identifying a multi-touch user gesture.

11. The method of claim 1, further comprising:

rendering a line that follows a path defined by the user gesture to the display device.

12. A system comprising:

a display device;

at least one user input device;

a memory that stores at least a diagramming module and a user gesture processing module; and

a processing unit that executes the diagramming module to render at least a portion of a diagram to the display device, the diagram comprising a plurality of elements;

the processing unit further executes the user gesture processing module to identify a user gesture based on input received via the at least one user input device, and

the processing unit further executes the diagramming module to select, subsequent to identifying the user gesture, a subset of elements from among a plurality of different elements of the diagram based on the identified user gesture, to automatically select a layout constraint from among a plurality of layout constraints based on a path or a shape of the identified user gesture, wherein each layout constraint controls a manner in which elements of a diagram are arranged when rendered to a display device, to apply the selected layout constraint to the selected subset of elements and to render at least a portion of the diagram to the display device after application of the selected layout constraint to the selected subset of elements.

13. The system of claim 12, wherein the memory further stores a library of predefined user gestures and wherein the processing unit executes the user gesture processing module to determine if the input corresponds to a user gesture in the library of predefined user gestures.

14. The system of claim 12, wherein the at least one user input device comprises at least one of a mouse, a touch screen, or a video capture device.

15. The system of claim 12, wherein the processing unit executes the diagramming module to select elements of the diagram that are intersected by a path defined by the user gesture.

16. The system of claim 12, wherein the processing unit executes the diagramming module to select elements of the diagram that are enclosed within a shape defined by the user gesture.

17. The system of claim 12, wherein the processing unit executes the diagramming module to render a line that follows a path defined by the user gesture to the display device.

18. A computer-readable program storage device having instructions stored thereon that, when executed by a processing device, perform a method, the method comprising:

rendering at least a portion of a diagram to a display device coupled to the computing system, the diagram comprising a plurality of elements;

identifying a user gesture based on input received via at least one user input device coupled to the computing system;

selecting, subsequent to said identifying, a subset of elements from among a plurality of different elements of the diagram based on the identified user gesture;

selecting, automatically, a layout constraint from among a plurality of layout constraints based on a path or a shape of the identified user gesture, each layout constraint controlling a manner in which elements of a diagram are arranged when rendered to a display device;

applying the selected layout constraint to the selected subset of elements; and

rendering at least a portion of the diagram to the display device after application of the selected layout constraint to the selected subset of elements.

19. The computer-readable program storage device of claim 18, wherein selecting the subset of elements from among the plurality of different elements of the diagram based on the identified user gesture comprises selecting elements of the diagram that are intersected by a path defined by the user gesture.

20. The computer-readable program storage device of claim 18, wherein selecting the subset of elements from among the plurality of different elements of the diagram based on the identified user gesture comprises selecting elements of the diagram that are enclosed within a shape defined by the user gesture.