Layout constraint manipulation via user gesture recognition

• US 10,599,311 B2
• Filed: 07/22/2016
• Issued: 03/24/2020
• Est. Priority Date: 01/14/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Winternitz, Fabian O., Rowan, Kael R., Dwyer, Timothy G., Kent, Stuart
• Primary Examiner(s): Patel, Manglesh M
• Application Number:
  US15/217,339
• Publication Number:
  US 20160328129A1
• Time in Patent Office: 1,341 Days
• Field of Search: 715243, 715200, 715246, 715788, 715863
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

Systems and methods are described herein for enabling a user of a diagramming system to select elements of a diagram and apply a layout constraint thereto. In accordance with certain embodiments described herein, the systems and methods provide the user with a means to perform these tasks in a manner that is faster, easier and more intuitive than interfaces provided by conventional diagramming systems. For example, in accordance with at least one embodiment described herein, a single user gesture is obtained and the steps of selecting elements of a diagram to which a layout constraint is to be applied and selecting the layout constraint are both performed based on the single user gesture.

Claims

1. A method implemented in a processor-based computing system, comprising:

rendering at least a portion of a diagram to a display device of the processor-based computing system, the diagram comprising a plurality of different elements;

identifying a first user gesture and subsequently selecting a first subset of elements from among the plurality of different elements of the diagram based on the identified first user gesture;

selecting, automatically, a layout constraint from among a plurality of layout constraints based on a path or a shape of the identified first user gesture, the selected layout constraint controlling a manner in which elements of the diagram are arranged;

applying the selected layout constraint to the selected first subset of elements to generate a first layout of the diagram;

identifying a second user gesture and subsequently selecting the first subset of elements, and a second subset of elements from among the plurality of different elements of the diagram, based on the identified second user gesture; and

applying the selected layout constraint to the selected second subset of elements based on the identified second user gesture to generate a second layout of the diagram.

2. The method of claim 1, further comprising at least one of:

rendering at least a portion of the diagram in the first layout to the display device;

or

rendering at least a portion of the diagram in the second layout to the display device.

3. The method of claim 1, wherein identifying the first user gesture or identifying the second user gesture comprises:

determining if the first user gesture or the second user gesture respectively corresponds to a user gesture in a library of predefined user gestures.

4. The method of claim 1, wherein at least one of the first user gesture or the second user gesture comprises motions of a user in a space.

5. The method of claim 4, wherein identifying the first user gesture or identifying the second user gesture comprises identifying the first user gesture or identifying the second user gesture respectively based on input received via at least one of a video capture device or one or more motion tracking sensors.

6. The method of claim 1, wherein applying the selected layout constraint to the selected second subset of elements comprises merging the second subset of elements with the selected first subset of elements according to the identified layout constraint.

7. The method of claim 1, wherein selecting the first subset of elements comprises selecting elements of the diagram that are enclosed within a shape defined by the user gesture.

8. The method of claim 7, wherein selecting the layout constraint comprises selecting one of a horizontal alignment constraint, a vertical alignment constraint, a diagonal alignment constraint, a circular alignment constraint, or a grouping constraint.

9. A computer-readable storage device storing computer-executable instructions that, when executed by a processor-based computing system, cause the processor-based computing system to perform a method, the method comprising:

rendering at least a portion of a diagram to a display device coupled to the computing system, the diagram comprising a plurality of elements;

identifying a user gesture based on input received via at least one user input device coupled to the computing system, the user gesture comprising one of a touch gesture or a tracked spatial motion of a user;

selecting, subsequent to said identifying, one or more first elements of the diagram and one or more second elements of the diagram based on the identified user gesture;

identifying a layout constraint associated with the selected first element(s) from among a plurality of layout constraints, the identified layout constraint controlling a manner in which the selected first element(s) are arranged when rendered to the display device;

applying the identified layout constraint to the selected second elements(s) based on the identified user gesture; and

rendering at least a portion of the diagram to the display device after application of the identified layout constraint to the selected second element(s).

10. The computer-readable storage device of claim 9, wherein the identified layout constraint is based on a path or a shape of a prior user gesture associated with the selected first element(s).

11. The computer-readable storage device of claim 9, wherein identifying the user gesture based on input received via the at least one user input device comprises identifying the user gesture based on input received via at least one of a video capture device or one or more motion tracking sensors.

12. The computer-readable storage device of claim 9, wherein applying the identified layout constraint to the second selected elements(s) comprises merging the second selected elements(s) with the first selected elements(s) according to the identified layout constraint.

13. The computer-readable storage device of claim 9, wherein applying the identified layout constraint to the selected second elements(s) is performed on the selected second elements(s) separately from the selected first element(s).

14. The computer-readable storage device of claim 9, wherein the identified layout constraint comprises one of a horizontal alignment constraint, a vertical alignment constraint, a diagonal alignment constraint, a circular alignment constraint, or a grouping alignment constraint.

15. A method implemented in a processor-based computing system, comprising:

rendering at least a portion of a diagram to a display device coupled to the computing system, the diagram comprising a plurality of elements;

identifying a user gesture based on input received via at least one user input device coupled to the computing system, the user gesture comprising one of a touch gesture or a tracked spatial motion of a user;

selecting, subsequent to said identifying, one or more first elements of the diagram and one or more second elements of the diagram based on the identified user gesture;

identifying a layout constraint associated with the selected first element(s) from among a plurality of layout constraints, the identified layout constraint controlling a manner in which the selected first element(s) are arranged when rendered to the display device;

applying the identified layout constraint to the selected second elements(s) based on the identified user gesture; and

rendering at least a portion of the diagram to the display device after application of the identified layout constraint to the selected second element(s).

16. The method of claim 15, wherein the identified layout constraint is based on a path or a shape of a prior user gesture associated with the selected first element(s).

17. The method of claim 15, wherein identifying the user gesture based on input received via the at least one user input device comprises identifying the user gesture based on input received via at least one of a video capture device or one or more motion tracking sensors.

18. The method of claim 15, wherein applying the identified layout constraint to the selected second elements(s) comprises merging the selected second elements(s) with the selected first elements(s) according to the identified layout constraint.

19. The method of claim 15, wherein applying the identified layout constraint to the selected second elements(s) is performed on the selected second elements(s) separately from the selected first element(s).

20. The method of claim 15, wherein the identified layout constraint comprises one of a horizontal alignment constraint, a vertical layout constraint, a diagonal alignment constraint, a circular alignment constraint, or a grouping alignment constraint.