Sketch beautification and completion of partial structured-drawings

• US 9,098,191 B2
• Filed: 01/24/2012
• Issued: 08/04/2015
• Est. Priority Date: 01/24/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Gulwani, Sumit, Cheema, Salman Shaukat, LaViola, Joseph J. Jr.
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): VANCHY JR, MICHAEL J
• Application Number:
  US13/357,612
• Publication Number:
  US 20130188877A1
• Time in Patent Office: 1,288 Days
• Field of Search: 382/181, 382/182, 382/186, 382/187, 382/195, 382/202, 382/203
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A sketch processing system is described herein for assisting a user in producing a drawing. In one implementation, the sketch processing system operates by: receiving ink strokes in response to creation of an original drawing; recognizing components and geometric constraints within the original drawing, to produce a recognized drawing; beautifying the original drawing by modifying at least one aspect of the recognized drawing in accordance with the recognized constraints, to produce a beautified drawing; and recognizing a recurring pattern in the beautified pattern (if any) and using that pattern to produce at least one added component to the beautified drawing.

Claims

1. A method, performed by computing functionality, for assisting a user in producing a drawing comprising:

receiving ink strokes in response to creation of an original drawing;

recognizing components within the original drawing;

recognizing constraints associated with the components that have been recognized, the components and the constraints yielding a recognized drawing;

storing the components and the constraints in a data store;

revising at least one aspect of the recognized drawing in accordance with the constraints to produce a beautified drawing;

identifying a transformation between each pair of objects in the beautified drawing to produce transformation information, wherein the transformation is based on intercept information;

identifying one or more predominant transformations in the transformation information that correspond to a predominant recurring pattern in the beautified drawing;

adding an additional component in the beautified drawing based on the predominant recurring pattern; and

providing an output drawing which is based on the beautified drawing.

2. The method of claim 1, wherein the components include at least one of:

one or more lines; and

one or more circles.

3. The method of claim 1, wherein each of said recognizing of components and said recognizing of constraints is based on at least one data store of rules that is both extensible and customizable.

4. The method of claim 1, wherein at least one constraint expresses a relationship between at least two components.

5. The method of claim 1, further comprising:

displaying information regarding the constraints; and

receiving at least one modification to the constraints from a user, to provide at least one explicit constraint.

6. The method of claim 1, further comprising recognizing a marking that appears in the original drawing, the marking being associated with at least one explicit constraint.

7. The method of claim 1, wherein identifying one or more predominant transformations comprises using a voting technique to identify one or more predominant transformations in the transformation information.

8. The method of claim 1, wherein said producing of the beautified drawing comprises parsing the components into a set of unresolved sub-components, each component being made up of two or more unresolved sub-components.

9. The method of claim 8, wherein said producing the beautified drawing further comprises iteratively performing operations of:

determining whether an unresolved sub-component selected from the set of unresolved sub-components can be resolved based on a set of previously resolved sub-components, the constraints, and a data store of rules; and

if the unresolved sub-component can be resolved, producing a corresponding resolved sub-component and adding the resolved sub-component to the set of previously resolved sub-components.

10. The method of claim 9, wherein, if an unresolved sub-component cannot be found that is able to be resolved, further comprising:

identifying an unresolved sub-component that is deemed most appropriate for resolution, to provide a selected sub-component, the selected sub-component being associated with a parent component that is assessed as being most appropriate for resolution; and

resolving the selected sub-component by making reference to drawing information in the original drawing, and updating the set of resolved sub-components based on said resolving of the selected sub-component.

11. The method of claim 10, wherein said identifying employs a ranking function that assigns a rank to each unresolved candidate sub-component based on:

a weight assigned to a type of the candidate sub-component;

a canonical order of a parent component to which the candidate sub-component belongs; and

a weighted ratio of a number of resolved sub-components associated with the parent component to which the candidate sub-component belongs.

12. The method of claim 1, further comprising requesting user input to guide application of the predominant recurring pattern in the beautified drawing.

13. The method of claim 12, wherein adding an additional component in the beautified drawing comprises adding an additional component in the beautified drawing based on the predominant recurring pattern and the user input.

14. The method of claim 13, wherein each pair of objects comprises a pair of sub-components in the beautified drawing.

15. The method of claim 13, wherein said at least one prevalent transformation corresponds to two or more prevalent transformations.

16. The method of claim 13, wherein the transformation between each pair of objects is an affine transformation.

17. A device comprising:

a processor; and

executable instructions operable by the processor, the executable instructions comprising a method for assisting a user in producing a drawing, the method comprising;

receiving ink strokes in response to creation of an original drawing;

recognizing components and constraints within the original drawing, the components and the constraints yielding a recognized drawing;

decomposing each component in the recognized drawing into two or more sub-components;

successively resolving unresolved sub-components in the recognized drawing to produce a beautified drawing;

identifying transformations between different pairs of sub-components in the beautified drawing to produce transformation information, wherein the transformations are based on slope information;

identifying one or more predominant transformations in the transformation information that correspond to a predominant recurring pattern in the beautified drawing; and

adding at least one additional component to the beautified drawing based on the predominant recurring pattern.

18. The device of claim 17, wherein successively resolving unresolved sub-components in the recognized drawing comprises ranking the unresolved sub-components and successively selecting the unresolved sub-components for resolution based on the rankings.

19. A method, performed by computing functionality, for assisting a user in producing a drawing comprising:

receiving ink strokes in response to creation of an original drawing;

recognizing components and constraints within the original drawing, the components and the constraints yielding a recognized drawing;

decomposing each component in the recognized drawing into two or more sub-components;

ranking unresolved sub-components in the recognized drawing;

selecting one or more unresolved sub-components for resolution based on the rankings;

resolving the selected sub-components in the recognized drawing to produce a beautified drawing;

identifying transformations between different pairs of sub-components in the beautified drawing to produce transformation information, wherein the transformations are based on intercept information;

identifying one or more predominant transformations in the transformation information that correspond to a predominant recurring pattern in the beautified drawing; and

adding at least one additional component to the beautified drawing based on the predominant recurring pattern.

20. The method of claim 19, wherein identifying one or more predominant transformations comprises using a voting technique to identify one or more predominant transformations in the transformation information.