SKETCH BEAUTIFICATION AND COMPLETION OF PARTIAL STRUCTURED-DRAWINGS

• US 20130188877A1
• Filed: 01/24/2012
• Published: 07/25/2013
• Est. Priority Date: 01/24/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Gulwani, Sumit, Cheema, Salman Shaukat, LaViola, Joseph J. Jr.
• Granted Patent: US 9,098,191 B2
• Time in Patent Office: Days
• US Class Current: 382/203
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A sketch processing system is described herein for assisting a user in producing a drawing. In one implementation, the sketch processing system operates by: receiving ink strokes in response to creation of an original drawing; recognizing components and geometric constraints within the original drawing, to produce a recognized drawing; beautifying the original drawing by modifying at least one aspect of the recognized drawing in accordance with the recognized constraints, to produce a beautified drawing; and recognizing a recurring pattern in the beautified pattern (if any) and using that pattern to produce at least one added component to the beautified drawing.

Claims

1. A method, performed by computing functionality, for assisting a user in producing a drawing:

receiving ink strokes in response to creation of an original drawing;

recognizing components within the original drawing;

recognizing constraints associated with the components that have been recognized, the components and the constraints yielding a recognized drawing;

storing the components and the constraints in a data store; and

providing an output drawing which is based on the recognized drawing,

where said receiving, recognizing components, recognizing constraints, storing, and providing are performed by the computing functionality.

2. The method of claim 1, wherein the components include at least one of:

one or more lines; and

one or more circles.

3. The method of claim 1, wherein each of said recognizing of components and said recognizing of constraints is based on at least one data store of rules that is both extensible and customizable.

4. The method of claim 1, wherein at least one constraint expresses a relationship between at least two components.

5. The method of claim 1, further comprising:

displaying information regarding the constraints; and

receiving at least one modification to the constraints from a user, to provide at least one explicit constraint.

6. The method of claim 1, further comprising recognizing a marking that appears in the original drawing, the marking being associated with at least one explicit constraint.

7. The method of claim 1, further comprising producing a beautified drawing by revising at least one aspect of the recognized drawing in accordance with the constraints.

8. The method of claim 7, wherein said producing of the beautified drawing comprises parsing the components into a set of unresolved sub-components, each component being made up of two or more unresolved sub-components.

9. The method of claim 8, wherein said producing the beautified drawing further comprises iteratively performing operations of:

determining whether an unresolved sub-component selected from the set of unresolved sub-components can be resolved based on a set of previously resolved sub-components, the constraints, and a data store of rules; and

if the unresolved sub-component can be resolved, producing a corresponding resolved sub-component and adding the resolved sub-component to the set of previously resolved sub-components.

10. The method of claim 9, wherein, if an unresolved sub-component cannot be found that is able to be resolved, further comprising:

identifying an unresolved sub-component that is deemed most appropriate for resolution, to provide a selected sub-component, the selected sub-component being associated with a parent component that is assessed as being most appropriate for resolution; and

resolving the selected sub-component by making reference to drawing information in the original drawing, and updating the set of resolved sub-components based on said resolving of the selected sub-component.

11. The method of claim 10, wherein said identifying employs a ranking function that assigns a rank to each unresolved candidate sub-component based on:

a weight assigned to a type of the candidate sub-component;

a canonical order of a parent component to which the candidate sub-component belongs; and

a weighted ratio of a number of resolved sub-components associated with the parent component to which the candidate sub-component belongs.

12. The method of claim 7, further comprising:

recognizing a pattern in the beautified drawing, to provide a recognized pattern; and

producing at least one additional component in the beautified drawing based on the pattern.

13. The method of claim 12, wherein said recognizing of the pattern comprises:

determining a transformation between each pair of objects in the beautified drawing, to produce transformation information; and

determining at least one prevalent transformation in the transformation information using a voting technique, said at least one prevalent transformation being associated with the recognized pattern.

14. The method of claim 13, wherein each pair of objects comprises a pair of sub-components in the beautified drawing.

15. The method of claim 13, wherein said at least one prevalent transformation corresponds to two or more prevalent transformations.

16. The method of claim 13, wherein the transformation between each pair of objects is an affine transformation.

17. A computer readable storage medium for storing computer readable instructions, the computer readable instructions providing a beautification module when executed by one or more processing devices, the computer readable instructions comprising:

logic configured to receive a recognized drawing, the recognized drawing being populated with recognized components and constraints, each component in the recognized drawing being associated with two or more sub-components; and

logic configured to beautify the recognized drawing to produce a beautified drawing by successively resolving unresolved sub-components in the recognized drawing,

said successively resolving being based on a set of previously resolved sub-components, the constraints, and a data store of rules.

18. The computer readable storage medium of claim 17, further comprising:

logic configured identify an unresolved sub-component that is deemed most appropriate for resolution, in the event that no sub-component can be found that can be resolved, to provide a selected sub-component; and

logic configured to resolve the selected sub-component by making reference to drawing information in an original drawing.

19. A pattern processing module, implementing by computing functionality, for processing a pattern in a drawing, comprising:

logic configured to receive a drawing having a plurality of components, each component, in turn, having two or more sub-components;

logic configured to determine a transformation between each pair of sub-components in the drawing, to produce transformation information; and

logic configured to determine at least one prevalent transformation in the transformation information using a voting technique, said at least one prevalent transformation being associated with a recognized pattern.

20. The pattern processing module of claim 19, further comprising logic configured to produce at least one additional component in the beautified drawing based on the recognized pattern.