Recognition of user drawn graphical objects based on detected regions within a coordinate-plane

• US 9,317,937 B2
• Filed: 12/30/2014
• Issued: 04/19/2016
• Est. Priority Date: 12/30/2013
• Status: Active Grant

• Current Assignee: Skribb.it Inc.
• Original Assignee: Skribb.it Inc.
• Inventors: Le, Vietlong
• Primary Examiner(s): Bycer, Eric J
• Application Number:
  US14/586,168
• Publication Number:
  US 20150205398A1
• Time in Patent Office: 476 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 11/80 Creating or modifying a manually drawn or painted image using a manual input device, e.g. mouse, light pen, direction keys on keyboard
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/68 of symmetry

Abstract

A system, method, and apparatus for managing the drawing of graphical objects are disclosed. An example apparatus includes a touchscreen interface configured to receive at least one gesture from a user on a touchscreen of a client device. The apparatus also includes a coordinate detector configured to determine a coordinate-plane for the at least one gesture and determine coordinates of points along the at least one gesture. The apparatus further includes an object detector configured to determine a graphical object based on the coordinates of points corresponding to the at least one gesture and an object manager configured to cause the graphical object to be displayed by the client device. The graphical object includes a formalized editable version of the at least one gesture in a format native to an application selected to receive the graphical object.

Claims

1. An apparatus comprising:

a touchscreen interface configured to receive at least one gesture from a user on a touchscreen of a client device;

a coordinate detector configured to;

determine a coordinate-plane with dimensions substantially equal to a width and a height of the at least one gesture,

determine at least four rectangular-shaped moat regions along a perimeter of the coordinate-plane, and

determine coordinates of points along the at least one gesture;

an object detector configured to;

determine whether a threshold number of the coordinates are within each of the moat regions,

designate each moat region as a detected moat region responsive to determining that the threshold number of coordinates is reached for the respective moat region, and

determine a graphical object based on the number of detected moat regions; and

an object manager configured to cause the graphical object to be displayed by the client device,

wherein the object detector is configured to determine the graphical object includes;

at least one of a square and a rectangle responsive to determining there are four detected moat regions,

a U-shape responsive to determining there are three detected moat regions,

at least one of an L-shape and a triangle responsive to determining there are two detected moat regions,

at least one of a British trapezium and a triangle responsive to determining there is one detected moat region, and

a circular object responsive to determining there are zero detected moat regions.

2. The apparatus of claim 1, wherein the object detector is configured to:

determine an analysis ellipse that includes a predetermined width within the coordinate-plane and a predetermined height within the coordinate-plane; and

determine the circular object includes;

a circle responsive to determining (i) that at least a predetermined number of the coordinates are outside of the analysis ellipse but within the coordinate-plane, and (ii) the width of the gesture is within 10% of the height of the gesture,

an ellipse responsive to determining (i) that at least the predetermined number of coordinates are outside of the analysis ellipse but within the coordinate-plane, and (ii) the width of the gesture is greater than 10% of the height of the gesture, and

a cloud responsive to determining (i) that less than the predetermined number of coordinates are outside of the analysis ellipse, and (ii) the gesture includes at least four waves.

3. The apparatus of claim 2, wherein the object detector is configured to determine a wave by:

determining a distance between each of the coordinates and a center of the analysis ellipse;

determining local minimum points among the coordinates; and

designating the wave as the coordinates that are located between two adjacent local minimum points.

4. The apparatus of claim 1, wherein the coordinate detector is configured to:

determine a greatest width of the at least one gesture;

determine a greatest height of the at least one gesture; and

set a width of the coordinate-plane to substantially match the greatest width and set a height of the coordinate-plane to substantially match the greatest height.

5. The apparatus of claim 1, further comprising an object exporter configured to create a native version of the graphical object for export to a corresponding application.

6. The apparatus of claim 1, further comprising an object exporter configured to:

determine a native version of the graphical object is unsupported for an application specified to receive the graphical object;

responsive to determining the native version of the graphical object is not supported, create one or more line segments as an editable object compatible with the receiving application, the editable object configured to visually resemble the graphical object; and

provide the editable object to the application.

7. The apparatus of claim 1, wherein the object manager is configured to cause the graphical object to be displayed by:

selecting an object definition corresponding to the graphical object, the object definition including at least one property specifying how the graphical object is to be displayed;

setting the at least one property of the graphical object based on the coordinates of points; and

rendering the graphical object for display.

8. The apparatus of claim 7, wherein the at least one property includes a size, an orientation, and a position of the graphical object relative to the touchscreen.

9. An apparatus comprising:

a touchscreen interface configured to receive a partial gesture from a user on a touchscreen of a client device;

a coordinate detector configured to;

determine a rectangular-shaped coordinate-plane with dimensions substantially equal to a width and a height of the partial gesture,

determine at least four moat regions along a perimeter of the coordinate-plane, and

determine coordinates of points along the partial gesture;

an object predictor configured to;

determine whether a threshold number of the coordinates are within each of the moat regions, the threshold number related to at least 25% of a length of the respective moat region,

designate each moat region as a detected moat region responsive to determining that the threshold number of coordinates is reached for the respective moat region, and

predict a graphical object based on the number of detected moat regions; and

an object manager configured to cause the graphical object to be displayed by the client device.

10. The apparatus of claim 9, wherein the object predictor is configured to:

determine the partial gesture corresponds to at least two possible graphical objects; and

provide a prompt to the user via a display of the client device to select one of the at least two possible graphical objects.

11. The apparatus of claim 9, wherein the object predictor is configured to:

compare the partial gesture to previous partial gestures;

determine the partial gesture matches one of the previous partial gestures; and

determine the graphical object by identifying a graphical object type that corresponds to the matching previous partial gesture.

12. The apparatus of claim 9, wherein:

the touchscreen interface is configured to receive a second partial gesture from the user on the touchscreen of the client device;

the coordinate detector is configured to;

determine a second coordinate-plane for the second partial gesture, and

determine second coordinates of points along the second partial gesture;

the object predictor is configured to determine a second graphical object based on the second coordinates of points corresponding to the second partial gesture; and

the object manager is configured to;

determine the second partial gesture is within the graphical object,

nest the second graphical object within the graphical object, and

cause the second graphical object to be displayed within the graphical object by the client device.

13. The apparatus of claim 9, wherein the partial gesture corresponds to less than a complete gesture of the graphical object that the user is attempting to draw.

14. A method comprising:

prompting a user, via a processor, to draw an object on a touchscreen of a client device;

receiving at least one gesture from the user on the touchscreen;

applying, via the processor, a coordinate-plane to the at least one gesture, the coordinate-plane including at least two moat regions along a perimeter of the coordinate-plane;

for each of the moat regions;

(i) determine coordinates of points along the at least one gesture,

(ii) determine whether a threshold number of the coordinates are within each of the moat regions, the threshold number related to at least 25% of a length of the respective moat region, and

(iii) designate each moat region as a detected moat region responsive to determining that the threshold number of coordinates is reached for the respective moat region;

determining, via the processor, a graphical object based on a number of detected moat regions; and

causing, via the processor, the graphical object to be displayed by the client device.

15. The method of claim 14, wherein the graphical object is a formalized editable version of the at least one gesture provided by the user, the graphical object being approximately the same size as the at least one gesture.

16. The method of claim 14, wherein the coordinate-plane has a rectangular shape with dimensions substantially equal to a greatest width and a greatest height of the at least one gesture.

17. The method of claim 14, wherein the coordinate-plane includes four rectangular shaped moat regions, one moat region positioned along each side of the coordinate-plane.

18. The method of claim 14, further comprising exporting the object to an application in a format native to the application.

19. The method of claim 14, further comprising:

determining, via the processor, a context in which the at least one gesture was provided; and

determining the graphical object based on the context in addition to the number of detected moat regions.

20. A method comprising:

prompting a user, via a processor, to draw an object on a touchscreen of a client device;

receiving at least one gesture from the user on the touchscreen;

applying, via the processor, a coordinate-plane to the at least one gesture, the coordinate-plane including four rectangular shaped moat regions, one moat region positioned along each side of the coordinate-plane;

for each of the moat regions;

(i) determine coordinates of points along the at least one gesture,

(ii) determine whether a threshold number of the coordinates are within each of the moat regions, and

(iii) designate each moat region as a detected moat region responsive to determining that the threshold number of coordinates is reached for the respective moat region;

determining, via the processor, a graphical object based on a number of detected moat regions; and

causing, via the processor, the graphical object to be displayed by the client device, wherein;

the detection of four moat regions causes at least one of a square and a rectangle to be displayed on the client device;

the detection of three moat regions causes a U-shape to be displayed on the client device;

the detection of two moat regions causes at least one of an L-shape and a triangle to be displayed on the client device;

the detection of one moat region causes at least one of a British trapezium and a triangle to be displayed on the client device; and

the detection of zero moat regions causes at least one of a circle, ellipse, and cloud to be displayed on the client device.

21. The method of claim 20, further comprising:

determining, via the processor, an analysis ellipse that includes a predetermined width within the coordinate-plane and a predetermined height within the coordinate-plane;

causing, via the processor, the circle to be displayed responsive to determining (i) that at least a predetermined number of the coordinates are outside of the analysis ellipse but within the coordinate-plane, and (ii) the width of the gesture is within 10% of the height of the gesture;

causing, via the processor, the ellipse to be displayed responsive to determining (i) that at least the predetermined number of coordinates are outside of the analysis ellipse but within the coordinate-plane, and (ii) the width of the gesture is greater than 10% of the height of the gesture; and

causing, via the processor, the cloud to be displayed responsive to determining (i) that less than the predetermined number of coordinates are outside of the analysis ellipse, and (ii) the gesture includes at least four waves.

22. The method of claim 21, wherein the processor is to determine a wave by:

determining a distance between each of the coordinates and a center of the analysis ellipse;

determining local minimum points among the coordinates; and

designating the wave as the coordinates that are located between two adjacent local minimum points.

23. A non-transitory machine-accessible device having instructions stored thereon that, when executed, cause a machine to at least:

prompt a user to draw a graphical object on a touchscreen of a client device;

for each touch gesture received on the touchscreen;

(a) determine coordinates for the touch gesture including a first coordinate corresponding to the start of the touch gesture and a last coordinate corresponding to an end of the touch gesture;

(b) determine a length of a moat region based on the first coordinate and the last coordinate, the moat region have a predetermined width;

(c) determine whether a threshold number of coordinates are within the moat region having dimensions of the determined length and the predetermined width, the threshold number related to at least 25% of a length of the moat region;

(d) designate the moat region as a detected moat region responsive to determining that the threshold number of coordinates is reached;

determine a graphical object based on a number of detected moat regions; and

cause the graphical object to be displayed on the client device as a formalized editable version of the at least one touch gesture in a format native to an application selected to receive the graphical object.