Recognition of user drawn graphical objects based on detected regions within a coordinate-plane

• US 10,372,321 B2
• Filed: 04/18/2016
• Issued: 08/06/2019
• Est. Priority Date: 12/30/2013
• Status: Active Grant

• Current Assignee: Skribb.it Inc.
• Original Assignee: Skribb.it Inc.
• Inventors: Le, Vietlong
• Primary Examiner(s): Bycer, Eric J.
• Application Number:
  US15/131,521
• Publication Number:
  US 20160231924A1
• Time in Patent Office: 1,205 Days
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 11/80 Creating or modifying a manually drawn or painted image using a manual input device, e.g. mouse, light pen, direction keys on keyboard
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/68 of symmetry

Abstract

A system, method, and apparatus for managing the drawing of graphical objects are disclosed. An example apparatus includes an interface configured to receive at least one gesture from a user in relation to a client device. The apparatus also includes a coordinate detector configured to determine a coordinate-plane for the at least one gesture and determine coordinates of points along the at least one gesture. The apparatus further includes an object detector configured to determine a graphical object based on the coordinates of points corresponding to the at least one gesture and an object manager configured to cause the graphical object to be displayed by the client device. The graphical object includes a formalized editable version of the at least one gesture in a format native to an application selected to receive the graphical object.

Claims

1. An apparatus comprising:

an interface of a client device configured to detect at least one gesture from a user;

a coordinate detector configured to;

determine a rectangular-shaped coordinate-plane with dimensions substantially equal to a width and a height of the at least one gesture,

determine at least four moat regions along a perimeter of the coordinate-plane, and

determine coordinates of points along the at least one gesture;

an object detector configured to;

determine whether a threshold number of the coordinates are within each of the moat regions, the threshold number including at least 25% of a length of the respective moat region,

for each moat region, designate the moat region as a detected moat region responsive to determining that the threshold number of coordinates is reached for the respective moat region and designate the moat region as an undetected moat region responsive to determining that the threshold number of coordinates is not reached for the respective moat region,

determine a first set of graphical object types that correspond to graphical object types which include the number of detected moat regions, and

determine a second set of graphical object types that correspond to graphical object types which include at least the number of detected moat regions and additionally include at least one undetected moat region; and

an object manager configured to cause the first set of graphical object types and the second set of graphical object types to be displayed by the client device for selection by the user for creation of a formalized editable version of the selected graphical object type.

2. The apparatus of claim 1, wherein the object detector is configured to determine the first set of graphical object types includes:

at least one of a square and a rectangle responsive to determining there are four detected moat regions,

a U-shape responsive to determining there are three detected moat regions,

at least one of an L-shape, a triangle, and a quarter-circle responsive to determining there are two detected moat regions,

at least one of a British trapezium and a triangle responsive to determining there is one detected moat region, and

a circular object responsive to determining there are zero detected moat regions.

3. The apparatus of claim 2, wherein the object detector is configured to determine the second set of graphical object types includes:

at least one of a square and a rectangle responsive to determining there are three detected moat regions and at least one undetected moat region,

a U-shape responsive to determining there are two detected moat regions and at least one undetected moat region,

at least one of an L-shape, a triangle, and a quarter-circle responsive to determining there is one detected moat region and at least one undetected moat region,

at least one of a British trapezium and a triangle responsive to determining there are zero detected moat regions and at least one undetected moat region.

4. The apparatus of claim 2, wherein the object detector is configured to, responsive to determining there are two detected moat regions:

determine whether a predetermined number of the coordinates of points along the at least one gesture are located within a center portion of the coordinate-plane;

determine the first set of graphical object types includes the L-shape responsive to determining that a number of the coordinates of points along the at least one gesture are less than the predetermined number; and

determine the first set of graphical object types includes the triangle responsive to determining that the number of the coordinates of points along the at least one gesture are greater than or equal to the predetermined number.

5. The apparatus of claim 2, wherein the object detector is configured to, responsive to determining there are two detected moat regions:

determine whether the coordinates of points along the at least one gesture include coordinates that connect first and last ends of the at least one gesture together to form an apex; and

determine that the first set of graphical object types includes the British trapezium responsive to determining that the first and the last ends of the at least one gesture do not form the apex.

6. The apparatus of claim 5, wherein the apex includes a furthest point from the single detected moat region.

7. The apparatus of claim 2, wherein the object detector is configured to, responsive to determining there are zero detected moat regions:

create an analysis ellipse that includes a predetermined width within the coordinate-plane and a predetermined height within the coordinate-plane; and

determine the circular object includes;

a circle responsive to determining (i) that at least a predetermined number of the coordinates are outside of the analysis ellipse but within the coordinate-plane, and (ii) the width of the at least one gesture is within 10% of the height of the at least one gesture,

an ellipse responsive to determining (i) that at least the predetermined number of coordinates are outside of the analysis ellipse but within the coordinate-plane, and (ii) the width of the at least one gesture is greater than 10% of the height of the at least one gesture, and

a cloud responsive to determining (i) that less than the predetermined number of coordinates are outside of the analysis ellipse, and (ii) the at least one gesture includes at least four waves, wherein

a wave is determined by (i) determining a distance between each of the coordinates and a center of the analysis ellipse, (ii) determining local minimum points among the coordinates, and (iii) designating the wave as the coordinates that are located between two adjacent local minimum points.

8. The apparatus of claim 1, wherein the interface includes at least one of (i) a touchscreen interface, (ii) a free-space movement detection device configured to detect movement of at least one of a hand, a wand, and a glove within a defined three-dimensional space, (iii) a digitally enabled wall, a digitally enabled whiteboard, or a digitally enabled window, (iv) a mouse or a movement pad, and (v) an eye movement tracker.

9. An apparatus comprising:

an interface of a client device configured to detect at least one gesture from a user;

a coordinate detector configured to;

determine a rectangular-shaped coordinate-plane with dimensions substantially equal to a width and a height of the at least one gesture,

determine at least four moat regions along a perimeter of the coordinate-plane, and

determine coordinates of points along the at least one gesture;

an object detector configured to;

determine whether a threshold number of the coordinates are within each of the moat regions, the threshold number including at least 25% of a length of the respective moat region,

for each moat region, designate the moat region as a detected moat region responsive to determining that the threshold number of coordinates is reached for the respective moat region and designate the moat region as an undetected moat region responsive to determining that the threshold number of coordinates is not reached for the respective moat region, and

determine a graphical object type that corresponds to a graphic object type which includes the number of detected moat regions; and

an object manager configured to display on the client device a formalized editable version of the determined graphical object type.

10. The apparatus of claim 9, wherein the object detector is configured to determine the graphical object type includes:

at least one of a square and a rectangle responsive to determining there are four detected moat regions,

a U-shape responsive to determining there are three detected moat regions,

at least one of an L-shape and a triangle responsive to determining there are two detected moat regions,

at least one of a British trapezium and a triangle responsive to determining there is one detected moat region, and

a circular object responsive to determining there are zero detected moat regions.

11. The apparatus of claim 10, wherein the object detector is configured to determine a size ratio of lengths of two of the adjacent moat regions conditioned upon detecting the four moat regions;

determine the first set of graphical object types includes a square responsive to determining that the size ratio is at least a predetermined percentage of 90%; and

determine the first set of graphical object types includes a rectangle responsive to determining that the size ratio is less than the predetermined percentage of 90%.

12. The apparatus of claim 11, wherein the predetermined percentage is configurable by a user.

13. The apparatus of claim 10, wherein the object detector is configured to determine whether a size of a length is within a predetermined percentage of 10% of a size of a width of the at least one gesture conditioned upon detecting the four moat regions;

determine the first set of graphical object types includes a square responsive to determining that the size of the length is within the predetermined percentage of 10% of the size of the width of the at least one gesture; and

determine the first set of graphical object types includes a rectangle responsive to determining that the size of the length is greater than 10% of the size of the width of the at least one gesture.

14. The apparatus of claim 13, wherein the predetermined percentage is configurable by a user.

15. An apparatus comprising:

an interface of a client device configured to detect a gesture from a user;

a coordinate detector configured to;

determine coordinates of points along the gesture, and

create an analysis ellipse that includes a predetermined width and a predetermined height; and

an object detector configured to determine a graphical object type includes;

a circle responsive to determining (i) that at least a predetermined number of the coordinates are outside of the analysis ellipse, and (ii) the width of the gesture is within a predetermined percentage of 10% of the height of the gesture,

an ellipse responsive to determining (i) that at least the predetermined number of coordinates are outside of the analysis ellipse, and (ii) the width of the gesture is greater than the predetermined percentage of 10% of the height of the gesture, and

a cloud responsive to determining (i) that less than the predetermined number of coordinates are outside of the analysis ellipse, and (ii) the gesture includes at least four waves; and

an object manager configured to display on the client device a formalized editable version of the determined graphical object type.

16. The apparatus of claim 15, wherein the predetermined percentage is configurable by a user.

17. The apparatus of claim 15, wherein the object detector is configured to determine a wave by (i) determining a distance between each of the coordinates and a center of the analysis ellipse, (ii) determining local minimum points among the coordinates, and (iii) designating the wave as the coordinates that are located between two adjacent local minimum points.

18. The apparatus of claim 15, wherein the object detector is configured to determine other graphical object types that would fit within the analysis ellipse and the object manager is configured to display a list of the other graphical object types that would fit within the analysis ellipse.

19. The apparatus of claim 18, wherein the object manager is configured to:

receive a selection of one of the other graphical object types; and

replace the display of the formalized editable version of the determined graphical object type with a formalized editable version of the selected other graphical object type.

20. The apparatus of claim 15, wherein the object detector is configured to determine that at least one of an ellipse, cloud, amoeba, star polygon, speech bubble, thought bubble, and a polygon would fit within the analysis ellipse.