METHOD AND TOOL FOR RECOGNIZING A HAND-DRAWN TABLE

• US 20090245645A1
• Filed: 03/28/2008
• Published: 10/01/2009
• Est. Priority Date: 03/28/2008
• Status: Active Grant

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: XING, ZHAOHUI
• Granted Patent: US 8,634,645 B2
• Time in Patent Office: Days
• US Class Current: 382/189
• CPC Class Codes: G06F 40/177 of tables; using ruled lines
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

A computerized method of recognizing an input hand-drawn table formed by freeform line objects comprises transforming each freeform line object of the table into one of a vertical line segment and a horizontal line segment, generating a grid system based on the vertical and horizontal line segments and converting the generated grid system into a table object.

Claims

1. A computerized method of recognizing an input hand-drawn table formed by freeform line objects, said method comprising:

transforming each freeform line object of said table into one of a vertical line segment and a horizontal line segment;

generating a grid system based on the vertical and horizontal line segments; and

converting the generated grid system into a table object.

2. The method of claim 1 wherein said transforming comprises for each freeform line object:

examining the orientation of the freeform line object; and

creating one of the horizontal and vertical line segment representing the freeform line object based on the orientation thereof.

3. The method of claim 2 wherein said transforming further comprises for each freeform line object:

determining the start point and end point of the freeform line object;

generating a temporary line segment extending between the determined start point and end point;

determining the orientation of said temporary line segment; and

based on the determined orientation, creating one of said horizontal line segment and vertical line segment.

4. The method of claim 3 wherein said transforming further comprises:

for each created horizontal line segment, determining whether it overlaps with an existing horizontal line segment and, if so, merging the created and existing horizontal line segments; and

for each created vertical line segment, determining whether it overlaps with an existing vertical line segment and, if so, merging the created and existing vertical line segments.

5. The method of claim 4 wherein said transforming further comprises:

comparing each created horizontal line segment with a horizontal line segment list to determine if it overlaps with an existing horizontal line segment;

if not, adding the created horizontal line segment to the list and, if so, performing the merging;

comparing each created vertical line segment with a vertical line segment list to determine if it overlaps with an existing vertical line segment; and

if not, adding the created vertical line segment to the list and, if so, performing the merging.

6. The method of claim 5 wherein said transforming further comprises, prior to said comparing to determine overlap:

examining each horizontal line segment to determine if it is within a threshold vertical distance of an existing horizontal line segment in the horizontal line segment list;

if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the horizontal line segment to the list; and

examining each vertical line segment to determine if it is within a threshold horizontal distance of an existing vertical line segment in the vertical line segment list; and

if so, performing said comparing to determine overlap and, if not, bypassing the overlap comparing and adding the vertical line segment to the list.

7. The method of claim 1 wherein said grid system is generated based on deemed intersections of the horizontal and vertical line segments.

8. The method of claim 7 wherein said grid system generating comprises:

selecting pairs of adjacent vertical and adjacent horizontal line segments; and

for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, determining the intersection points therebetween thereby to yield four corner coordinates of each grid in the grid system.

9. The method of claim 8 wherein said grid system generating comprises:

for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, arranging the vertical and horizontal line segments into different combinations, each combination comprising one vertical line segment and one horizontal line segment; and

for each combination, calculating the intersection point between the vertical and horizontal line segments thereby to calculate said corner coordinates.

10. The method of claim 9 further comprising calculating an effective intersection point for each combination of non-intersecting vertical and horizontal line segments that are within a threshold separation distance.

11. The method of claim 8 wherein said converting comprises using the four corner coordinates of each grid to define corners of a corresponding cell of said table object.

12. The method of claim 11 wherein said grid system generating comprises:

for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, arranging the vertical and horizontal line segments into different combinations, each combination comprising one vertical line segment and one horizontal line segment; and

for each combination, calculating the intersection point between the vertical and horizontal line segments thereby to calculate said corner coordinates.

13. The method of claim 11 wherein said transforming comprises for each freeform line object:

examining the orientation of the freeform line object; and

creating one of the horizontal and vertical line segment representing the freeform line object based on the orientation thereof.

14. The method of claim 13 wherein said transforming further comprises for each freeform line object:

determining the start point and end point of the freeform line object;

generating a temporary line segment extending between the determined start point and end point;

determining the orientation of said temporary line segment; and

based on the determined orientation, creating one of said horizontal line segment and vertical line segment.

15. The method of claim 14 wherein said transforming further comprises:

for each created horizontal line segment, determining whether it overlaps with an existing horizontal line segment and, if so, merging the created and existing horizontal line segments; and

for each created vertical line segment, determining whether it overlaps with an existing vertical line segment and, if so, merging the created and existing vertical line segments.

16. The method of claim 1 further comprising:

detecting input non-freeform line objects that are within the table object; and

inserting the detected non-freeform line objects as table object content.

17. The method of claim 16 wherein during said inserting, each detected non-freeform table object is placed into a corresponding cell of said table object as table cell content.

18. The method of claim 17 wherein said detecting comprises determining non-freeform line objects having designated locations positioned within a cell of the table object.

19. The method of claim 17 wherein said transforming comprises for each freeform line object:

examining the orientation of the freeform line object; and

creating one of the horizontal and vertical line segment representing the freeform line object based on the orientation thereof.

20. The method of claim 19 wherein said transforming further comprises for each freeform line object:

determining the start point and end point of the freeform line object;

generating a temporary line segment extending between the determined start point and end point;

determining the orientation of said temporary line segment; and

based on the determined orientation, creating one of said horizontal line segment and vertical line segment.

21. The method of claim 20 wherein said transforming further comprises:

for each created horizontal line segment, determining whether it overlaps with an existing horizontal line segment and, if so, merging the created and existing horizontal line segments; and

for each created vertical line segment, determining whether it overlaps with an existing vertical line segment and, if so, merging the created and existing vertical line segments.

22. The method of claim 21 wherein said grid system is generated based on deemed intersections of the horizontal and vertical line segments.

23. The method of claim 22 wherein said grid system generating comprises:

selecting pairs of adjacent vertical and adjacent horizontal line segments; and

for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, determining the intersection points therebetween thereby to yield four corner coordinates of each grid in the grid system.

24. The method of claim 23 wherein said grid system generating comprises:

for each pair of adjacent vertical line segments and each pair of adjacent horizontal line segments, arranging the vertical and horizontal line segments into different combinations, each combination comprising one vertical line segment and one horizontal line segment; and

for each combination, calculating the intersection point between the vertical and horizontal line segments thereby to calculate said corner coordinates.

25. The method of claim 17 further comprising processing the table object based on the table cell content placed in the table object.

26. A computerized digital ink recognition method comprising:

selecting digital ink presented on a graphical user interface, said digital ink at least comprising intersecting freeform lines representing a hand-drawn table;

performing a recognition procedure on said intersecting freeform lines to recognize the hand-drawn table;

generating a table object corresponding to the hand-drawn table in response to recognition thereof.

27. The method of claim 26 wherein said recognition procedure recognizes both regular and irregular hand-drawn tables.

28. The method of claim 27 wherein said recognition procedure is performed in response to user input.

29. The method of claim 28 further comprising:

detecting other objects presented on said graphical user interface that overlap with said hand-drawn table; and

placing the detected other objects as table cell content in cells of said table object.

30. The method of claim 29 wherein said detecting comprises determining other objects having designated locations positioned within cells of said table object.

31. The method of claim 29 further comprising processing the table object based on the table cell content placed in the table object.

32. A computer readable medium embodying computer program code for digital ink recognition, said computer program code comprising:

program code for selecting digital ink presented on a graphical user interface, said digital ink at least comprising intersecting freeform lines representing a hand-drawn table;

program code for performing a recognition procedure on said intersecting freeform lines to recognize the hand-drawn table;

program code for generating a table object corresponding to the hand-drawn table in response to recognition thereof.

33. A computer readable medium embodying computer program code for recognizing an input hand-drawn table formed by freeform line objects, said computer program code comprising:

program code for transforming each freeform line object of said table into one of a vertical line segment and a horizontal line segment;

program code for generating a grid system based on the vertical and horizontal line segments; and

program code for converting the generated grid system into a table object.

34. An interactive input system comprising:

a coordinate input device having a display surface on which an image is presented, said coordinate input device being responsive to user input; and

processing structure communicating with said coordinate input device and updating the image presented on said display surface to reflect user input, said processing structure executing a table recognition procedure when invoked to recognize a hand-drawn table entered on said coordinate input device and convert the recognized hand-drawn table into a table object.

35. An interactive input system according to claim 34 wherein said processing structure invokes the table recognition procedure in response to user input.

36. An interactive input system according to claim 35 wherein during execution of said table recognition procedure, said processing structure detects other displayed objects that overlap with said hand-drawn table and places the detected other objects as table cell content in cells of said table object.

37. An interactive input system according to claim 35 wherein during said hand-drawn table recognition, said processing structure transforms each freeform line object forming said hand-drawn table into one of a vertical line segment and a horizontal line segment, generates a grid system based on the vertical and horizontal line segments and converts the generated grid system into said table object.