Generating digital document content from a digital image

• US 10,204,082 B2
• Filed: 03/31/2017
• Issued: 02/12/2019
• Est. Priority Date: 03/31/2017
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Antony, Mathew, Wu, Michael Q.
• Primary Examiner(s): Huynh, Cong-Lac
• Application Number:
  US15/475,599
• Publication Number:
  US 20180285321A1
• Time in Patent Office: 683 Days
• Field of Search: 715256, 715263, 715268
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/177 of tables; using ruled lines
• CPC Class Codes: G06F 40/18 of spreadsheets form-filling G06F40/174
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: G06V 30/1988 Graph matching graphical image representation G06V30/18181
• CPC Class Codes: G06V 30/268 Lexical context
• CPC Class Codes: G06V 30/293 of characters other than Kanji, Hiragana or Katakana
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561

Abstract

One or more embodiments of systems and methods for a digital content management system for creating a digital document from handwritten content are described herein. For example, the digital content management system receives a digital image of handwritten content and analyzes the digital image to identify handwritten content as well as to identify specific command indicators. In response to identifying a command indicator associated with a command to create a digital document, the digital content management system creates a new digital document and adds digital content portions to the digital document that correspond to the identified content portions identified within the handwritten content depicted within the digital image.

Claims

1. A method comprising:

receiving a first digital image comprising handwritten content, wherein the handwritten content comprises at least one command indicator and at least one content portion;

analyzing the first digital image to identify a first command indicator and one or more content portions within the handwritten content of the first digital image;

identifying a file name used in conjunction with the first command indicator within the first digital image;

searching, based on the file name, a digital content management system to locate a matching digital document using the file name; and

adding, based on a first command associated with the first command indicator, the one or more content portions within the handwritten content to the matching digital document.

2. The method of claim 1, wherein adding the one or more content portions within the handwritten content to the matching digital document comprises:

creating a digital content portion for each of the one or more content portions within the handwritten content; and

adding the digital content portion to the matching digital document.

3. The method of claim 2, wherein creating the digital content portion for each of the one or more content portions comprises converting handwritten text within the one or more content portions identified within the digital image into digital text.

4. The method of claim 1, wherein the one or more content portions within the handwritten content comprise one or more of handwritten text or a handdrawn figure.

5. The method of claim 1, wherein the first command indicator comprises a handwritten symbol.

6. The method of claim 1, further comprising:

identifying an arrangement of content portions within the first digital image; and

arranging the one or more content portions within the matching digital document based on the arrangement of content portions within the first digital image.

7. The method of claim 1, further comprising:

identifying a second file name used in conjunction with the first command indicator within the first digital image;

searching, based on the second file name, the digital content management system to determine that a second matching digital document using the second file name does not exist within the digital content management system;

creating, in response to determining that the second matching digital document does not exist within the digital content management system, a second digital document; and

adding the one or more content portions within the handwritten content to the second digital document.

8. The method of claim 1, further comprising identifying a second command indicator based on the analysis of the first digital image.

9. The method of claim 8, further comprising sending, based on a second command associated with the second command indicator, an invitation to join a group associated with the matching digital document to one or more users of the digital content management system.

10. The method of claim 8, further comprising inserting, based on a second command associated with the second command indicator, a figure within the matching digital document.

11. The method of claim 8, further comprising:

creating, based on a second command associated with the second command indicator, a task list comprising at least one task assignment;

identifying one or more user names used in conjunction with the second command indicator; and

sending, to one or more users associated with the one or more user names, an invitation to access the task list.

12. The method of claim 1, further comprising:

receiving a second digital image comprising one or more changes to the one or more content portions within the handwritten content;

analyzing the second digital image to identify differences between the first digital image and the second digital image; and

updating, in response to identifying differences between the first digital image and the second digital image, the matching digital document to reflect the differences between the first digital image and the second digital image.

13. The method of claim 1, further comprising:

receiving a second digital image comprising one or more additional content portions;

analyzing the second digital image to identify the one or more additional content portions; and

adding, based on identifying the one or more additional content portions, the one or more additional content portions of the second digital image to the matching digital document.

14. The method of claim 1, further comprising:

identifying a first user associated with the first digital image;

receiving a second digital image associated with a second user and comprising additional handwritten content, wherein the additional handwritten content comprises a second command indicator and one or more additional content portions;

analyzing the second digital image to identify the second command indicator and the one or more additional content portions; and

based on determining the second command indicator from the second digital image matches the first command indicator within the first digital image;

adding the one or more additional content portions from the second digital image to the matching digital document; and

sharing the matching digital document with the first user and the second user.

15. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computer system to:

receive a digital image comprising handwritten content, wherein the handwritten content comprises a plurality of command indicators and at least one content portion;

analyze the digital image to identify a first command indicator and one or more content portions within the handwritten content;

identify a file name used in conjunction with the first command indicator within the first digital image;

search, based on the file name, a digital content management system to locate a matching digital document using the file name; and

add, based on a first command associated with the first command indicator, the one or more content portions within the handwritten content to the matching digital document.

16. The non-transitory computer readable medium of claim 15, wherein identifying the one or more content portions comprises:

identifying one or more text portions within the digital image; and

identifying one or more figure portions within the digital image.

17. The non-transitory computer readable medium of claim 16, wherein the one or more figure portions comprise one or more of a chart, a table, a list, a graph, an image, a spreadsheet, an equation, a text box, or a link.

18. The non-transitory computer readable medium of claim 16, further comprising instructions that, when executed by the at least one processor, cause the computer system to:

identify, within the digital image, an arrangement of content portions relative to each other; and

arrange, within the matching digital document, the one or more content portions relative to each other based on the identified arrangement within the digital image.

19. A system comprising:

at least one processor; and

at least one non-transitory computer readable storage medium storing instructions thereon that, when executed by the at least one processor, cause the system to;

receive a first digital image comprising handwritten content, wherein the handwritten content comprises at least one command indicator and at least one content portion;

analyze the first digital image to identify a first command indicator and one or more content portions within the handwritten content of the first digital image;

identify a file name used in conjunction with the first command indicator within the first digital image;

search, based on the file name, a digital content management system to locate a matching digital document using the file name; and

add, based on a first command associated with the first command indicator, the one or more content portions within the handwritten content to the matching digital document.

20. The system of claim 19, further comprising instructions that, when executed by the at least one processor, further cause the system to:

detect formatting characteristics of the one or more content portions within the handwritten content of the first digital image; and

format the one or more content portions within the matching digital document to correspond to the format characteristics of the one or more content portions within the handwritten content of the first digital image.