Editing digital notes representing physical notes

• US 10,325,389 B2
• Filed: 01/27/2016
• Issued: 06/18/2019
• Est. Priority Date: 10/16/2013
• Status: Active Grant

• Current Assignee: 3M Innovative Properties Company (3M Company)
• Original Assignee: 3M Innovative Properties Company (3M Company)
• Inventors: Moore, Richard J., Leighton, Michael C., Somasundaram, Guruprasad, Åkerlund, Linus, Persson, Olof Roland, Aybar Lopez, Hector M., Wolk, Diane R., Axelsson, Pontus, Kjellberg, Staffan H.
• Primary Examiner(s): Bhargava, Anil K
• Application Number:
  US15/008,033
• Publication Number:
  US 20160180565A1
• Time in Patent Office: 1,238 Days
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

In one example, a method includes receiving a digital note of a plurality of digital notes generated based on image data comprising a visual representation of a scene that includes a plurality of physical notes such that each of the plurality of digital notes respectively corresponds to a particular physical note of the plurality of physical notes, wherein each of the physical notes includes respective recognizable content. In this example, the method also includes receiving user input indicating a modification to one or more visual characteristics of the digital note. In this example, the method also includes editing, in response to the user input, the one or more visual characteristics of the digital note. In this example, the method also includes outputting, for display, a modified version of the digital note that includes the one or more visual characteristics.

Claims

1. A method comprising:

receiving, by one or more processors of a device, a digital note of a plurality of digital notes generated based on image data comprising a visual representation of a scene that includes a plurality of physical notes such that each of the plurality of digital notes respectively corresponds to a particular physical note of the plurality of physical notes, wherein each of the physical notes includes respective recognizable content;

receiving, by the one or more processors, user input indicating a modification to one or more visual characteristics of the digital note;

editing, by the one or more processors and in response to the user input, the one or more visual characteristics of the digital note; and

outputting, by the one or more processors and for display, a modified version of the digital note that includes the one or more visual characteristics,

wherein receiving the user input comprises receiving user input that indicates additional content for the digital note and a command to remove previously entered additional content for the digital note,

wherein editing the one or more visual characteristics of the digital note comprises creating a layer object that includes the additional content and removing another layer object that includes the previously entered additional content, and

wherein outputting the modified updated version of the digital note comprises outputting the modified version of the digital note such that the layer object that includes the additional content is to be displayed on top of content corresponding to the respective recognizable content of the physical note to which the digital note corresponds and without the another layer object in the digital note.

2. The method of claim 1, wherein the additional content is first additional content, the layer object is a first layer object of a plurality of layer objects, and the modified version of the digital note is a first modified version of the digital note, the method further comprising:

responsive to receiving user input indicating second additional content;

creating a second layer of the plurality of layer objects that includes the second additional content; and

outputting, for display, a second modified version of the digital note such that the second layer object that includes the second additional content is displayed on top of the first layer object that includes the first additional content.

3. The method of claim 2, further comprising:

responsive to receiving user input to remove the second layer object from the updated version of the digital note, outputting, for display, a third modified version of the digital note such that the second additional content included in the second layer object is not displayed.

4. The method of claim 1, wherein receiving the user input that indicates the additional content comprises receiving one or more of:

a sequence of typed characters, and

a stroke gesture.

5. The method of claim 1, wherein:

receiving the user input further comprises receiving user input that indicates an updated background color,

editing the one or more visual characteristics of the digital note further comprises setting a background color of the digital note to the updated background color.

6. The method of claim 1, wherein receiving the digital note of the plurality of digital notes comprises:

receiving, by the one or more processors and from a camera of the device, the image data comprising the visual representation of the scene that includes the plurality of physical notes; and

generating, by the one or more processors and based on the image data, the plurality of digital notes that each respectively correspond to the particular physical note of the plurality of physical notes.

7. A method comprising:

receiving, by one or more processors of a device, a digital note of a plurality of digital notes generated based on image data comprising a visual representation of a scene that includes a plurality of physical notes such that each of the plurality of digital notes respectively corresponds to a particular physical note of the plurality of physical notes, wherein each of the physical notes includes respective recognizable content;

receiving, by the one or more processors, user input indicating a modification to one or more visual characteristics of the digital note;

editing, by the one or more processors and in response to the user input, the one or more visual characteristics of the digital note; and

outputting, by the one or more processors and for display, a modified version of the digital note that includes the one or more visual characteristics,

wherein receiving the user input comprises receiving user input that indicates an updated background color, and

wherein editing the one or more visual characteristics of the digital note comprises setting a background color of the digital note to the updated background color,

wherein receiving the user input comprises receiving user input that indicates additional content for the digital note and a command to remove previously entered additional content for the digital note,

wherein editing the one or more visual characteristics of the digital note comprises creating a layer object that includes the additional content and removing another layer object that includes the previously entered additional content, and

wherein outputting the modified updated version of the digital note comprises outputting the modified version of the digital note such that the layer object that includes the additional content is displayed on top of content corresponding to the respective recognizable content of the physical note to which the digital note corresponds and without the another layer object in the digital note.

8. The method of claim 7, further comprising:

identifying, for the digital note, a foreground region that includes content corresponding to recognizable content of the physical note to which the digital note corresponds, wherein at least a portion of the content of the foreground region is associated a foreground color; and

modifying the foreground color.

9. The method of claim 8, wherein:

receiving the user input comprises receiving user input that indicates an updated foreground color, and

modifying the foreground color comprises setting the foreground color to the updated foreground color in response to receiving user input that indicates the updated foreground color.

10. The method of claim 8, wherein modifying the foreground color comprises automatically modifying the foreground color as a function of the updated background color.

11. The method of claim 10, wherein prior to modifying the foreground color, a plurality of pixels included in the foreground region have current foreground pixel colors, and wherein modifying the foreground color comprises modifying colors of the plurality of pixels included in the foreground region by at least:

selecting, by the one or more processors and for each pixel of the plurality of pixels included in the foreground region, an updated foreground pixel color such that a contrast ratio between the updated foreground pixel color and the updated background color is greater than a contrast ratio between the current foreground pixel color and the updated background color, and

setting each respective pixel of the plurality of pixels to the selected respective updated foreground pixel color.

12. The method of claim 8, further comprising:

identifying, subsequent to identifying the foreground region of the digital note, a background region of the digital note by at least identifying pixels of the digital note not included in the foreground region as the background region.

13. The method of claim 8, further comprising:

identifying a background region of the digital note by at least identifying pixels of the digital note not included in the foreground region as the background region, wherein identifying the foreground region is performed subsequent to identifying the background region, and wherein identifying the foreground region for the particular digital note comprises;

identifying pixels of the particular digital note not included in the background region as the foreground region.

14. The method of claim 7, wherein receiving the digital note of the plurality of digital notes comprises:

receiving, by the one or more processors and from a camera of the device, the image data comprising the visual representation of the scene that includes the plurality of physical notes; and

generating, by the one or more processors and based on the image data, the plurality of digital notes that each respectively correspond to the particular physical note of the plurality of physical notes.

15. The method of claim 14, further comprising:

enhancing, by the one or more processors, the digital note of the plurality of digital notes by at least;

identifying, for the digital note, a foreground region that includes content corresponding to the respective recognizable content of the physical note to which the digital note corresponds;

identifying a background region of the digital note; and

setting pixel values from the background region of the particular digital note to a uniform background pixel value.

16. A computing device comprising:

a memory configured to store digital notes; and

one or more processors configured to;

receive a digital note of a plurality of digital notes generated based on image data comprising a visual representation of a scene that includes a plurality of physical notes such that each of the plurality of digital notes respectively corresponds to a particular physical note of the plurality of physical notes, wherein each of the physical notes includes respective recognizable content;

receive user input indicating an additional content for the digital note and a command to remove previously entered additional content for the digital note,

edit, in response to the user input, the digital note by at least creating a layer object that includes the additional content and removing another layer object that includes the previously entered additional content; and

output a modified version of the digital note by at least outputting the modified version of the digital note such that the layer object that includes the additional content and the content of the respective recognizable content of the physical note to which the digital note corresponds is displayed without the another layer object in the digital note.

17. The computing device of claim 16, wherein the one or more processors are configured to receive the digital note of the plurality of digital notes by at least:

receiving, from a camera of the device, the image data comprising the visual representation of the scene that includes the plurality of physical notes; and

generating, based on the image data, the plurality of digital notes that each respectively correspond to the particular physical note of the plurality of physical notes.

18. The computing device of claim 16, wherein:

the additional content is first additional content,

the layer object is a first layer object of a plurality of layer objects,

the modified version of the digital note is a first modified version of the digital note, and

responsive to receiving user input indicating second additional content, the one or more processors are configured to;

create a second layer of the plurality of layer objects that includes the second additional content; and

output, for display, a second modified version of the digital note such that the second layer object that includes the second additional content is displayed on top of the first layer object that includes the first additional content.

19. The computing device of claim 16, wherein the one or more processors are configured to receive the user input that indicates the additional content by at least receiving one or more of:

a sequence of typed characters, and

a stroke gesture.