Sharing editable ink annotated images with annotation-unaware applications

• US 7,975,215 B2
• Filed: 05/14/2007
• Issued: 07/05/2011
• Est. Priority Date: 05/14/2007
• Status: Expired due to Fees

• Current Assignee: Impinj, Inc., Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Gear, Gavin M., Perry, David B., Duncan, Richard J., Spencer, Richard L., Wilairat, Weerapan
• Primary Examiner(s): Hong; Stephen S.
• Assistant Examiner(s): Patel; Manglesh M
• Application Number:
  US11/748,265
• Publication Number:
  US 20080288857A1
• Time in Patent Office: 1,513 Days
• Field of Search: 715/230, 715/200, 715232-233
• US Class Current: 715/230
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

Data associated with pixels obscured by annotations in an annotated image is stored using an annotation mask enabling annotation-aware applications to reconstruct the original image, while annotation-unaware applications can display the annotated image with optimized increase in extra data to be stored. The annotation mask, same size as the image, is initialized to a monochrome default and the color depth information for obscured pixels saved within the annotation mask. Because a majority area of the annotation mask is monochrome, it compresses well and the obscured image content can be saved in a side channel, metadata header, or a separate file. During reconstruction, changes to annotated image such as rotation, size change, and the like, may be detected and accounted for when obscured image content is carried over to the annotated image for reconstruction.

Claims

1. A method to be executed at least in part in a computing device for using obscured image data to reconstruct an annotated image, the method comprising:

receiving an image to be annotated and annotations to be superimposed onto the image;

determining obscured image content (OIC) based on a position of the annotations to be superimposed onto the image;

persisting OIC data separate from the image, wherein the OIC data comprises a minimal amount of information necessary to reconstruct the image, the minimal amount of information comprising pixel data from the image which has been obscured by the annotations;

rendering the annotations on the image, wherein the annotations are persisted on the image such that the annotated image is a single layer flat image;

making the annotated image and the OIC data available for use by annotation-aware and annotation-unaware applications;

receiving the annotated image and the OIC data;

generating a bitmap based on the annotated image;

reading the OIC data;

copying only the pixel data from the image which has been obscured by the annotations onto the image to reconstruct the obscured pixels;

displaying the reconstructed image; and

storing an identifier to mitigate changes by at least one of the annotation-unaware applications, wherein the identifier uniquely identifies exact binary data in the image, wherein the identifier is stored with restore data for restoring the image from the annotated image.

2. The method of claim 1, further comprising:

generating a monochrome annotation mask based on the image;

rendering the annotations on the annotation mask;

generating the OIC on the annotation mask by copying pixels obscured by the annotations on the image to the annotation mask; and

persisting the annotation mask and the OIC data.

3. The method of claim 2, further comprising:

adjusting the annotation mask to compensate for rendering effects;

expanding the annotation mask to reduce compression artifacts; and

if the annotations cause an increase in image size, adjusting the image size and the annotation mask size based on the increase.

4. The method of claim 2, wherein persisting the annotation mask and the OIC data includes:

compressing the annotation mask and the OIC data; and

saving the compressed annotation mask and the OIC data in one of;

a side channel of the image, a metadata header of the image, a separate file associated with the image, and a supplemental data store.

5. The method of claim 2, further comprising:

initializing all pixels of the annotation mask to a default color prior to rendering the annotations on the annotation mask.

6. The method of claim 2, further comprising:

expanding a count of pixels on the annotation mask for capturing OIC from the image.

7. The method of claim 1, wherein the image includes at least one from a set of:

a still image, a video image, a text document, a spreadsheet document, and a graphic.

8. The method of claim 1, further comprising:

scaling down the original image to its original size, if the size of the image has been increased to accommodate the annotations.

9. The method of claim 1, further comprising:

displaying the annotations along with the reconstructed image for further editing by a user.

10. A system for using obscured image data to reconstruct an annotated image, the system comprising:

a memory;

a processor coupled to the memory, wherein the processor is configured to execute program modules including;

an annotation integration module configured to;

receive an image to be annotated and annotations to be superimposed onto the image;

determine obscured image content (OIC) based on a position of the annotations to be superimposed onto the image;

persist OIC data separate from the image, wherein the OIC data comprises a minimal amount of information necessary to reconstruct the image, the minimal amount of information comprising pixel data from the image which has been obscured by the annotations;

render the annotations on the image, wherein the annotations are persisted on the image such that the annotated image is a single layer flat image;

make the annotated image and the OIC data available for use by annotation-aware and annotation-unaware applications;

store an identifier to mitigate changes by at least one of the annotation-unaware applications, wherein the identifier uniquely identifies exact binary data in the image, wherein the identifier is stored with restore data for restoring the image from the annotated image; and

a reconstruction module configured to;

receive the annotated image and the OIC data;

generate a bitmap based on the annotated image;

copy only the pixel data from the image which has been obscured by the annotations onto the image based on a location of each obscured pixel to reconstruct the obscured pixels on the annotated image; and

render the reconstructed image.

11. The system of claim 10, wherein the annotation integration module is further configured to:

generate a monochrome annotation mask based on the received image, wherein all pixels of the annotation mask are initialized to a default color;

render the received annotations on the annotation mask;

generate the OIC data on the annotation mask by copying pixels obscured by the annotations on the image to the annotation mask; and

persist the annotation mask and the OIC data in one of;

a side channel of the image, a metadata header of the image, a separate file associated with the image, and a supplemental data store.

12. The system of claim 10, wherein the annotation integration module is further configured to:

store at least one from a set of;

a Cyclic Redundancy Check (CRC) of image data, a CRC of the OIC data, and a property associated with the image along with the OIC data.

13. The system of claim 12, wherein the property associated with the image includes at least one from a set of:

a size, a width, a height, a color property, a contrast property, and a brightness property of the image.

14. The system of claim 12, wherein the construction module is further configured to:

detect a modification of the annotated image after the annotation mask and the OIC data were saved;

determine a type of modification; and

adjust the OIC data before copying to the image, if the modification includes one of;

a rotation, a horizontal flip, a vertical flip, and a resize with preserved aspect ratio.

15. The system of claim 14, wherein the construction module is further configured to:

perform an image analysis, if the modification is other than one of;

a rotation, a horizontal flip, a vertical flip, and a resize with preserved aspect ratio;

if a predefined confidence level is reached by the image analysis, providing a user with a User Interface (UI) to adjust a placement of the annotations; and

provide the user a warning that the original image is not recoverable, if the predefined confidence level is not reached by the image analysis.

16. The system of claim 10, wherein the annotation integration module and the reconstruction module are part of an Application Programming Interface (API) associated with an annotation-aware application.

17. A computer-readable storage medium comprising computer-executable instructions which, when executed by a computer, will cause the computer to perform a method of using obscured image data to reconstruct an annotated image, the method comprising:

receiving an image to be annotated and annotations to be superimposed onto the image, wherein the annotations are persisted on the image such that an annotated image comprising the image and the superimposed annotations is a single layer flat image;

generating a monochrome annotation mask based on the image, wherein the annotation mask has a same size as the image and pixels of the annotation mask are initialized to a default color;

expanding a count of pixels on the annotation mask for capturing OIC data from the image, wherein the OIC data comprises a minimal amount of information necessary to reconstruct the image, the minimal amount of information comprising pixel data from the image which has been obscured by the annotations;

rendering the annotations on the annotation mask;

adjusting the annotation mask to compensate for rendering effects;

generating obscured image content (OIC) on the annotation mask by copying color depth information associated with pixels obscured by the annotations on the image to the annotation mask;

persisting the annotation mask and the OIC data in one of;

a side channel of the image, a metadata header of the image, a separate file associated with the image, and a supplemental data store;

storing at least one from a set of;

a Cyclic Redundancy Check (CRC) of image data to mitigate changes by at least one annotation-unaware application, wherein the identifier uniquely identifies exact binary data in the image, wherein the identifier is stored with restore data for restoring the image from the annotated image;

generating a bitmap based on the annotated image;

reading the annotation mask and the OIC data;

detecting a modification of the annotated image after the annotation mask and the OIC data were saved;

determining a type of modification;

adjusting the OIC data, if the modification includes one of;

a rotation, a horizontal flip, a vertical flip, and a resize with preserved aspect ratio;

copying only the pixel data from the image which has been obscured by the annotations onto the image to reconstruct the obscured pixels; and

displaying the reconstructed image.

18. The computer-readable storage medium of claim 17, wherein the method further comprises:

displaying the annotations to enable a user to perform further edits on the annotations.