Image action based on automatic feature extraction

• US 10,157,190 B2
• Filed: 03/28/2016
• Issued: 12/18/2018
• Est. Priority Date: 03/28/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Shih, Nai Wen, Tan, Chia-Jiun, Perez, Carlos
• Primary Examiner(s): Lam, Hung H
• Application Number:
  US15/083,120
• Publication Number:
  US 20170277719A1
• Time in Patent Office: 995 Days
• Field of Search: 725105
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

The present disclosure describes a system and method to automatically extract image features from an image uploaded to an image processing service executing on a cloud server from an image capture device, determine an image classification based at least in part on the image features, transmit a request to the image capture device for an image action based at least in part on the image classification, and execute the image action on the image based at least in part on receiving the image action from the image capture device in response to the request.

Claims

1. A system, comprising:

a memory configured to store instructions; and

one or more processors configured to execute the instructions stored in the memory to;

automatically extract image features from an image uploaded to an image processing service executing on the one or more processors from an image capture device;

determine an image type classification based at least in part on the image features;

transmit a request to the image capture device for user selection of an image action based at least in part on the image type classification;

receive the user selection of the image action from the image capture device; and

in response to the user selection, execute the image action on the image, the image action including at least one selected from a group consisting of sharing the image, storing the image, emailing the image, turning extracted features of the image into corresponding fields of a contact data structure, sending the image to other attendees of a meeting where the image was acquired, identifying individuals in the image, and sending a copy of the image to at least one individual identified in the image.

2. The system of claim 1, wherein the one or more processors are configured to execute the instructions stored in the memory further to:

automatically receive the image at the image processing service from the image capture device by establishing a connection between the image processing service and the image capture device in response to the image capture device detecting capture of the image.

3. The system of claim 2, wherein the one or more processors are configured to execute the instructions stored in the memory further to:

automatically receive the image from the image capture device in response to the image capture device detecting a shutter release.

4. The system of claim 1, wherein the one or more processors are configured to execute the instructions stored in the memory further to:

automatically store the image in a data store.

5. The system of claim 1, wherein the image features comprise color, color saturation, color tint, color hue, color depth, contrast, gamma correction, histogram parameters, brightness, noise, facial recognition parameters, scene recognition parameters, or object recognition parameters.

6. The system of claim 1, wherein the one or more processors are configured to execute the instructions stored in the memory further to:

automatically enhance the image based at least in part on the image type classification or the image features.

7. The system of claim 1, wherein the one or more processors are configured to execute the instructions stored in the memory further to:

determine metadata corresponding to the image; and

store the image together with the metadata.

8. The system of claim 7, wherein the one or more processors are configured to execute the instructions stored in the memory further to:

access secondary information about the image; and

transmit the request to the image capture device for the image action based at least in part on the secondary information.

19. The system of claim 1, wherein the image type classification is at least one selected from a group consisting of whiteboard, business card, event ticket, receipt, scene, photograph, meeting note, calendar entry, document, and group photo.

9. A method, comprising:

receiving, at an image processor executing on a cloud server, an image automatically uploaded from an image capture device;

automatically identifying at least one image feature from the image;

determining an image type classification based at least in part on the at least one image feature;

transmitting a request to the image capture device for user selection of an action based at least in part on the image type classification;

receiving the user selection of the action from the image capture device; and

in response to receiving the user selection, executing the action on the image, the action including at least one selected from a group consisting of sharing the image, storing the image, emailing the image, turning extracted features of the image into corresponding fields of a contact data structure, sending the image to other attendees of a meeting where the image was acquired, identifying individuals in the image, and sending a copy of the image to at least one individual identified in the image.

10. The method of claim 9, further comprising:

automatically establishing a communication connection between the image processor and the image capture device in response to the image capture device detecting capture of the image.

11. The method of claim 10, further comprising:

automatically receiving the image from the image capture device via the communication connection in response to the image capture device detecting a shutter release.

12. The method of claim 9, wherein the at least one image feature comprises color, color saturation, color tint, color hue, color depth, contrast, gamma correction, histogram parameters, brightness, noise, facial recognition parameter, scene recognition parameter, or object recognition parameter.

13. The method of claim 9, further comprising:

automatically enhancing the image based at least in part on the image type classification or the at least one image feature.

14. The method of claim 9, further comprising:

determining metadata corresponding to the image; and

storing the image together with the metadata.

15. The method of claim 9, further comprising:

accessing secondary information about the image; and

transmitting the request to the image capture device for the image action based at least in part on the secondary information.

16. A non-transitory computer-readable storage medium comprising instructions that, when executed by one or more processing devices, cause the one or more processing devices to:

automatically extract image features from an image automatically uploaded to an image processing service executing on a cloud server from an image capture device;

determine an image type classification based at least in part on the image features;

access secondary information about the image from a secondary application;

transmit a request to the image capture device for user selection of an action to be applied to the image based at least in part on the image type classification or the secondary information;

receive the user selection of the action to be applied to the image from the image capture device; and

in response to receiving the user selection, apply the action to the image, the action including at least one selected from a group consisting of sharing the image, storing the image, emailing the image, turning extracted features of the image into corresponding fields of a contact data structure, sending the image to other attendees of a meeting where the image was acquired, identifying individuals in the image, and sending a copy of the image to at least one individual identified in the image.

17. The non-transitory computer-readable storage medium of claim 16, wherein execution of the instructions by the one or more processing devices, cause the one or more processing devices further to:

automatically enhance the image based at least in part on the image type classification or the image features.

18. The non-transitory computer-readable storage medium of claim 16, wherein execution of the instructions by the one or more processing devices, cause the one or more processing devices further to:

determine metadata corresponding to the image; and

store the image together with the metadata.