CONTEXT-AWARENESS THROUGH BIASED ON-DEVICE IMAGE CLASSIFIERS

• US 20160267324A1
• Filed: 05/18/2015
• Published: 09/15/2016
• Est. Priority Date: 03/11/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: LI, Jin, SHOAIB, Mohammed, LIU, Jie
• Granted Patent: US 10,268,886 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/451 with interaction between the filter responses, e.g. cortical complex cells
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/955 using specific electronic processors
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324

Abstract

Examples of the disclosure enable efficient processing of images. One or more features are extracted from a plurality of images. Based on the extracted features, the plurality of images are classified into a first set including a plurality of first images and a second set including a plurality of second images. One or more images of the plurality of first images are false positives. The plurality of first images and none of the plurality of second images are transmitted to a remote device. The remote device is configured to process one or more images including recognizing the extracted features, understanding the images, and/or generating one or more actionable items. Aspects of the disclosure facilitate conserving memory at a local device, reducing processor load or an amount of energy consumed at the local device, and/or reducing network bandwidth usage between the local device and the remote device.

Claims

1. A computer-implemented method for classifying one or more images, the method comprising executing on one or more computing devices the operations of:

extracting one or more features from a plurality of images;

based on the extracted features, classifying the plurality of images into a first set including a plurality of first images and a second set including a plurality of second images, one or more images of the plurality of first images being false positives; and

transmitting the plurality of first images and none of the plurality of second images to a remote device, the remote device configured to process one or more images including one or more of recognizing the extracted features, understanding the images, and generating one or more actionable items.

2. The computer-implemented method of claim 1, wherein extracting one or more features comprises identifying one or more interest points, the features extracted from the identified interest points.

3. The computer-implemented method of claim 1, wherein extracting one or more features from the plurality of images comprises extracting features from one image in parallel with extracting features from another image.

4. The computer-implemented method of claim 1, wherein classifying the plurality of images comprises classifying the plurality of second images into the second set such that no image classified into the second set is a true positive.

5. The computer-implemented method of claim 1, wherein classifying the plurality of images comprises classifying one image in parallel with classifying another image.

6. The computer-implemented method of claim 1, further comprising filtering the plurality of images such that the first set is partitioned from the second set.

7. The computer-implemented method of claim 1, further comprising receiving, from the remote device, the actionable items.

8. A mobile device comprising:

a sensor module configured to capture data corresponding to a plurality of images, one or more images of the plurality of images associated with an object;

a memory area storing computer-executable instructions for classifying the plurality of images; and

a processor configured to execute the computer-executable instructions to;

determine whether a first image of the plurality of images is associated with the object, the processor biased towards determining that the first image is associated with the object; and

transmit the first image to a remote device configured to process the first image including one or more of recognize extracted features, understand the images, and generate one or more actionable items.

9. The mobile device of claim 8, wherein the sensor module is configured to stream the plurality of images.

10. The mobile device of claim 8, wherein the processor is configured to execute the computer-executable instructions to:

extract one or more features from the plurality of images; and

based on the extracted features, classify the plurality of images into a first set of images and a second set of images.

11. The mobile device of claim 10, wherein the processor is configured to execute the computer-executable instructions to identify one or more interest points, the features extracted from the identified interest points.

12. The mobile device of claim 8, wherein the processor is configured to execute the computer-executable instructions to partition the first image from a second image of the plurality of images.

13. The mobile device of claim 8, wherein the processor is configured to execute the computer-executable instructions to determine whether the first image is associated with the object in parallel with determining whether a second image of the plurality of images is associated with the object.

14. The mobile device of claim 8, wherein the processor is configured to execute the computer-executable instructions to receive, from the remote device, the actionable items.

15. A computing device comprising:

a sensor module configured to capture data corresponding to a plurality of images, one or more images of the plurality of images associated with an object;

a feature classification module configured to;

classify the plurality of images into a first set including a plurality of first images and a second set including a plurality of second images, one or more images of the plurality of first images being false positives, and

transmit the plurality of first images and none of the plurality of second images to a remote device, the remote device configured to process one or more images including one or more of recognizing one or more features, understanding the images, and generating one or more actionable items.

16. The computing device of claim 15, wherein the feature classification module is configured to partition the first set from the second set.

17. The computing device of claim 15, wherein the feature classification module is biased such that no image classified into the second set is a true positive.

18. The computing device of claim 15, further comprising a feature computation module configured to extract one or more features from the plurality of images, wherein the plurality of images are classified based on the extracted features.

19. The computing device of claim 18, wherein the feature computation module is configured to identify one or more interest points in the plurality of images, wherein the features are extracted based on the identified interest points.

20. The computing device of claim 15, wherein the feature classification module is configured to classify one image in parallel with classifying another image.