Computer vision as a service

• US 9,152,870 B2
• Filed: 03/14/2014
• Issued: 10/06/2015
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Sawhney, Harpreet Singh, Eledath, Jayakrishan, Ali, Saad, Matei, Bogdan C., Weiner, Steven S., Lv, Xutao
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Dunphy, David F
• Application Number:
  US14/212,237
• Publication Number:
  US 20140270494A1
• Time in Patent Office: 571 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures

Abstract

A computer vision service includes technologies to, among other things, analyze computer vision or learning tasks requested by computer applications, select computer vision or learning algorithms to execute the requested tasks based on one or more performance capabilities of the computer vision or learning algorithms, perform the computer vision or learning tasks for the computer applications using the selected algorithms, and expose the results of performing the computer vision or learning tasks for use by the computer applications.

Claims

1. A platform for providing computer vision algorithm services, the platform comprising an application-algorithm interface, the application-algorithm interface comprising a plurality of instructions embodied in one or more non-transitory machine accessible storage media, the application-algorithm interface to:

receive a computer vision task from a computer application, the computer vision task to be performed on one or more digital images accessed by the computer application;

determine one or more parameters relating to the performing of the computer vision task on the one or more digital images;

select one or more computer vision algorithms from a library of computer vision algorithms based on capabilities of the computer vision algorithms that have a high level of performance in comparison to the capabilities of the other computer vision algorithms in the library of computer vision algorithms to perform the computer vision task on the one or more digital images with the one or more parameters;

expose, for use by the computer application, output of the selected computer vision algorithm performing the computer vision task on the one or more digital images using the one or more parameters; and

wherein the platform is to organize the library of computer vision algorithms according to a plurality of different levels of abstraction and determine a level of abstraction at which to select the computer vision algorithm based on a characteristic of the computer vision task or a characteristic of the computer application.

2. The platform of claim 1, wherein the platform is to execute one or more algorithm performance characterization techniques to determine the one or more parameters relating to the performing of the computer vision task.

3. The platform of claim 2, wherein the platform is to map one or more application parameters supplied by the computer application to one or more algorithm parameters of the selected computer vision algorithms based on the executing of the one or more algorithm performance characterization techniques.

4. The platform of claim 1, wherein the platform is to determine a content characteristic or an application-defined parameter of the one or more digital images, wherein the application-defined parameter is defined by the computer application, and select the computer vision algorithm based on the content characteristic or the application-defined parameter of the one or more digital images.

5. The platform of claim 4, wherein the platform is to analyze the performance of the selected computer vision algorithm on the computer vision task based on the content characteristic or the application-defined parameter of the one or more digital images, and determine a set of algorithm parameters for the selected computer vision algorithm based on analyzing the performance of the selected computer vision algorithm.

6. The platform of claim 1, wherein the platform is to select a combination of different computer vision algorithms to perform the computer vision task, execute the combination of different computer vision algorithms on the one or more digital images using the parameter, and expose, for use by the computer application, output of the combination of computer vision algorithms executing the computer vision task.

7. The platform of claim 1, wherein the platform is to determine a hardware computing resource available to perform the computer vision task, estimate the computational performance of the selected computer vision algorithm on the hardware computing resource, and determine an algorithm parameter for use with the selected computer vision algorithm based on the computational performance of the selected computer vision algorithm on the computing resource.

8. The platform of claim 1, wherein the platform is to select a data store of a plurality of available data stores for use in performing the computer vision task, wherein each of the data stores comprises reference data usable to perform the computer vision task, and the platform selects the selected data store based on a content characteristic or an application-defined parameter of the one or more digital images, wherein the application-defined parameter is defined by the computer application.

9. The platform of claim 1, wherein the platform is to compute a plurality of different performance measures for the selected computer vision algorithm, and determine an algorithm parameter for use with the selected computer vision algorithm based on the plurality of different performance measures.

10. The platform of claim 9, wherein the platform is to determine an algorithm type of the selected computer vision algorithm, and customize the performance measures for the selected computer vision algorithm based on the algorithm type.

11. The platform of claim 1, wherein the platform is to predictively characterize the performance of the selected one or more computer vision algorithms without executing the algorithm.

12. The platform of claim 11, wherein the platform is to predictively characterize the performance of the selected one or more computer vision algorithms based on one or more content characteristics of the one or more digital images and/or one or more application-defined parameters supplied by the computer application.

13. The platform of claim 1, wherein the platform is to characterize the performance of the selected one or more computer vision algorithms by executing the selected one or more computer vision algorithms on the one or more digital images and analyzing the output of the selected one or more computer vision algorithms.

14. A platform for providing computer vision and learning algorithm services to user-oriented computer applications, the platform comprising:

an application-algorithm interface embodied in one or more non-transitory machine accessible storage media, the application-algorithm interface to determine application parameters to perform a computer vision or learning task on digital content, the computer vision or learning task received from a computer application, at least one of the application parameters indicating a characteristic of the digital content;

an algorithm capabilities interface embodied in one or more non-transitory machine accessible storage media, the algorithm capabilities interface to, based on the application parameters, identify candidate computer vision or learning algorithms to perform the computer vision or learning task on the digital content based on the application parameters;

a performance interface embodied in one or more non-transitory machine accessible storage media, the performance interface to evaluate a capability of each of the candidate computer vision or learning algorithms to perform the computer vision or learning task on the digital content, the performance capability determined at least in part by the characteristic of the digital content; and

an algorithm organization framework embodied in one or more non-transitory machine accessible storage media, the algorithm organization framework to organize the candidate computer vision or learning algorithms according to a plurality of different levels of abstraction, wherein the platform is to select a level of abstraction based on the computer vision or learning task.

15. The platform of claim 14, wherein the platform is to select a computer vision or learning algorithm of the candidate computer vision or learning algorithms based on the evaluating of the capability of the selected computer vision or learning algorithm to perform the computer vision or learning task on the digital content.

16. The platform of claim 15, comprising an algorithms parameter mapping module to map the application parameters to one or more algorithm parameters to use with the selected computer vision or learning algorithm to perform the computer vision or learning task on the digital content.

17. The platform of claim 16, wherein the platform is to perform the computer vision or learning task on the digital content by executing the selected computer vision or learning algorithm using the one or more algorithm parameters.

18. The platform of claim 17, wherein the platform is to communicate a result of executing the selected computer vision or learning algorithm to the computer application.

19. The platform of claim 14, comprising a computational performance interface to determine a hardware computing resource available to perform the computer vision or learning task, and estimate the computational performance of each of the candidate computer vision or learning algorithms on the hardware computing resource.

20. The platform of claim 14, comprising a task performance interface to compute a plurality of different performance measures for each of the candidate computer vision or learning algorithms, and select the computer vision or learning algorithm based on the plurality of different performance measures.

21. The platform of claim 14, comprising a data organization service to index a plurality of reference data for use by the platform in executing the computer vision or learning algorithms.

22. The platform of claim 14, comprising a plurality of application programming interfaces to expose computer vision or learning algorithms for use at a plurality of different levels of abstraction.

23. The platform of claim 14, comprising a plurality of application programming modules embodied in one or more non-transitory machine accessible storage media, the application programming modules to expose computer vision or learning algorithms for use at a plurality of different levels of abstraction.

24. A platform for providing computer vision and learning algorithm services to user-oriented computer applications, the platform comprising:

an application-algorithm interface embodied in one or more non-transitory machine accessible storage media, the application-algorithm interface to determine application parameters to perform a computer vision or learning task on digital content, the computer vision or learning task received from a computer application, at least one of the application parameters indicating a characteristic of the digital content;

an algorithm capabilities interface embodied in one or more non-transitory machine accessible storage media, the algorithm capabilities interface to, based on the application parameters, identify candidate computer vision or learning algorithms to perform the computer vision or learning task on the digital content based on the application parameters;

a performance interface embodied in one or more non-transitory machine accessible storage media, the performance interface to evaluate a capability of each of the candidate computer vision or learning algorithms to perform the computer vision or learning task on the digital content, the performance capability determined at least in part by the characteristic of the digital content; and

a plurality of application programming interfaces each embodied in one or more non-transitory machine accessible storage media, the application programming interfaces to expose computer vision or learning algorithms for use at a plurality of different levels of abstraction.

25. The platform of claim 24, wherein the platform is to select a computer vision or learning algorithm of the candidate computer vision or learning algorithms based on the evaluating of the capability of the selected computer vision or learning algorithm to perform the computer vision or learning task on the digital content.

26. The platform of claim 25, comprising an algorithms parameter mapping module embodied in one or more non-transitory machine accessible storage media, the algorithms parameter mapping module to map the application parameters to one or more algorithm parameters to use with the selected computer vision or learning algorithm to perform the computer vision or learning task on the digital content.

27. The platform of claim 26, wherein the platform is to perform the computer vision or learning task on the digital content by executing the selected computer vision or learning algorithm using the one or more algorithm parameters.

28. The platform of claim 27, wherein the platform is to communicate a result of executing the selected computer vision or learning algorithm to the computer application.

29. The platform of claim 24, comprising a computational performance module embodied in one or more non-transitory machine accessible storage media, the computational performance module to determine a hardware computing resource available to perform the computer vision or learning task, and estimate the computational performance of each of the candidate computer vision or learning algorithms on the hardware computing resource.

30. The platform of claim 24, comprising a task performance module embodied in one or more non-transitory machine accessible storage media, the task performance module to compute a plurality of different performance measures for each of the candidate computer vision or learning algorithms, and select the computer vision or learning algorithm based on the plurality of different performance measures.

31. The platform of claim 24, comprising a data organization service embodied in one or more non-transitory machine accessible storage media, the data organization service to index a plurality of reference data for use by the platform in executing the computer vision or learning algorithms.