Object recognition in an adaptive resource management system

• US 10,346,700 B1
• Filed: 03/28/2017
• Issued: 07/09/2019
• Est. Priority Date: 05/03/2016
• Status: Active Grant

• Current Assignee: Cynny Spa
• Original Assignee: Cynny Spa
• Inventors: Bargagni, Stefano, Rinaldelli, Mauro, Tassone, Daniele
• Primary Examiner(s): Mariam, Daniel G
• Application Number:
  US15/471,678
• Time in Patent Office: 833 Days
• Field of Search: 382103, 382224
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

A digital image processing method including: retrieving by a mobile computing device an object recognition model, where the object recognition model includes one or more object recognition definitions usable to identify within a digital image one or more objects captured by the digital image; receiving a first indication from the mobile computing device that there is a first computer resource available; determining, based on the first computer resource available, that the first computer resource available is sufficient to load a first object recognition definition defined in the object recognition model; loading into the first computer resource the first object recognition definition; analyzing a particular digital image, using the first object recognition definition loaded into the first computer resource, to determine whether a first object defined by the first object recognition definition is captured within the particular digital image; receiving a second indication from the mobile computing device that there is a second computer resource available; determining, based on the second computer resource available, that the second computer resource available is sufficient to load a second object recognition definition defined in the object recognition model; loading into the second computer resource the second object recognition definition; analyzing the particular digital image, using the second object recognition definition loaded into the second computer resource, to determine whether a second object defined by the second object recognition definition is captured within the particular digital image; associating as metadata of the particular digital image that the first object or the second object has been captured within the particular digital image.

Claims

1. A digital image processing method comprising:

retrieving by a mobile computing device an object recognition model, wherein the object recognition model comprises one or more object recognition definitions usable to identify within a digital image one or more objects captured by the digital image, the mobile computing device having a collection of computer resources available to process the digital image;

receiving a first indication from the mobile computing device that there is a first collection of computer resources available, the first collection of computer resources available representing availability of a first collection of computer resources of the mobile computing device required to process object recognition definitions defined in the object recognition model by the mobile computing device;

determining, based on the first collection of computer resources available to load a first object recognition definition defined in the object recognition model into the first collection of computer resources of the mobile computer device;

loading into the first collection of computer resources the first object recognition definition;

analyzing a particular digital image, using the first object recognition definition loaded into the first collection of computer resources to determine whether a first object defined by the first object recognition definition is captured within the particular digital image;

receiving a second indication from the mobile computing device that there is a second collection of computer resources available, the second collection of computer resources available representing availability of a second collection of computer resources of the mobile computing device required to process object recognition definitions defined in the object recognition model by the mobile computing device;

determining, based on the second collection of computer resources available to load a second object recognition definition defined in the object recognition model into the second collection of computer resources of the mobile computer device;

loading into the second collection of computer resources the second object recognition definition;

analyzing the particular digital image, using the second object recognition definition loaded into the second collection of computer resources to determine whether a second object defined by the second object recognition definition is captured within the particular digital image; and

associating with the particular digital image, as metadata of the particular digital image, data specifying that the first object or the second object has been captured within the particular digital image.

2. The method of claim 1 wherein the mobile computing device is unable to load all the object recognition definitions included in the object recognition model at a given time.

3. The method of claim 1 wherein the object recognition model is compressed before retrieval by the mobile computing device.

4. The method of claim 1 wherein the determining that the first collection of computer resources is available is in response to receiving the first indication.

5. The method of claim 1 wherein the determining that the first collection of computer resources is available comprises determining that the first collection of computer resources available is insufficient to load the second object recognition definition.

6. The method of claim 1 wherein the object recognition model is loaded onto memory of the mobile computing device before analyzing the particular digital image.

7. The method of claim 1 further comprising:

transmitting the metadata of the particular digital image to a second computing device;

causing storage at the second computing device that an object has been captured within the particular digital image.

8. The method of claim 1 wherein the first indication that there is a first collection of computer resources available comprises memory, processor bandwidth, or input device for user input.

9. The method of claim 1 wherein the object recognition model comprises output of a machine learning technique based on object recognition data stored separate from the mobile computing device.

10. The method of claim 1 wherein the first indication is received at a first time and the second indication is received at a second time, after the first time.

11. The method of claim 1 wherein the first object defined by the first object recognition definition, but not the second object defined by the second object recognition definition, is captured within the particular digital image, and the method further comprises

associating as metadata of the particular digital image that the first object has been captured within the particular digital image.

12. The method of claim 1 wherein the second object defined by the second object recognition definition, but not the first object defined by the first object recognition definition, is captured within the particular digital image, and the method further comprises

associating as metadata of the particular digital image that the second object has been captured within the particular digital image.

13. The method of claim 1 wherein the first and second objects defined by the first and second object recognition definitions, respectively, are captured within the particular digital image, and the method further comprises

associating as metadata of the digital image that the first and second objects have been captured within the particular digital image.

14. A digital image processing system comprising:

one or more processors;

a non-transitory computer-readable medium having instructions embodied thereon, the instructions, when executed by the one or more processors, cause;

retrieving by a mobile computing device an object recognition model, wherein the object recognition model comprises one or more object recognition definitions usable to identify within a digital image one or more objects captured by the digital image, the mobile computing device having a collection of computer resources available to process the digital image;

receiving a first indication from the mobile computing device that there is a first collection of computer resources available, the first collection of computer resources available representing availability of a first collection of computer resources of the mobile computing device required to process object recognition definitions defined in the object recognition model by the mobile computing device;

determining, based on the first collection of computer resources available to load a first object recognition definition defined in the object recognition model into the first collection of computer resources of the mobile computer device;

loading into the first collection of computer resources the first object recognition definition;

analyzing a particular digital image, using the first object recognition definition loaded into the first collection of computer resources to determine whether a first object defined by the first object recognition definition is captured within the particular digital image;

receiving a second indication from the mobile computing device that there is a second collection of computer resources available, the second collection of computer resources available representing availability of a second collection of computer resources of the mobile computing device required to process object recognition definitions defined in the object recognition model by the mobile computing device;

determining, based on the second collection of computer resources available to load a second object recognition definition defined in the object recognition model into the second collection of computer resources of the mobile computer device;

loading into the second collection of computer resources the second object recognition definition;

analyzing the particular digital image, using the second object recognition definition loaded into the second collection of computer resources to determine whether a second object defined by the second object recognition definition is captured within the particular digital image; and

associating with the particular digital image, as metadata of the particular digital image, that the first object or the second object has been captured within the particular digital image.

15. The system of claim 14 wherein the mobile computing device is unable to load all the object recognition definitions included in the object recognition model at a given time.

16. The system of claim 14 wherein the object recognition model is compressed before retrieval by the mobile computing device.

17. The system of claim 14 wherein the determining that the first collection of computer resources is available is in response to receiving the first indication.

18. The system of claim 14 wherein determining that the first collection of computer resources is available comprises determining that the first collection of computer resources available is insufficient to load the second object recognition definition.

19. The system of claim 14 wherein the second collection of computer resources available was previously unavailable.

20. The system of claim 14 further comprising:

transmitting the metadata of the particular digital image to a second computing device;

causing storage at the second computing device that an object has been captured within the particular digital image.