Using available data to assist in object recognition

• US 9,600,720 B1
• Filed: 03/18/2014
• Issued: 03/21/2017
• Est. Priority Date: 03/18/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Gray, Timothy Thomas
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US14/218,216
• Time in Patent Office: 1,099 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

Processes such as image matching, computer vision, and object recognition can utilize additional data, such as spatial data, to attempt to improve the accuracy of the results of those processes. For example, a computing device acquiring scene data including a representation of an object can also determine spatial data (e.g., location and orientation data). By determining the spatial data, a set of potential matches can be found which can help to more quickly and accurately identify the object based on one or more objects known to be at a corresponding position. The data acquired by the computing device can also be used to update matching data stored for that location, which can assist with subsequent processing.

Claims

1. A computing system, comprising:

at least one processor;

memory including instructions that, when executed by the at least one processor, cause the computing system to;

in a first time period;

obtain first image data acquired using a camera of a first device;

compare at least a portion of the first image data to second image data previously acquired by at least one second device;

identify, using a first confidence threshold, a first previously-acquired image of the second image data, the first previously-acquired image matching at least a portion of the first image data;

determine the first device is approaching a global position;

obtain third image data associated with the global position, the third image data acquired by the at least one second device;

in a second time period after the first time period;

obtain fourth image data acquired using the camera, the fourth image data including a representation of an object;

compare at least a portion of the fourth image data to the third image data;

identify, using a second confidence threshold, a second previously-acquired image of the third image data, the second previously-acquired image matching at least a portion of the fourth image data, the second confidence threshold being lower than the first confidence threshold; and

provide, to the first device, at least one of the second previously-acquired image or information associated with the second previously-acquired image, wherein the information associated with the second previously-acquired image includes actionable content causing the computing system to provide a purchasing option for an item related to the object.

2. The computer system of claim 1, wherein the instructions when executed further cause the computing system to:

identify at least one additional previously-acquired image of the third image data; and

provide the at least one additional previously-acquired image for display on the first device.

3. The computer system of claim 1, wherein the instructions when executed further cause the computing system to:

compare at least one quality parameter for the fourth image data and the third image data; and

identify the second previously-acquired image based at least in part upon the at least one quality parameter.

4. The computer system of claim 1, wherein the instructions when executed further cause the computing system to:

compare the fourth image data to the third image data using at least one image matching algorithm until a set of image features of the second previously-acquired image is determined to match a set of image features of the fourth image data with at least a minimum level of confidence.

5. A computer-implemented method, comprising:

in a first time period;

obtaining first image data acquired using a camera of a first device;

comparing at least a portion of the first image data to second image data previously-acquired by at least one second device;

identifying, using a first confidence threshold, a first previously-acquired image of the second image data, the first previously-acquired image matching at least a portion of the first image data;

determining that the first device is approaching a global position;

obtaining third image data associated with the global position, the third image data acquired by at least one second device;

in a second time period after the first time period;

obtaining fourth image data using the camera, the fourth image data including a representation of an object;

comparing the fourth image data to the third image data;

identifying, using a second confidence threshold, a second previously-acquired image of the third image data, the second previously-acquired image matching at least a portion of the fourth image data, the second confidence threshold being lower than the first confidence threshold; and

providing actionable content associated with the object, the actionable content enabling at least one action to be performed that is related to the object.

6. The computer-implemented method of claim 5, wherein comparing the fourth image data to the third image data includes at least one of:

performing an object recognition process, or

sending at least a portion of the third image data to the first device for performing the object recognition process.

7. The computer-implemented method of claim 5, wherein determining that the first device is approaching the global position further comprises:

determining one or more locations within an angular view of the camera at a location specified by location data and an orientation specified by orientation data.

8. The computer-implemented method of claim 5, wherein the actionable content includes at least one purchasing option for an item related to the object.

9. The computer-implemented method of claim 5, further comprising:

performing a matching process against media data using the fourth image data; and

adding at least a portion of the fourth image data to the third image data in response to the object being recognized using the media data.

10. The computer-implemented method of claim 5, further comprising:

determining that a matching media file in the third image data has at least one improved quality parameter over a corresponding media file in the fourth image data; and

providing the matching media file for presentation via the first device.

11. The computer-implemented method of claim 10, further comprising:

in a third time period after the second time period;

receiving a higher-resolution version of at least a portion of the fourth image data;

processing at least a portion of the higher-resolution version to extract additional information; and

storing the additional information with the matching media file to assist with future object recognition.

18. The computer-implemented method of claim 5, further comprising:

determining the global position by determining an area having a radial distance within a threshold distance from the first device.

12. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a computing system, cause the computing system to:

in a first time period;

obtain first image data acquired using a camera of a first device;

compare at least a portion of the first image data to second image data previously-acquired by at least one second device;

identify, using a first confidence threshold, a first previously-acquired image of the second image data, the first previously-acquired image matching at least a portion of the first image data;

determine that the first device is approaching a global position;

obtain third image data associated with the global position, the third image data acquired by at least one second device;

in a second time period after the first time period;

obtain fourth image data acquired using the camera, the fourth image data including a representation of an object;

compare the fourth image data to the third image data;

identify, using a second confidence threshold, a second previously-acquired image of the third image data, the second previously-acquired image matching at least a portion of the fourth image data, the second confidence threshold being lower than the first confidence threshold; and

provide actionable content associated with the object, the actionable content enabling at least one action to be performed that is related to the object.

13. The non-transitory computer-readable storage medium of claim 12, wherein the fourth image data includes representations of a plurality of objects, and wherein the instructions when executed further cause the computing system to:

identify, using the second confidence threshold, a plurality of previously-acquired images of the third image data, the plurality of previously-acquired images including portions that match the plurality of objects.

14. The non-transitory computer-readable storage medium of claim 13, wherein the instructions when executed further cause the computing system to:

provide actionable content associated with each of the plurality of objects.

15. The non-transitory computer-readable storage medium of claim 12, wherein the instructions when executed further cause the computing system to:

store at least a portion of the fourth image data as representing the object as a current object at the global position.

16. The non-transitory computer-readable storage medium of claim 12, wherein the instructions when executed further cause the computing system to:

obtain at least a portion of the third image data from an authority associated with the global position, the at least a portion of the third image data given priority over other image data associated with the global position.

17. The non-transitory computer-readable storage medium of claim 12, wherein the instructions when executed further cause the computing system to:

analyze the actionable content to determine the at least one action enabled to be performed; and

cause at least one user-selectable element to be displayed on the first device, selection of the at least one user-selectable element causing a respective action of the at least one action to be performed via the first device.