Context dependent recognition

• US 8,694,522 B1
• Filed: 03/28/2012
• Issued: 04/08/2014
• Est. Priority Date: 03/28/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Pance, Aleksander
• Primary Examiner(s): Nguyen, Phong
• Application Number:
  US13/432,274
• Time in Patent Office: 741 Days
• Field of Search: 707/706, 707/758, 707/913, 707/914, 707/915, 707/916
• US Class Current: 707/758
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

Context data can be used to determine the current context and/or to predict the future context of a user. When the disclosed technology knows of the user'"'"'s likely context, it can prepare for object recognition (e.g., image recognition, speech recognition, etc.) by (downloading and) locally storing (i.e., holding) object data related to the context. This allows for the object recognition to be performed locally and for any additional information about the object to be provided without communication over a network, thereby reducing resources such as time, cost, and processing power. If, however, the object data related to the context is not available locally, such object data can still be downloaded from a server/cloud. In some embodiments, if a likely future context is predicted and the object data related to that future context is not available locally, the object data can be downloaded from a server/cloud prior to the future context.

Claims

1. A method implemented by a computing device comprising at least one processor, the method comprising:

obtaining context data using one or more sensors of the computing device, the context data being associated with a current environment in which the computing device is situated;

determining a context of the current environment in which the computing device is situated, wherein the determining is based at least in part upon the context data and includes predicting a future context;

determining object data associated with the determined context is not stored locally on the computing device;

downloading, from a storage device remote from the computing device, the object data associated with the determined context, the downloaded object data being stored locally on the computing device;

receiving an identification inquiry including information about an object to be identified;

searching the locally stored object data for a potential match for the information about the object in the identification inquiry; and

providing a response in reply to the identification inquiry when at least one potential match is located in the locally stored object data.

2. The method of claim 1, wherein the at least one potential match is located when selected object data for at least one object in the object data stored locally on the computing device matches information in the identification inquiry within at least one determined matching criterion.

3. The method of claim 1, further comprising:

searching for a potential match over a network if the at least one potential match is not found by searching the locally stored object data.

4. The method of claim 1, wherein the at least one potential match includes object data for at least one of an image, a video file, an audio file, or an olfactory object.

5. A method implemented by a computing device comprising at least one processor, the method comprising:

obtaining context data using at least one device sensor of the computing device;

determining a context based at least in part upon the context data, the context data relating to a current environment in which the computing device is situated, wherein the determining includes predicting a future context;

determining object data associated with the determined context is not stored locally on the computing device;

downloading, from one or more devices external to the computing device, the object data associated with the determined context, the downloaded object data being stored locally on the computing device;

receiving information for an object to be identified;

searching the locally stored object data for a potential match for the object; and

providing identifying information to a user of the computing device when at least one potential match is located using the locally stored object data.

6. The method of claim 5, further comprising:

determining a current context upon receiving information for the object to be identified, wherein the locally stored object data is searched when the current context corresponds to at least a portion of the locally stored object data.

7. The method of claim 5, further comprising:

searching object data stored on a storage device remote from the computing device when the at least one potential match is not able to be located using the locally stored object data.

8. The method of claim 5, wherein the object data associated with the context includes data for one or more objects associated with the current environment in which the computing device is situated.

9. The method of claim 5, wherein the at least one device sensor includes at least one of a camera, a microphone, a light sensor, an olfactory sensor, a biometric sensor, a location sensor, an orientation sensor, or a climate sensor.

10. The method of claim 5, wherein the at least one potential match includes object data for at least one of an image, a video file, an audio file, or an olfactory object.

11. The method of claim 10, wherein the at least one potential match includes at least one speech file.

12. A computing device, comprising:

a processor;

at least one device sensor; and

a memory device including instructions that, when executed by the processor, cause the computing device to;

obtain context data using the at least one device sensor;

determine a context based at least in part upon context data obtained by the computing device, the context data relating to a current environment in which the computing device is situated, wherein the determining includes predicting a future context;

determine object data associated with the determined context is not stored locally on the computing device;

download, from one or more devices external to the computing device, the object data associated with the determined context, the downloaded object data being stored locally on the computing device;

receive information for an object to be identified;

search the locally stored object data for a potential match for the object; and

provide identifying information to a user of the computing device when at least one potential match is located using the locally stored object data.

13. A system, comprising:

at least one processor; and

a memory device including instructions that, when executed by the at least one processor, cause the system to;

receive context data from a client computing device, the context data acquired using at least one sensor of the client computing device;

attempt to determine a context associated with the context data, wherein the context data is related to a current environment in which the client computing device is situated, wherein determining the context includes predicting a future context;

determine object data for the determined context and send the object data to be downloaded and stored locally on the client computing device, the object data useful in matching objects identified to the client computing device;

receive over a network a request to identify an object when a potential match is unable to be located using the locally stored object data on the client computing device;

locate a potential match among object data stored on the system; and

transmit information for the potential match to the client computing device.

14. The system of claim 13, wherein the instructions cause the system to attempt to update one or more sets of object data stored locally on the client computing device when new object data is available.

15. A non-transitory computer-readable storage medium including instructions, the instructions when executed by a processor of a computing device causing the computing device to:

obtain context data using one or more sensors of the computing device;

determine a context based at least in part upon context data obtained by the computing device, the context data relating to a current environment in which the computing device is situated, wherein determining the context includes predicting a future context;

determine object data associated with the determined context is not stored locally on the computing device;

download, from one or more devices external to the computing device, the object data associated with the determined context, the downloaded object data being stored locally on the computing device;

receive information for an object to be identified;

search the locally stored object data for a potential match for the object; and

provide a response to a user of the computing device when at least one potential match is located.

16. The non-transitory computer-readable storage medium of claim 15, wherein the instructions cause the computing device to search object data stored on a storage device remote from the computing device, wherein the search is for at least one potential match for the object to be identified when the at least one potential match is not located using the locally stored object data.

17. The non-transitory computer-readable storage medium of claim 15, wherein the at least one potential match includes object data for at least one of an image, a video file, an audio file, or an olfactory object.

18. The non-transitory computer-readable storage medium of claim 17, wherein the at least one potential match includes at least one speech file.