Method and apparatus for enabling a searchable history of real-world user experiences

• US 9,087,058 B2
• Filed: 06/11/2012
• Issued: 07/21/2015
• Est. Priority Date: 08/03/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Neven, Hartmut, Petrou, David, Smullyan, Jacob, Adam, Hartwig
• Primary Examiner(s): Wong, Leslie
• Application Number:
  US13/493,423
• Publication Number:
  US 20130036134A1
• Time in Patent Office: 1,135 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/53 Querying
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 2212/171 Portable consumer electronics, e.g. mobile phone
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

A method and apparatus for enabling a searchable history of real-world user experiences is described. The method may include capturing media data by a mobile computing device. The method may also include transmitting the captured media data to a server computer system, the server computer system to perform one or more recognition processes on the captured media data and add the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match. The method may also include transmitting a query of the user to the server computer system to initiate a search of the history or real-world experiences, and receiving results relevant to the query that include data indicative of the media data in the history of real-world experiences.

Claims

1. A computer-implemented method comprising:

receiving one or more settings of a mobile computing device that indicate when media data is to be captured by the mobile computing device;

initiating capture of media data by the mobile computing device based on the one or more settings and one or more real-world conditions associated with the mobile computing device;

transmitting the captured media data to a server computer system, the server computer system to perform one or more recognition processes on the captured media data and add the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match;

transmitting a query of the user to the server computer system to initiate a search of the history of real-world experiences; and

receiving results relevant to the query that include data indicative of the captured media data in the history of real-world experiences,

wherein the one or more settings comprise initiation of media data capture by the mobile computing device when a current location of the mobile computing device is proximate to a popular location, wherein the location is a popular location when a number of other users that have captured media data at the location exceeds a threshold.

2. The computer-implemented method of claim 1, wherein the mobile computing device is a user wearable computing device with a head mounted display.

3. The computer-implemented method of claim 1, wherein the query is a voice activated query received via a microphone of the mobile computing device.

4. The computer-implemented method of claim 3, further comprising:

parsing audio data for the voice activated query to locate one or more keywords in the audio data; and

forming a text based keyword query for transmission to the server computer system.

5. The computer-implemented method of claim 1, wherein the one or more settings comprise initiation of media data capture by the mobile computing device when the mobile computing device is proximate to a specific location.

6. The computer-implemented method of claim 1, wherein the one or more settings comprise how frequently the media data is to be captured.

7. The computer-implemented method of claim 1, wherein the media data comprises one or more of digital image data and digital audio data.

8. A non-transitory computer readable storage medium including instructions that, when executed by a processor, cause the processor to perform a method comprising:

receiving one or more settings of a mobile computing device that indicate when media data is to be captured by the mobile computing device;

initiating capture of media data by the mobile computing device based on the one or more settings and one or more real-world conditions associated with the mobile computing device;

transmitting the captured media data to a server computer system, the server computer system to perform one or more recognition processes on the captured media data and add the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match;

transmitting a query of the user to the server computer system to initiate a search of the history of real-world experiences; and

receiving results relevant to the query that include data indicative of the captured media data in the history of real-world experiences,

wherein the one or more settings comprise initiation of media data capture by the mobile computing device when a current location of the mobile computing device is proximate to a popular location, wherein the location is a popular location when a number of other users that have captured media data at the location exceeds a threshold.

9. The non-transitory computer readable storage medium of claim 8, wherein the mobile computing device is a user wearable computing device with a head mounted display.

10. The non-transitory computer readable storage medium of claim 8, wherein the query is a voice activated query received via a microphone of the mobile computing device.

11. The non-transitory computer readable storage medium of claim 10, further comprising:

parsing audio data for the voice activated query to locate one or more keywords in the audio data; and

forming a text based keyword query for transmission to the server computer system.

12. The non-transitory computer readable storage medium of claim 8, wherein the one or more settings comprise initiation of media data capture by the mobile computing device when the mobile computing device is proximate to a specific location.

13. The non-transitory computer readable storage medium of claim 8, wherein the one or more settings comprise how frequently the media data is to be captured.

14. The non-transitory computer readable storage medium of claim 8, wherein the media data comprises one or more of digital image data and digital audio data.

15. A mobile computing device comprising:

a memory; and

a processor coupled with the memory to

receive one or more settings of a mobile computing device that indicate when media data is to be captured by the mobile computing device,

initiate capture of media data based on the one or more settings and one or more real-world conditions associated with the mobile computing device,

transmit the captured media data to a server computer system, the server computer system to perform one or more recognition processes on the captured media data and add the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match,

transmit a query of the user to the server computer system to initiate a search of the history of real-world experiences, and

receive results relevant to the query that include data indicative of the captured media data in the history of real-world experiences,

wherein the one or more settings comprise initiation of media data capture by the mobile computing device when a current location of the mobile computing device is proximate to a popular location, wherein the location is a popular location when a number of other users that have captured media data at the location exceeds a threshold.

16. The system of claim 15, wherein the mobile computing device is a user wearable computing device with a head mounted display.

17. The system of claim 15, wherein the query is a voice activated query received via a microphone of the mobile computing device, and wherein the processor is to parse audio data for the voice activated query to locate one or more keywords in the audio data, and form a text based keyword query for transmission to the server computer system.

18. The system of claim 15, wherein the one or more settings comprise initiation of media data capture by the mobile computing device when the mobile computing device is proximate to a specific location.

19. The system of claim 15, wherein the one or more settings comprise how frequently the media data is to be captured.

20. The system of claim 15, wherein the media data comprises one or more of digital image data and digital audio data.

21. A computer-implemented method comprising:

receiving media data, at a server computer system, captured by a mobile computing device, the media data captured based on one or more setting received by the mobile device indicative of when media data is to be captured and based on one or more real-world conditions associated with the mobile computing device, wherein the one or more settings comprise initiation of media data capture by the mobile computing device when a current location of the mobile computing device is proximate to a popular location, wherein the location is a popular location when a number of other users that have captured media data at the location exceeds a threshold;

performing one or more recognition processes on the captured media data;

adding the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match;

receiving a query from the mobile computing device to initiate a search of the history of real-world experiences; and

transmitting results relevant to the query that include data indicative of the captured media data in the history of real-world experiences.

22. A non-transitory computer readable storage medium including instructions that, when executed by a processor, cause the processor to perform a method comprising:

receiving media data, at a server computer system, captured by a mobile computing device, the media data captured based on one or more setting received by the mobile device indicative of when media data is to be captured and based on one or more real-world conditions associated with the mobile computing device, wherein the one or more settings comprise initiation of media data capture by the mobile computing device when a current location of the mobile computing device is proximate to a popular location, wherein the location is a popular location when a number of other users that have captured media data at the location exceeds a threshold;

performing one or more recognition processes on the captured media data;

adding the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match;

receiving a query from the mobile computing device to initiate a search of the history of real-world experiences; and

transmitting results relevant to the query that include data indicative of the captured media data in the history of real-world experiences.