METHOD AND APPARATUS FOR ENABLING A SEARCHABLE HISTORY OF REAL-WORLD USER EXPERIENCES

• US 20130036134A1
• Filed: 06/11/2012
• Published: 02/07/2013
• Est. Priority Date: 08/03/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Neven, Hartmut, Petrou, David, Adam, Hartwig, Smullyan, Jacob
• Granted Patent: US 9,087,058 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/53 Querying
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 2212/171 Portable consumer electronics, e.g. mobile phone
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

A method and apparatus for enabling a searchable history of real-world user experiences is described. The method may include capturing media data by a mobile computing device. The method may also include transmitting the captured media data to a server computer system, the server computer system to perform one or more recognition processes on the captured media data and add the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match. The method may also include transmitting a query of the user to the server computer system to initiate a search of the history or real-world experiences, and receiving results relevant to the query that include data indicative of the media data in the history of real-world experiences.

Claims

1. A computer-implemented method comprising:

capturing media data by a mobile computing device;

transmitting the captured media data to a server computer system, the server computer system to perform one or more recognition processes on the captured media data and add the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match;

transmitting a query of the user to the server computer system to initiate a search of the history or real-world experiences; and

receiving results relevant to the query that include data indicative of the media data in the history of real-world experiences.

2. The computer-implemented method of claim 1, wherein the mobile computing device is a user wearable computing device with a head mounted display.

3. The computer-implemented method of claim 1, wherein the query is a voice activated query received via a microphone of the mobile computing device.

4. The computer-implemented method of claim 3, further comprising:

parsing audio data for the voice activated query to locate one or more keywords in the audio data; and

forming a text based keyword query for transmission to the server computer system.

5. The computer-implemented method of claim 1, further comprising:

receiving one or more settings that indicate when media data is to be captured by the mobile computing device;

monitoring real-world conditions at the mobile computing device; and

initiating capture of the media data based on the one or more settings and the monitored real-world conditions.

6. The computer-implemented method of claim 5, wherein the one or more settings include initiation of media data capture by the mobile computing device when the mobile computing device is proximate to a specific location.

7. The computer-implemented method of claim 5, wherein the one or more settings include initiation of media data capture by the mobile computing device when a current location of the mobile computing device is proximate to a popular location, wherein the location is a popular location when a number of other users that have captured media data at the location exceeds a threshold.

8. The computer-implemented method of claim 5, wherein the one or more settings include how frequently the media data is to be captured.

9. The computer-implemented method of claim 1, wherein the media data includes one or more of digital image data and digital audio data.

10. A non-transitory computer readable storage medium including instructions that, when executed by a processor, cause the processor to perform a method comprising:

capturing media data by a mobile computing device;

transmitting the captured media data to a server computer system, the server computer system to perform one or more recognition processes on the captured media data and add the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match;

transmitting a query of the user to the server computer system to initiate a search of the history or real-world experiences; and

receiving results relevant to the query that include data indicative of the media data in the history of real-world experiences.

11. The non-transitory computer readable storage medium of claim 10, wherein the mobile computing device is a user wearable computing device with a head mounted display.

12. The non-transitory computer readable storage medium of claim 10, wherein the query is a voice activated query received via a microphone of the mobile computing device.

13. The non-transitory computer readable storage medium of claim 12, further comprising:

parsing audio data for the voice activated query to locate one or more keywords in the audio data; and

forming a text based keyword query for transmission to the server computer system.

14. The non-transitory computer readable storage medium of claim 10, further comprising:

receiving one or more settings that indicate when media data is to be captured by the mobile computing device;

monitoring real-world conditions at the mobile computing device; and

initiating capture of the media data based on the one or more settings and the monitored real-world conditions.

15. The non-transitory computer readable storage medium of claim 14, wherein the one or more settings include initiation of media data capture by the mobile computing device when the mobile computing device is proximate to a specific location.

16. The non-transitory computer readable storage medium of claim 14, wherein the one or more settings include initiation of media data capture by the mobile computing device when a current location of the mobile computing device is proximate to a popular location, wherein the location is a popular location when a number of other users that have captured media data at the location exceeds a threshold.

17. The non-transitory computer readable storage medium of claim 14, wherein the one or more settings include how frequently the media data is to be captured.

18. The non-transitory computer readable storage medium of claim 10, wherein the media data includes one or more of digital image data and digital audio data.

19. A mobile computing device comprising:

a memory; and

a processor coupled with the memory to

capture media data,

transmit the captured media data to a server computer system, the server computer system to perform one or more recognition processes on the captured media data and add the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match,

transmit a query of the user to the server computer system to initiate a search of the history or real-world experiences, and

receive results relevant to the query that include data indicative of the media data in the history of real-world experiences.

20. The system of claim 19, wherein the mobile computing device is a user wearable computing device with a head mounted display.

21. The system of claim 19, wherein the query is a voice activated query received via a microphone of the mobile computing device, and wherein the processor is to parse audio data for the voice activated query to locate one or more keywords in the audio data, and form a text based keyword query for transmission to the server computer system.

22. The system of claim 19, wherein the processor is to receive one or more settings that indicate when media data is to be captured by the mobile computing device, monitor real-world conditions at the mobile computing device, and initiate capture of the media data based on the one or more settings and the monitored real-world conditions.

23. The system of claim 22, wherein the one or more settings include initiation of media data capture by the mobile computing device when the mobile computing device is proximate to a specific location.

24. The system of claim 22, wherein the one or more settings include initiation of media data capture by the mobile computing device when a current location of the mobile computing device is proximate to a popular location, wherein the location is a popular location when a number of other users that have captured media data at the location exceeds a threshold.

25. The system of claim 22, wherein the one or more settings include how frequently the media data is to be captured.

26. The system of claim 19, wherein the media data includes one or more of digital image data and digital audio data.

27. A computer-implemented method comprising:

receiving media data, at a server computer system, captured by a mobile computing device;

performing one or more recognition processes on the captured media data;

adding the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match;

receiving a query from the mobile computing device to initiate a search of the history or real-world experiences; and

transmitting results relevant to the query that include data indicative of the media data in the history of real-world experiences.

28. A non-transitory computer readable storage medium including instructions that, when executed by a processor, cause the processor to perform a method comprising:

receiving media data, at a server computer system, captured by a mobile computing device;

performing one or more recognition processes on the captured media data;

adding the captured media data to a history of real-world experiences of a user of the mobile computing device when the one or more recognition processes find a match;

receiving a query from the mobile computing device to initiate a search of the history or real-world experiences; and

transmitting results relevant to the query that include data indicative of the media data in the history of real-world experiences.

29. A system comprising:

a mobile computing device to

capture media data,

transmit the captured media data to a server computer system,

transmit a query of a user to the server computer system to initiate a search of a history or real-world experiences, and

receive results relevant to the query that include data indicative of the media data in the history of real-world experiences; and

a server computer system to

receive the media data captured by the mobile computing device

perform one or more recognition processes on the captured media data,

add the captured media data to the history of real-world experiences of the user of the mobile computing device when the one or more recognition processes find a match,

receive the query from the mobile computing device to initiate a search of the history or real-world experiences, and

transmit results relevant to the query that include the data indicative of the media data in the history of real-world experiences.