Method and apparatus for enabling event-based media data capture

• US 9,137,308 B1
• Filed: 01/09/2012
• Issued: 09/15/2015
• Est. Priority Date: 01/09/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Petrou, David, Garcia-Barrio, Laura, Neven, Hartmut, Adam, Hartwig
• Primary Examiner(s): DOAN, DUYEN MY
• Application Number:
  US13/346,529
• Time in Patent Office: 1,345 Days
• Field of Search: 709/217, 348/135, 348/148
• US Class Current: 1/1
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/185 by embedding added-value information into content, e.g. geo-tagging

Abstract

A method and apparatus for enabling media data capture based on a real-world user experience is described herein. According to embodiments of the invention, media data is captured by a mobile computing device and transmitted to a server computer system in response to identifying one or more events from sensor data. The server computer system adds the captured media data, along with data identifying the event that initiated the capture of the media data, to a history of real-world experiences of a user of the mobile computing device. The server computer system may further perform one or more recognition processes on the captured media data and, when the one or more recognition processes find a match, associate the recognized items to the captured media data.

Claims

1. A method comprising:

receiving sensor data from a sensor included in a mobile computing device;

identifying a person, object, or place from the sensor data;

identifying an event from the sensor data based, at least in part, on whether the identified person, object, or place is included in a history of real-world experiences of a user of the mobile computing device;

initiating the mobile computing device to capture media data in response to identifying the event from the sensor data;

associating the media data captured by the mobile computing device with the event; and

transmitting media capture event data identifying the event and the media data associated with the event to a server computer system for adding the media capture event data and the media data associated with the event to the history of real-world experiences of the user of the mobile computing device.

2. The method of claim 1, wherein the mobile computing device comprises a user wearable computing device including a head mounted display.

3. The method of claim 1, wherein the sensor data includes audio data, and identifying the person, object, or place from the sensor data comprises parsing the audio data to identify one or more key words from the audio data.

4. The method of claim 1, wherein the sensor data includes image data, and identifying the person, object, or place from the sensor data comprises performing a recognition process to identify the object from the image data.

5. The method of claim 1, wherein the sensor data comprises a media data stream, and initiating the mobile computing device to capture media data in response to identifying the event from the sensor data comprises storing a segment of the media data stream.

6. The method of claim 1, wherein the sensor data includes location data and the event comprises the sensor being proximate to a specific location.

7. The method of claim 1, further comprising:

transmitting a query of the user of the mobile computing device to the server computer system for initiating a search of one or more events included in the history of real-world experiences of the user; and

receiving results relevant to the query that include data indicative of the one or more events and media data associated with the one or more events included in the history of real-world experiences of the user.

8. The method of claim 1, wherein the media data includes one or more of digital image data and digital audio data.

9. A non-transitory computer readable storage medium including instructions that, when executed by a processor, cause the processor to perform a method comprising:

receiving sensor data from a sensor included in a mobile computing device;

identifying a person, object, or place from the sensor data;

identifying an occurrence of an event from the sensor data based, at least in part, on whether the identified person, object, or place is included in a history of real-world experiences of a user of the mobile computing device;

automatically initiating the mobile computing device to capture media data for adding to the history of real-world experiences of the user of the mobile computing device, in response to identifying the occurrence of the event from the sensor data;

associating the media data captured by the mobile computing device with the event; and

transmitting media capture event data identifying the event and the media data associated with the event to a server computer system for adding the media capture event data and the media data associated with the event to the history of real-world experiences of the user of the mobile computing device.

10. The non-transitory computer readable storage medium of claim 9, wherein the mobile computing device comprises a user wearable computing device including a head mounted display.

11. The non-transitory computer readable storage medium of claim 9, wherein the sensor data includes audio data, and identifying the person, object, or place from the sensor data comprises parsing the audio data to identify one or more key words from the audio data.

12. The non-transitory computer readable storage medium of claim 9, wherein the sensor data includes image data, and identifying the person, object, or place from the sensor data comprises performing a recognition process to identify the object from the image data.

13. The non-transitory computer readable storage medium of claim 9, wherein the sensor data comprises a media data stream, and initiating the mobile computing device to capture media data in response to identifying the event from the sensor data comprises storing a segment of the media data stream.

14. The non-transitory computer readable storage medium of claim 9, wherein the sensor data includes location data and the event comprises the sensor being proximate to a specific location.

15. The non-transitory computer readable storage medium of claim 9, the method further comprising:

transmitting a query of the user of the mobile computing device to the server computer system for initiating a search of one or more events included in the history of real-world experiences of the user; and

receiving results relevant to the query that include data indicative of the one or more events and media data associated with the one or more events included in the history of real-world experiences of the user.

16. The non-transitory computer readable storage medium of claim 9, wherein the media data includes one or more of digital image data and digital audio data.

17. A system comprising:

a memory;

a processor;

a sensor; and

a capture event module included in the memory and executed via the processor to;

receive sensor data from the sensor;

identify a person, object, or place from the sensor data;

identify an event from the sensor data based, at least in part, on whether the identified person, object, or place is included in a history of real-world experiences of a user of the mobile computing device;

initiate one or more media data files to be captured in response to identifying the event;

associate the captured one or more media data files with the event; and

transmit media capture event data identifying the event and the media data associated with the event to a server computer system for adding the media capture event data and the media data associated with the event to the history of real-world experiences of the user of the system.

18. The system of claim 17, further comprising

a head mounted display;

and

a frame assembly for wearing on a head of the user and to include the sensor and the head mounted display.

19. The system of claim 17, wherein the sensor comprises a microphone to capture audio data, and identifying the person, object, or place from the sensor data comprises parsing the audio data to identify one or more key words from the audio data.

20. The sensor of claim 17, wherein the sensor comprises an image sensor to capture image data, and identifying the person, object, or place from the sensor data comprises performing a recognition process to identify the object from the image data.

21. The system of claim 17, wherein the sensor data comprises a media data stream, and the one or more media data files each comprise a segment of the media data stream.

22. The system of claim 17, wherein the sensor data comprises location data and the event comprises the sensor being proximate to a specific location.

23. The system of claim 17, further comprising a query module included in the memory and executed via the processor to

transmit a query of the user of the system to the server computer system for initiating a search of one or more events included in the history of real-world experiences of the user;

and

receive results relevant to the query that include data indicative of the one or more events and media data associated with the one or more events included in the history of real-world experiences of the user.

24. The system of claim 17, wherein each of the one or more media data files includes one or more of digital image data and digital audio data.