GENERATING METADATA FOR USER EXPERIENCES

• US 20130177296A1
• Filed: 11/29/2012
• Published: 07/11/2013
• Est. Priority Date: 11/15/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Geisner, Kevin A., Markovic, Relja, Latta, Stephen G., McCulloch, Daniel
• Application Number:
  US13/689,413
• Publication Number:
  US 20130177296A1
• Time in Patent Office: Days
• US Class Current: 386/241
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 9/80 Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

A system and method for efficiently managing life experiences captured by one or more sensors (e.g., video or still camera, image sensors including RGB sensors and depth sensors). A “life recorder” is a recording device that continuously captures life experiences, including unanticipated life experiences, in image, video, and/or audio recordings. In some embodiments, video and/or audio recordings captured by a life recorder are automatically analyzed, tagged with a set of one or more metadata, indexed, and stored for future use. By tagging and indexing life recordings, a life recorder may search for and acquire life recordings generated by itself or another life recorder, thereby allowing life experiences to be shared minutes or even years later.

Claims

1. A method for managing data captured by a recording device, comprising:

acquiring a recording of user experiences captured throughout one or more days by the recording device;

generating context information, the context information including information associated with a user of the recording device, the context information including information associated with the recording device, the context information generated by one or more sensors;

identifying a particular situation from the recording;

detecting a tag event, the step of detecting includes automatically determining whether one or more rules associated with the recording device are satisfied by the context information and the particular situation, said one or more rules are configured for determining when to generate a set of one or more metadata tags for the recording;

automatically generating a set of one or more metadata tags for the recording responsive to the step of detecting, each of the one or more metadata tags including one or more keywords that describe the recording related to a location associated with the recording device, a timestamp associated with the recording, an event associated with the user, and/or a situation associated with the recording, the set of one or more metadata tags allowing subsequent search of the recording by the user or another user associated with one or more different recording devices; and

storing the set of one or more metadata tags in the recording device or in a remote storage device.

2. The method of claim 1, wherein:

the step of generating context information includes acquiring location information associated with the recording device, acquiring time information associated with the recording, acquiring calendar information associated with the user of the recording device, and acquiring biometric information associated with the user of the recording device; and

the location information includes a GPS location of the recording device, locations obtained from one or more cell towers, co-location from another computing device, locations based on one or more landmarks recognized, and locations identified from known map data, the time information includes a particular date and time associated with the recording, the calendar information includes a description of a calendar event associated with the user of the recording device, the biometric information includes a description of biological and physiological properties associated with the user of the of the recording device.

3. The method of claim 1, wherein:

the step of identifying a particular situation is performed using a voice recognition, a facial recognition, an eye-tracking system for detecting gaze of the user, gesture recognitions, a machine learning, and/or a pattern recognition technique on life recording; and

the particular situation is one of recognition of a particular person, recognition of a particular place, recognition of a particular emotion, or recognition of a particular object.

4. The method of claim 1, wherein:

the step of detecting includes comparing the particular situation with the one or more rules, the step detecting includes comparing the context information with the one or more rules.

5. The method of claim 1, further comprising:

identifying a first portion of the recording during which the particular situation is identified from the recording; and

generating a recording summary for the first portion of the recording responsive to the step of identifying, the recording summary including the first portion of the recording.

6. The method of claim 5, wherein:

the step of automatically generating a set of one or more metadata tags includes automatically generating a set of one or more metadata tags associated with the first portion of the recording based on the context information generated and the particular situation identified.

7. The method of claim 6, further comprising:

generating a metadata tag file associated with the recording device, wherein;

the metadata file is stored locally in the life recorder, in a remote storage device, or in the cloud.

8. The method of claim 7, wherein:

the metadata tag file stores searchable metadata tags related to a location, a timestamp, an event, and/or a situation; and

the searchable metadata tags include the set of one or more metadata tags associated with the first portion of the recording.

9. The method of claim 8, wherein:

the metadata tag file stores the set of one or more metadata tags associated with the first portion of the recording and a corresponding link to the recording summary that comprises the first portion of the recording.

10. The method of claim 8, wherein:

the metadata tag file stores the set of one or more metadata tags associated with the first portion of the recording and corresponding time stamps associated with the first portion of the recording; and

the corresponding time stamps include a start time stamp and an end time stamp indicating a particular time duration in the recording when the particular situation was identified.

11. The method of claim 8, further comprising:

receiving search criteria;

searching the metadata tags stored in the metadata tag file based on the search criteria in real-time or upon a triggering event to identify a portion of the recording based on the metadata tags; and

reporting the portion of the recording identified, the step of reporting includes displaying the portion of the recording identified on a display device.

12. An article of manufacture comprising:

a computer-readable storage medium having stored therein a computer program executable by a processor, the computer program comprising instructions for;

capturing one of a video recording, an audio recording, or an audiovisual recording of user experiences associated with a user by one or more recording devices;

analyzing the recording, the step of analyzing includes detecting a particular situation from the recording and comparing the particular situation detected with one or more requirements to determine when to generate a set of one or more metadata tags for the recording;

identifying a first portion of the recording during which the particular situation is detected responsive to the step of analyzing; and

automatically determining a set of one or more metadata tags to be associated with the first portion of the recording, the step automatically determining a set of one or more metadata tags includes generating one or more key phrases describing the first portion of the recording such that the recording can be searched based on the one or more key phrases.

13. The article of manufacture of claim 12, further comprising:

generating context information associated with the recording, the context information including location and time information associated with the recording devices, the context information includes calendar information and biometric information associated with the user; and

the particular situation is one of recognition of a particular person, recognition of a particular place, recognition of a particular emotion, or recognition of a particular object;

wherein

the one or more recording devices include a depth sensing device to capture depth information.

14. The article of manufacture claim 12, further comprising:

generating a metadata tag file, wherein;

the metadata tag file stores one or more searchable metadata tags related to a location, a timestamp, an event, and a situation; and

the searchable metadata tags include the set of one or more metadata tags associated with the first portion of the recording.

15. The article of manufacture claim 14, wherein:

the metadata tag file stores the set of one or more metadata tags associated with the first portion of the recording and time stamps associated with the first portion of the recording; and

the time stamps include a start time stamp and an end time stamp indicating a particular time duration in the recording when the particular situation was identified.

16. The article of manufacture claim 14, further comprising:

searching the metadata tags stored in the metadata tag file to identify a portion of the recording based on search criteria; and

reporting the portion of the recording identified, the step of reporting includes displaying the portion of the recording identified on a display.

17. A system comprising:

one or more video devices, the one or more video devices capture a recording of user experiences associated with a user;

a memory, the memory stores the recording of user experiences; and

one or more processors, the one or more processors in communication with the one or more video devices and the memory, the one or more processors receive one or more rules for determining when to generate a set of one or more metadata tags for the recording, the one or more processors analyze the recording to detect context information associated with the recording and to identify a particular situation from the recording, the one or more processors determine when to generate a set of one or more metadata tags for the recording by comparing the set of one or more rules with the context information and the particular situation, the one or more processors generate a set of one or more metadata tags to be associated with a portion of the recording during which the particular situation was identified.

18. The system of claim 17, wherein:

the one or more video devices include a depth sensor to capture depth information; and

the one or more processors analyze the recording to identify the particular situation using a voice recognition, a facial recognition, an eye-tracking system for detecting gaze of the user, gesture recognitions, a machine learning, and/or a pattern recognition technique on the recording; and

the particular situation includes one of recognition of a particular person, recognition of a particular place, recognition of a particular emotion, or recognition of a particular object.

19. The system of claim 17, wherein:

the one or more processors analyze the recording to detect the context information by processing information captured by one or more sensors, the one or more sensors including at least one image sensor to capture digital and video recordings, at least one biometric sensor to capture biometric information associated with the user, and at least one environmental sensor to capture environmental information associated with the recording; and

the context information includes information associated with the user and information associated with the one or more video devices.

20. The system of claim 17, wherein:

the one or more processors generate a metadata tag file to store one or more searchable metadata tags related to a location, a timestamp, an event, and a situation associated with the recording, the searchable metadata tags stored in the metadata tag file include the set of one or more metadata tags associated with the portion of the recording.