Annotating and indexing broadcast video for searchability

• US 9,788,071 B2
• Filed: 04/01/2015
• Issued: 10/10/2017
• Est. Priority Date: 11/03/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Mason, Joanna, Wheeler, Craig J., Av, Daniel, Siebert, Michael Ahmed Sengan, Smith, Gregory M., Garcia, Antonio Vargas
• Primary Examiner(s): SHANG, ANNAN Q
• Application Number:
  US14/676,747
• Publication Number:
  US 20160127795A1
• Time in Patent Office: 923 Days
• Field of Search: 725 37-116, 709219
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/8545 for generating interactive applications

Abstract

A method providing searchable streaming video broadcasts via an interactive media server, the interactive media server generates a live broadcast video stream for each of a plurality of interactive media sessions and a video provider server providing a connection to the live broadcast video stream for viewing, the method comprising receiving an interactive media events stream for each interactive media session from the interactive media server, generating statistics data for each interactive media session based upon the events stream, and generating a searchable index of a directory of the live broadcast video streams, the searchable index including a plurality of entries, each entry including (a) a link to the live broadcast video stream for a corresponding interactive media session provided by the video provider server, and (b) metadata associated with the corresponding interactive media session, the metadata generated based upon the statistics data for the corresponding interactive media session.

Claims

1. A method for providing searchable streaming video broadcasts, the method comprising:

at an interactive media server, generating a live broadcast video stream for each of a plurality of interactive media sessions generated in response to a plurality of users interacting with interactive media applications on interactive media devices, wherein a state of the interactive media application and statistics produced by user input are associated with the live broadcast video stream and a video provider server provides a connection to the live broadcast video stream for viewing;

at a statistics server, receiving an interactive media events stream for each interactive media session from the interactive media server, the interactive media events stream generated by the user interacting with the interactive media application that generates interactive events including a state of the interactive media application in the interactive media events stream based on and in response to user input with the interactive media application, the interactive media events stream including at least one interactive media event;

generating data for each interactive media session based upon user interaction with the interactive media application, the data including the state of the interactive media application and statistics data about the interactive events in the interactive media events stream to be used for searching content available for viewing in near real-time and at a future time by a plurality of users; and

at an indexing server, generating a searchable index of a directory of live broadcast video streams or recordings of live broadcast video streams stored in a database, the searchable index searchable by data generated by user interaction with interactive media applications and including a plurality of entries, each entry including (a) a link to the live broadcast video stream or a recorded live broadcast video stream for a corresponding interactive media session provided by the video provider server, and (b) metadata associated with the corresponding interactive media session, the metadata being generated based upon the data for the corresponding interactive media session including the state of the interactive media application and the statistics data and used for subsequent searching by the plurality of users.

2. The method of claim 1, further comprising, providing a search interface configured to:

receive a search query from a computing device;

look up one or more terms in the search query in the searchable index; and

send to the computing device a displayable list of one or more search results including at least one of the links corresponding to one of the live broadcast video streams provided by the video provider server.

3. The method of claim 2, where the search query includes filters and search options related to the metadata.

4. The method of claim 2, further comprising programmatically sending a list of links to live broadcast video streams based on user inputs generated via the interactive media devices from the indexing server to the interactive media devices.

5. The method of claim 1, where a portion of the statistics data that is generated is determined by applying statistics rules set by a developer.

6. The method of claim 5, further comprising altering the statistics rules in response to receiving a rule change request from a developer computing device.

7. The method of claim 1, where the live broadcast video stream includes at least one of microphone input data and video input data from the interactive media device.

8. The method of claim 1, where the metadata is time stamped corresponding to a temporal location in the live broadcast video stream.

9. The method of claim 1, further comprising maintaining on a directory server the directory of the live broadcast video streams provided by the video provider server for each interactive media session, the directory including a list of links to live broadcast video streams for each interactive media session on the video provider server.

10. The method of claim 1, where the step of generating the searchable index is implemented in near real-time.

11. A system for providing searchable video data, the system comprising:

an interactive media server to generate a plurality of interactive media sessions and a live broadcast video stream for each of the interactive media sessions in response to a plurality of users interacting with interactive media applications on interactive media devices, wherein a state of the interactive media application and statistics produced by user input are associated with the live broadcast video stream;

a video provider server to provide connections to the video data for viewing;

one or more computing devices each having at least one processor and associated memory, the processor configured to execute the following software programs;

a statistics engine configured to;

receive an interactive media events stream for each interactive media session from the interactive media server, the interactive media events stream generated by the user interacting with the interactive media application that generates interactive events including a state of the interactive media application in the interactive media events stream based on and in response to user input with the interactive media application, the interactive media events stream including at least one interactive media event;

generate data for each interactive media session based upon user interaction with the interactive media application, the data including the state of the interactive media application and statistics data about the interactive events in the interactive media events stream to be used for searching content available for viewing in near real-time and at a future time by a plurality of users; and

an indexing engine on an indexing server configured to generate a searchable index of a directory of the live broadcast video streams or recordings of live broadcast video streams stored in a database, the searchable index searchable by data generated by user interaction with interactive media applications and including a plurality of entries, each entry including (a) a link to the live broadcast video stream or a recorded live broadcast video stream for a corresponding interactive media session provided by the video provider server, and (b) metadata associated with the corresponding interactive media session, the metadata being generated based upon the data for the corresponding interactive media session including the state of the interactive media application and the statistics data and used for subsequent searching by the plurality of users.

12. The system of claim 11, where the indexing server is configured to provide a search interface configured to:

receive a search query from a computing device;

look up one or more terms in the search query in the searchable index; and

send to the computing device a displayable list of one or more search results including at least one of the links corresponding to one of the live broadcast video streams provided by the video provider server.

13. The system of claim 11, where the indexing server is further configured to inhibit indexing of a selected portion of the statistics data when the interactive media session is discontinued.

14. The system of claim 11, where the metadata is time stamped corresponding to a temporal location in the corresponding live broadcast video stream.

15. The system of claim 11, further comprising a directory server configured to maintain the directory of the live broadcast video streams provided by the video provider server for each interactive media session, the directory including a list of links to live broadcast video streams for each interactive media session on the video provider server.

16. The system of claim 11, where the statistics data includes one or more of map type, weapon type, tool type, points data, and level data.

17. The system of claim 11, where the indexing engine is configured to update the searchable index periodically based on the live broadcast video streams.

18. A method for providing searchable streaming video broadcasts, the method comprising:

at a plurality of interactive media devices, implementing interactive media sessions that include generation of interactive media events streams, each media event stream being based upon a user interacting with a corresponding interactive media device via an interactive media application;

at an interactive media server;

receiving the interactive media events streams; and

sending video data to a video provider server;

at a media statistics server, generating data based on the interactive media events stream including at least one state of the interactive media application and statistics data related to the interactive media session;

at an indexing server, generating a searchable index of a directory of live broadcast video streams including a plurality of entries, each entry including a link to the live broadcast video stream for a corresponding interactive media session and metadata associated with the corresponding interactive media session based on the statistics data; and

receiving search queries from a plurality of users from computing devices at a search interface to locate entries in the searchable index and access links to the live broadcast video streams.