Smart streaming

• US 10,737,183 B1
• Filed: 10/27/2016
• Issued: 08/11/2020
• Est. Priority Date: 10/27/2015
• Status: Active Grant

• Current Assignee: Bluestack Systems, Inc.
• Original Assignee: Bluestack Systems, Inc.
• Inventors: Sharma, Shashi Kant, Li, Sabrina Marie Waun-Mei, Larson, Julia, Reese, Jr., David P.
• Primary Examiner(s): McCulloch, Jr., William H
• Assistant Examiner(s): Doshi, Ankit B
• Application Number:
  US15/336,528
• Time in Patent Office: 1,384 Days
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/352 involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
• CPC Class Codes: A63F 13/497 Partially or entirely replaying previous game actions
• CPC Class Codes: A63F 13/86 Watching games played by other players

Abstract

Embodiments of the present invention are directed to ancillary game data. Ancillary game data, including game state events and/or visual/audio elements, is generated from and stored in association with raw or original game play footage by a game state data (GSD) service. The ancillary game data is available for subsequently composition back with the corresponding original media stream, thereby generating a smart media stream, for viewers to consume. Game state events are available for searching and sorting archived media streams or for performing general data mining.

Claims

1. A non-transitory computer readable medium storing instructions that, when executed by a server, cause the server to perform a method comprising:

receiving ancillary game data associated with a raw video game stream that is generated at a video game-executing processor-implemented end-user streaming device, wherein raw video game stream data corresponding to the generated raw video game stream does not include the ancillary game data and is stored remotely from the server;

generating one or more correlations between the received ancillary game data and game play footage identifiers in the raw video game stream data;

maintaining by the server, the ancillary game data associated with the raw video game stream in a data store that is communicatively coupled with the server; and

responsive to receiving user input identifying an item of the ancillary game data, implementing at a processor-implemented end-user device, a game stream data display event, wherein implementing the game stream data display event comprises retrieving and displaying, on a display device coupled with the processor-implemented end-user device, selected raw video game stream data, and wherein selection of the selected raw video game stream data for display is based on a game play footage identifier of the game play footage identifiers that is correlated with the identified item of the ancillary game data;

wherein the ancillary game data associated with the raw video game stream comprises at least one of;

one or more game state events, each game state event comprising an event notification corresponding to an event within a video game to which the raw video game stream data corresponds; and

one or more badges, each badge comprising a user notification corresponding to the video game; and

wherein the game stream data display event comprises synchronizing display of a first set of game play footage from the raw video game stream with display of a second set of game play footage from another raw video game stream on the display device coupled with the processor-implemented end-user device, wherein the first set of game play footage and the second set of game play footage are respectively correlated with a common game state event or a common badge.

2. The non-transitory computer readable medium as claimed in claim 1, wherein the ancillary game data associated with the generated raw video game stream comprises one or more recorded game state events, each of the one or more recorded game state events comprising a processor-generated event notification, wherein recording any of the one or more game state events comprises:

identifying through image processing implemented on the generated raw video game stream, a game state event within the generated raw video game stream; and

recording the identified game state event as a second item of the ancillary game data corresponding to the generated raw video game stream.

3. The non-transitory computer readable medium as claimed in claim 2, wherein recording any of the one or more game state events comprises correlating the identified game state event with at least one of a time stamp value or a frame index value corresponding to the raw video game stream.

4. The non-transitory computer readable medium as claimed in claim 1, wherein the game stream data display event comprises:

generating a composed video stream that is distinct from the raw video game stream, wherein the composed video stream combines game play footage from the raw video game stream with the ancillary game data associated with the raw video game stream; and

displaying the composed video stream on the processor-implemented end-user device.

5. The non-transitory computer readable medium as claimed in claim 1, wherein the game stream data display event comprises:

performing a search based on a received search query, for identifying any of the raw video game stream data generated at one or more processor-implemented end-user streaming devices that is associated with ancillary game data that matches the received search query; and

displaying game play footage from the raw video game stream data that has been identified based on the performed search.

6. The non-transitory computer readable medium as claimed in claim 1, wherein the game stream data display event includes:

performing a sort function based on a received sort input, wherein performing the sort function comprises organizing a plurality of raw video game streams generated at one or more processor-implemented end-user streaming devices, and said organizing is based on ancillary data corresponding to each of the plurality of raw video game streams.

7. A method for streaming game play data, the method comprising, at a server:

receiving ancillary game data associated with a raw video game stream that is generated at a video game-executing processor-implemented end-user streaming device, wherein raw video game stream data corresponding to the raw video game stream does not include the ancillary game data, and is stored remotely from the server;

generating one or more correlations between the received ancillary game data and game play footage identifiers in the raw video game stream data;

maintaining by the server, the ancillary game data associated with the raw video game stream in a data store that is communicatively coupled with the server; and

responsive to receiving user input identifying an item of the ancillary game data, implementing at a processor-implemented end-user device, a game stream data display event, wherein implementing the game stream data display event comprises retrieving and displaying, on a display device coupled with the processor-implemented end-user device, selected raw video game stream data, and wherein selection of the selected raw video game stream data for display is based on a game play footage identifier of the game play footage identifiers that is correlated with the identified item of the ancillary game data;

wherein the ancillary game data associated with the raw video game stream comprises at least one of;

one or more game state events, each game state event comprising an event notification corresponding to an event within a video game to which the raw video game stream data corresponds; and

one or more badges, each badge comprising a user notification corresponding to the video game;

and wherein the game stream data display event comprises synchronizing display of a first set of game play footage from the raw video game stream with display of a second set of game play footage from another raw video game stream on the display device coupled with the processor-implemented end-user device, wherein the first set of game play footage and the second set of game play footage are respectively correlated with a common game state event or a common badge.

8. The method as claimed in claim 7, wherein the ancillary game data associated with the generated raw video game stream comprises one or more recorded game state events, each of the one or more recorded game state events comprising a processor-generated event notification, wherein recording any of the one or more of the game state events comprises:

identifying through image processing implemented on the generated raw video game stream, a game state event within the generated raw video game stream; and

recording the identified game state event as a second item of the ancillary game data corresponding to the generated raw video game stream.

9. The method as claimed in claim 8, wherein recording any of the one or more game state events further comprises correlating the identified game state event with at least one of a time stamp value or a frame index value corresponding to the raw video game stream.

10. The method as claimed in claim 7, wherein the game stream data display event comprises:

generating a composed video stream that is distinct from the raw video game stream, wherein the composed video stream combines game play footage from the raw video game stream with the ancillary game data associated with the raw video game stream; and

displaying the composed video stream on the processor-implemented end-user device.

11. The method as claimed in claim 7, wherein the game stream data display event comprises:

performing a search based on a received search query, for identifying any of the raw video game stream data generated at one or more processor-implemented end-user streaming devices that is associated with ancillary game data that matches the received search query; and

displaying game play footage from the raw video game stream data that has been identified based on the performed search.

12. The method as claimed in claim 7, wherein the game stream data display event includes:

performing a sort function based on a received sort input, wherein performing the sort function comprises organizing a plurality of raw video game streams generated at one or more processor-implemented end-user streaming devices, and said organizing is based on ancillary data corresponding to each of the plurality of raw video game streams.

13. A system for streaming game play data, the system comprising at least one server configured for:

receiving ancillary game data associated with a raw video game stream that is generated at a video game-executing processor-implemented end-user streaming device, wherein raw video game stream data corresponding to the generated raw video game stream does not include the ancillary game data, and is stored remotely from the server;

generating one or more correlations between the received ancillary game data and game play footage identifiers in the raw video game stream data;

maintaining by the server, the ancillary game data associated with the raw video game stream in a data store that is communicatively coupled with the server; and

responsive to receiving user input identifying an item of the ancillary game data, implementing at a processor-implemented end-user device, a game stream data display event, wherein implementing the game stream data display event comprises retrieving and displaying, on a display device coupled with the processor-implemented end-user device, selected raw video game stream data, and wherein selection of the selected raw video game stream data for display is based on a game play footage identifier of the game play footage identifiers that is correlated with the identified item of the ancillary game data;

wherein the ancillary game data associated with the raw video game stream comprises at least one of;

one or more game state events, each game state event comprising an event notification corresponding to an event within a video game to which the raw video game stream data corresponds; and

one or more badges, each badge comprising a user notification corresponding to the video game;

and wherein the game stream data display event comprises synchronizing display of a first set of game play footage from the raw video game stream with display of a second set of game play footage from another raw video game stream on the display device coupled with the processor-implemented end-user device, wherein the first set of game play footage and the second set of game play footage are respectively correlated with a common game state event or a common badge.

14. The system as claimed in claim 13, wherein the ancillary game data associated with the generated raw video game stream comprises one or more recorded game state events, each of the one or more recorded game state events comprising a processor-generated event notification, wherein recording any of the one or more game state events comprises:

identifying through image processing implemented on the generated raw video game stream, a game state event within the generated raw video game stream; and

recording the identified game state event as a second item of the ancillary game data corresponding to the generated raw video game stream.

15. The system as claimed in claim 14, wherein recording any of the one or more game state events further comprises correlating the identified game state event with at least one of a time stamp value or a frame index value corresponding to the raw video game stream.

16. The system as claimed in claim 13, wherein the game stream data display event comprises:

generating a composed video stream that is distinct from the raw video game stream, wherein the composed video stream combines game play footage from the raw video game stream with the ancillary game data associated with the raw video game stream; and

displaying the composed video stream on the processor-implemented end-user device.

17. The system as claimed in claim 13, wherein the game stream data display event comprises:

performing a search based on a received search query, for identifying any of the raw video game stream data generated at one or more processor-implemented end-user streaming devices that is associated with ancillary game data that matches the received search query; and

displaying game play footage from the raw video game stream data that has been identified based on the performed search.

18. The system as claimed in claim 13, wherein the game stream data display event includes:

performing a sort function based on a received sort input, wherein performing the sort function comprises organizing a plurality of raw video game streams generated at one or more processor-implemented end-user streaming devices, and said organizing is based on ancillary data corresponding to each of the plurality of raw video game streams.