Addition of Supplemental Multimedia Content and Interactive Capability at the Client

• US 20100306402A1
• Filed: 08/07/2009
• Published: 12/02/2010
• Est. Priority Date: 09/15/2003
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America (Sony Group Corp.)
• Inventors: Russell, Riley R., Zalewski, Gary
• Granted Patent: US 8,930,561 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/338 using television networks
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/61 using advertising information
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: A63F 13/88 Mini-games executed independently while main games are being loaded
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/513 server hierarchy, e.g. local, regional, national or dedicated for different tasks, e.g. authenticating, billing
• CPC Class Codes: A63F 2300/5506 using advertisements
• CPC Class Codes: A63F 2300/5573 player location
• CPC Class Codes: A63F 2300/572 Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video
• CPC Class Codes: A63F 2300/575 for trading virtual items
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: H04L 65/1089 by adding media; by removing media
• CPC Class Codes: H04L 65/60 : Network streaming of media packets
• CPC Class Codes: H04L 65/612 : for unicast
• CPC Class Codes: H04L 65/613 : for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/765 : intermediate
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04N 21/235 : Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/43074 : of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/435 : Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/47205 : for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/4758 : for providing answers, e.g. voting
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8545 : for generating interactive applications

Abstract

Methods, systems, and computer programs for delivering supplemental content within streamed media are provided. In one embodiment, a method includes receiving a request at a client for a media segment, which includes a plurality of video frames. Further, a stream of the media segment to the client from a media server is obtained and supplemental content (SC) is accessed from storage in the client or from an SC server. The access of the SC from the SC server can be performed before receiving the request, or while obtaining the stream, or after the stream has been obtained. Further, the method includes combining the media segment with the SC at the client, which inserts the SC to add or augment video frames from the media segment. The combination is then displayed to the user.

Claims

1. A method for delivering supplemental content within streamed media, the method comprising:

receiving a request at a client for a media segment, the media segment including a plurality of video frames;

obtaining a stream of the media segment to the client from a media server;

accessing supplemental content (SC) from storage of the client or from an SC server, the accessing SC being done before receiving the request, or while obtaining the stream, or after obtaining the stream;

combining the media segment with the SC at the client, the combining including inserting the SC to add or augment video frames from the media segment; and

displaying the combination of media segment and SC.

2. The method as recited in claim 1, further comprising:

selecting the SC based on the request or based on SC server delivery scheduling criteria;

wherein the SC server delivery scheduling criteria is selected from a group consisting of client computing device, client interfaces, user age, user income, user geographic location, time of SC delivery, number of impressions desired for the SC, SC age rating, or client bandwidth.

3. The method as recited in claim 1, wherein the SC is selected from a group consisting of a fixed-length video, an interactive video, an interactive game, an overlay placed on a subset of video frames, a computer program, a photographic display, or an interactive presentation.

4. The method as recited in claim 3, wherein the interactive game is selected from a group consisting of a game where the user says a password to finish displaying the SC, a single player game, a multiplayer game, an off-line game, an online game, an interactive game, a voice-command game, a motion detection game, or an image analysis game.

5. The method as recited in claim 1, wherein a duration of display for the SC is controlled by a user interacting with content presented at the client.

6. The method as recited in claim 1 further comprising:

sending a SC displayed confirmation to the SC server after the SC is displayed.

7. The method as recited in claim 1 further comprising:

storing downloaded media content at the client for future user viewing;

receiving information on media available on a plurality of servers for download; and

browsing at the client for media available locally or on the plurality of servers.

8. A system for delivering supplemental content within streamed media, the system comprising:

a supplemental content (SC) server; and

a plurality of clients on a network;

wherein each client from the plurality of clients downloads a media segment for a received request from a user interfacing through each client, the media segments for the plurality of clients being downloaded from a plurality of independent media servers, each media segment including a plurality of video frames;

wherein each client notifies the SC server of the received request;

wherein the SC server selects and transfers SC and SC instructions to each client;

wherein each client combines the media segment with the SC by inserting the SC to add or augment video frames within the media segment for displaying the media segment with the SC.

9. The system as recited in claim 8, wherein the SC server transfers a same SC to a subset of the plurality of clients, wherein the SC instructions for the same SC include inserting into the media segments the same SC at different points at each client depending on available insertion windows in each media segment and depending on the same SC being displayed within a predetermined time period, the available insertion windows defining where SC can be inserted within each media segment.

10. The system as recited in claim 8, wherein the media server provides media segment metadata, the metadata being selected from a group consisting of a place for insertion of SC in the media segment, information on existing embedded commercials in the media segment, a command to add SC, a command to replace a section of the media segment with SC, or a command to delete some video frames in the media segment.

11. The system as recited in claim 8, wherein the SC is combined with the media segment by placing the SC immediately before the media segment.

12. The system as recited in claim 8, wherein the SC is combined with the media segment by placing the SC immediately after a commercial embedded in the media segment.

13. The system as recited in claim 8, wherein displaying the media segment includes showing a status bar on a display indicating location of commercials in the media segment and inserted SC.

14. The system as recited in claim 8 further including:

a peripheral for interfacing each client with a user, the peripheral being selected from a group consisting of a game controller, a remote control, a microphone, a camera facing the user, a keyboard, or a mouse.

15. The client device as recited in claim 8, wherein the clients are selected from a group consisting of a game console, a personal computer, a laptop computer, a phone, a PDA, a set-top box, or a TV with a programmable module.

16. The system as recited in claim 8, wherein the SC server further includes:

a user database with information about the plurality of users;

an SC database holding SC downloadable to the plurality of users;

a media metadata database with information about media segments available for download; and

a partner database with information about media content owners, content distributors and advertisers.

17. The system as recited in claim 8, wherein the each client further includes:

a content buffer for buffering media segments being downloaded;

a media library for storing media content and information about media content available for download; and

a SC manager for managing the displaying of SC.

18. A computer program embedded in a computer-readable storage medium, when executed by one or more processors, for delivering supplemental content within streamed media, the computer program comprising:

program instructions for receiving a request at a client for a media segment, the media segment including a plurality of video frames;

program instructions for obtaining a stream of the media segment to the client from a media server;

program instructions for accessing supplemental content (SC) from storage of the client or from an SC server, the accessing SC being done before receiving the request, or while obtaining the stream, or after obtaining the stream;

program instructions for combining the media segment with the SC at the client, the combining including inserting the SC to add or augment video frames from the media segment; and

program instructions for displaying the combination of media segment and SC.

19. The computer program as recited in claim 18, wherein the SC is downloaded to the client before receiving the request at the client.

20. The computer program as recited in claim 19, wherein displaying the combination includes displaying the SC while buffering the media segment.

21. The computer program as recited in claim 18, further including:

program instructions for displaying SC without the media segment after receiving a second request at the client, the second request including a command to display the SC without the media segment.

22. The computer program as recited in claim 18, wherein the program instructions for displaying the combination of media segment further includes:

displaying simultaneously the media segment with the SC.

23. The computer program as recited in claim 22, where the SC includes one or more elements selected from a group consisting of an advertisement, a game score, game instructions, a video progress bar, live news, or sport updates.

24. The computer program as recited in claim 18, further including:

program instructions for receiving input from the plurality of users indicating when a commercial in the media segment starts.

25. The computer program as recited in claim 18, further including:

program instructions for receiving information from the plurality of users indicating when a user fast forwards while the media segment is being displayed.

26. A system for delivering supplemental content within streamed media, the system comprising:

means for receiving a request at a client for a media segment, the media segment including a plurality of video frames;

means for obtaining a stream of the media segment to the client from a media server;

means for accessing supplemental content (SC) from storage of the client or from an SC server, the accessing SC being done before receiving the request, or while obtaining the stream, or after obtaining the stream;

means for combining the media segment with the SC at the client, the combining including inserting the SC to add or augment video frames from the media segment; and

means for displaying the combination of media segment and SC.