Addition of supplemental multimedia content and interactive capability at the client

• US 8,930,561 B2
• Filed: 08/07/2009
• Issued: 01/06/2015
• Est. Priority Date: 09/15/2003
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Russell, Riley R., Zalewski, Gary
• Primary Examiner(s): Gilles, Jude Jean
• Assistant Examiner(s): Means, Jaren M
• Application Number:
  US12/537,606
• Publication Number:
  US 20100306402A1
• Time in Patent Office: 1,978 Days
• Field of Search: 709/231
• US Class Current: 709/231
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/338 using television networks
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/61 using advertising information
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: A63F 13/88 Mini-games executed independently while main games are being loaded
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/513 server hierarchy, e.g. local, regional, national or dedicated for different tasks, e.g. authenticating, billing
• CPC Class Codes: A63F 2300/5506 using advertisements
• CPC Class Codes: A63F 2300/5573 player location
• CPC Class Codes: A63F 2300/572 Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video
• CPC Class Codes: A63F 2300/575 for trading virtual items
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: H04L 65/1089 by adding media; by removing media
• CPC Class Codes: H04L 65/60 : Network streaming of media packets
• CPC Class Codes: H04L 65/612 : for unicast
• CPC Class Codes: H04L 65/613 : for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/765 : intermediate
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04N 21/235 : Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/43074 : of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/435 : Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/47205 : for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/4758 : for providing answers, e.g. voting
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8545 : for generating interactive applications

Abstract

Methods, systems, and computer programs for delivering supplemental content within streamed media are provided. In one embodiment, a method includes receiving a request at a client for a media segment, which includes a plurality of video frames. Further, a stream of the media segment to the client from a media server is obtained and supplemental content (SC) is accessed from storage in the client or from an SC server. The access of the SC from the SC server can be performed before receiving the request, or while obtaining the stream, or after the stream has been obtained. Further, the method includes combining the media segment with the SC at the client, which inserts the SC to add or augment video frames from the media segment. The combination is then displayed to the user.

Claims

1. A method for delivering supplemental content within streamed media, the method comprising:

receiving, at a client, scheduling criteria for addition of supplemental content (SC) at the client, the scheduling criteria including a plurality of SC items, wherein each SC item includes a criteria for insertion including a target demographic and a trigger condition that includes at least a media content type;

obtaining a stream of a media segment at a client from a media server;

receiving SC for insertion in the media segment;

analyzing the media segment against the scheduling criteria of the received SC to determine an insertion place of the SC in the media segment, wherein analyzing includes determining if the SC is for the target demographic and if the media segment satisfies the trigger condition associated with the SC;

combining the media segment with the SC at the client based on the analysis to add or augment a plurality of video frames of the media segment; and

displaying the media segment and the SC at a point in time after the target demographic and the trigger condition are satisfied, wherein a duration of SC display has a first value when the user abstains from interacting with the SC while the SC is displayed at the client, and wherein the duration of SC display is shortened when the user interacts with the SC while the SC is displayed.

2. The method as recited in claim 1, wherein the scheduling criteria further includes one or more of client computing device, client interfaces, user age, user income, user geographic location, time of SC delivery, number of impressions desired for the SC, SC age rating, or client bandwidth.

3. The method as recited in claim 1, wherein the SC is selected from a group consisting of a fixed-length video, an interactive video, an interactive game, an overlay placed on a subset of video frames, a computer program, a photographic display, or an interactive presentation.

4. The method as recited in claim 3, wherein the interactive game is selected from a group consisting of a game where a user says a password to finish displaying the SC, a single player game, a multiplayer game, an off-line game, an online game, an interactive game, a voice-command game, a motion detection game, or an image analysis game.

5. The method as recited in claim 1, wherein the SC display duration has a first value when a user abstains from interacting with the SC while the SC is presented at the client, and wherein the SC display duration is shortened when the user interacts with the SC while the SC is presented.

6. The method as recited in claim 1 further comprising:

sending a SC displayed confirmation to the SC server after the SC is displayed.

7. The method as recited in claim 1 further comprising:

storing downloaded media content at the client for future user viewing;

receiving information on media available on a plurality of servers for download; and

browsing at the client for media available locally or on the plurality of servers.

17. The method as recited in claim 1, wherein displaying the combination includes displaying the SC while buffering the media segment.

18. The method as recited in claim 1 further including:

displaying SC without the media segment after receiving a second request at the client, the second request including a command to display the SC without the media segment.

19. The method as recited in claim 1, wherein displaying the combination of media segment further includes:

displaying simultaneously the media segment with the SC.

20. The method as recited in claim 1 further including:

receiving input from a plurality of users indicating when a commercial in the media segment starts.

21. The method as recited in claim 1 further including:

receiving information from a plurality of users indicating when a user fast forwards reproduction of the media segment while the media segment is being displayed.

22. The method as recited in claim 1, wherein operations of the method are performed by a computer program when executed by one or more processors, the computer program being embedded in a non-transitory computer-readable storage medium.

24. The method as recited in claim 1, wherein combining the media segment with the SC further includes:

inserting the SC where the media content includes one or more of a fade to black, a period of silence, or a variation in sound properties.

8. A system for delivering supplemental content within streamed media, the system comprising:

a supplemental content (SC) server;

wherein the SC server selects SC and transfers SC and SC instructions to a client after the client notifies the SC server of a request received from a user for a media segment downloadable from one or more independent media servers;

wherein the client combines the media segment with the SC by inserting the SC to add or augment video frames within the media segment based on the SC instructions, wherein the SC instructions include a predetermined time period for displaying the SC, wherein the client determines a place for inserting the SC in the media segment based on characteristics of the SC and based on the predetermined time period, wherein the SC is combined with the media segment by placing the SC immediately after a commercial embedded in the media segment, and wherein the client displays the media segment with the SC.

9. The system as recited in claim 8, wherein the SC server transfers a same SC to a plurality of clients, wherein the SC instructions for the same SC include inserting into the media segments the same SC at different points at each client from the plurality of clients depending on available insertion windows in each media segment and depending on the same SC being displayed within the predetermined time period, the available insertion windows defining where SC can be inserted within each media segment.

10. The system as recited in claim 8, wherein the media server provides media segment metadata, the metadata being selected from a group consisting of a place for insertion of SC in the media segment, information on existing embedded commercials in the media segment, a command to add SC, a command to replace a section of the media segment with SC, or a command to delete some video frames in the media segment.

11. The system as recited in claim 8, wherein the SC is combined with the media segment by placing the SC immediately before the media segment.

12. The system as recited in claim 8, wherein displaying the media segment includes showing a status bar on a display indicating location of commercials in the media segment and inserted SC.

13. The system as recited in claim 8 further including:

a peripheral for interfacing the client with a user, the peripheral being selected from a group consisting of a game controller, a remote control, a microphone, a camera facing the user, a keyboard, or a mouse.

14. The system as recited in claim 8, wherein the client is selected from a group consisting of a game console, a personal computer, a laptop computer, a phone, a PDA, a set-top box, or a TV with a programmable module.

15. The system as recited in claim 8, wherein the SC server further includes:

a user database with information about a plurality of users;

an SC database holding a plurality of downloadable SC modules;

a media metadata database with information about media segments available for download; and

a partner database with information about media content owners, content distributors and advertisers.

16. The system as recited in claim 8, wherein the client further includes:

a content buffer for buffering media segments being downloaded;

a media library for storing media content and information about media content available for download; and

a SC manager for managing the displaying of SC.

23. A method for delivering supplemental content within streamed media, the method comprising:

receiving a request at a client for a media segment, the media segment including a plurality of video frames;

obtaining a stream of the media segment at the client sent from a media server;

receiving from the media server supplemental content (SC) and SC instructions for insertion of the SC in the media segment after obtaining at least part of the stream;

combining the media segment with the SC at the client according to the SC instructions, the combining including inserting the SC to add or augment video frames of the media segment; and

displaying the combination of media segment and SC, wherein displaying the combination further includes;

providing an option to a user to interact with the SC while displaying the SC;

determining if the user interact with the SC; and

terminating the display of the SC when the user interacts with the SC, wherein the display duration has a first value when the user abstains from interacting with the SC while the SC is displayed at the client, and wherein the SC display duration is shortened when the user interacts with the SC while the SC is displayed.

25. A method comprising:

receiving, at a client, media content in response to a request from a user having a user account for accessing the media content, the media content including one or more first commercial messages embedded in the media content;

receiving, at the client, second commercial content and scheduling criteria for the second commercial content, the scheduling criteria including at least a target demographic and a trigger condition for a media content type;

identifying a user demographic of the user based on the user account and a content type associated with the media content;

identifying, at the client, a location in the media content for inserting the second commercial content based on the user demographic, the target demographic, the content type associated with the media content, and the trigger condition in the scheduling criteria for the media content type; and

displaying, at the client, the media content and the second commercial content based on the identifying, wherein a duration of display of the second commercial content has a first value when the user abstains from interacting with the second commercial content while the second commercial content is displayed at the client, and wherein the duration of display of the second commercial content is shortened when the user interacts with the second commercial content while the second commercial content is displayed, wherein operations of the method are executed by a processor.

26. The method as recited in claim 25, wherein the commercial content is a game, wherein displaying the commercial content includes enabling user interaction to achieve a predefined result in the game, wherein achieving the predefined result enables a change in display time of the commercial content before resuming displaying the media content.