Method and device for handling multiple video streams using metadata

• US 9,167,176 B2
• Filed: 07/05/2006
• Issued: 10/20/2015
• Est. Priority Date: 07/18/2005
• Status: Active Grant

• Current Assignee: InterDigital CE Patent Holdings SAS (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Winter, Marco, Gandolph, Dirk, Peters, Hartmut, Ostermann, Ralf, Hoerentrup, Jobst, Schewzow, Andrej, Janssen, Uwe
• Primary Examiner(s): Le, Rong
• Application Number:
  US11/988,951
• Publication Number:
  US 20090115901A1
• Time in Patent Office: 3,394 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/6543 for forcing some client operations, e.g. recording remote booting in general G06F9/4416
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen
• CPC Class Codes: H04N 7/165 Centralised control of user terminal subsequent to an upstream request signal H04N7/17345; Registering at central by two-way working H04N7/17309

Abstract

A method and device for dynamically overlaying two or more video streams uses specialized metadata that are attached to at least one of the video streams.

Claims

1. Method for overlaying two or more video streams dynamically as picture-in-picture, wherein metadata are associated to at least one of the video streams by playlist file, comprising

retrieving from a pre-recorded storage medium and displaying a first video stream, the first video stream being one of said video streams;

extracting said metadata from the playlist file of a second video stream, wherein the playlist file and the second video stream are retrieved from a local storage within a player, the local storage being different from the pre-recorded storage medium;

detecting that the extracted metadata defines how to overlay the second video stream to said first video stream, wherein said metadata contains at least scaling information and transparency information for the second video stream, time synchronization information, time stamps, and position data defining a display position and area for overlaying, and a reference to the first video stream, and said metadata further contains at least format information of at least the second video stream and projection information, wherein the projection information defines at least one of resolution and aspect ratio;

defining a 3-dimensional model;

calculating a projection of the 3-dimensional model according to said projection information;

including data of said projection of the 3-dimensional model into the second video stream; and

overlaying at least the second video stream to the first video stream as defined by said metadata, wherein the overlaying results in a portion of the first video stream being not visible.

2. Method according to claim 1, wherein said time synchronization information specifies a start time and an end time for overlaying the second video stream to the first video stream.

3. Method according to claim 1, wherein said display area is relative to the size of the display on which the second video stream is overlaid to the first video stream.

4. Method according to claim 1, wherein said metadata contain priorities for the position data, and wherein in the case of collisions the position data with the higher priority is applied.

5. Method according to claim 1, wherein said metadata further contain content information and/or audio properties of the video stream to which it is associated.

6. Method according to claim 1, wherein said metadata contain color range information, flip information and/or rotation information for the second video stream.

7. Method according to claim 1, wherein said metadata contain special effects data, for example fade in/fade out data, colour change data, frame data for a frame boundary or an indication of an overlapping method to be used.

8. Method according to claim 1, wherein said metadata define said transparency information data only for particular display types.

9. Method according to claim 1, wherein said metadata define said special effects data only for particular display types.

10. Method according to claim 1, wherein the second video stream contains difference video data that change the appearance of the first video stream on pixel level.

11. Method according to claim 1, wherein said metadata may be modified at presentation time.

12. Method according to claim 11, further comprising the steps of

a. detecting that the second video stream is selected, and

b. upon the detecting, automatically increasing the display area for the second video stream.

13. Method according to claim 1, wherein the metadata further contains special effects data, and wherein a hierarchy exists which defines that the time synchronization information controls the position data and the special effects metadata.

14. Method according to claim 1, wherein first metadata associated with the first video stream indicate a brightness level of a display area, and second metadata associated with the second video stream contain a brightness level indication of the second video stream, further comprising the step of

automatically generating position information based on said brightness level of said display area of the first video stream and the brightness level indication of the second video stream, wherein the second video stream is overlaid to the first video stream according to the generated position information.

15. Method according to claim 1, wherein the pre-recorded storage medium is an optical disc.

16. A non-transitory storage medium having recorded thereon computer-executable instructions that when executed by a computer cause the computer to overlay two or more video streams dynamically as picture-in-picture, wherein metadata are associated to at least one of the video streams by a playlist file, the instructions, when executed by the computer:

retrieve from a first storage medium a first video stream being one of said two or more video streams, and displaying the first video stream;

extract said metadata from the playlist file of a second video stream, wherein the playlist file and the second video stream are retrieved from a local storage within a player, the local storage being different from the first storage medium;

detect that the extracted metadata defines how to overlay the second video stream to said first video stream, wherein said metadata contains at least scaling information and transparency information for the second video stream, time synchronization information, time stamps, and position data defining a display position and area for overlaying, and a reference to the first video stream, and said metadata further contains at least projection information of at least the second video stream and projection information, wherein the projection information defines at least one of resolution and aspect ratio;

define a 3-dimensional model;

calculate a projection of the 3-dimensional model according to said projection information;

including data of said projection of the 3-dimensional model into the second video stream; and

overlay at least the second video stream to the first video stream as defined by said metadata, wherein the overlaying results in a portion of the first video stream being not visible.