Image metadata creation for improved image processing and content delivery

• US 9,819,974 B2
• Filed: 02/26/2013
• Issued: 11/14/2017
• Est. Priority Date: 02/29/2012
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Kunkel, Timo, Ballestad, Anders, Geddert, Joel N.
• Primary Examiner(s): Sheleheda, James R
• Application Number:
  US14/372,747
• Publication Number:
  US 20150007243A1
• Time in Patent Office: 1,722 Days
• Field of Search: 725 22, 725 25, 725116
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2541 Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/25825 involving client display capabilities, e.g. screen resolution of a mobile phone optimising the visualisation of content during browsing in the Internet G06F16/9577; processing of terminal status or physical abilities in wireless networks H04W8/22; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/835 Generation of protective data, e.g. certificates protecting software against unauthorised usage in a vending or licensing environment G06F21/10
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

Several embodiments of a media processor for the creation of metadata from a set of images are given. The media processor receives a set of image data and computes metadata from the image data. The metadata may comprise statistical data based on the image data; from which the media processor may generate and store a multi-scale version of the metadata. The metadata may comprise statistical attributes derived directly from the image data or statistical attributes derived from image appearance correlates of the image data. The media processor may generate a subsampled set of the metadata and the subsampled set may be stored in an efficient data structure, such as a pyramidal structure. The pyramidal structure may be generated from spatial image information, from temporal image information or from spatial-temporal image information.

Claims

1. A media processor for the creation of metadata from a set of images, said media processor comprising:

an input for a first set of data, said first set of data comprising image data, metadata or image data/metadata associated with said set of images, said set of images comprising a sequence of images; and

a metadata processor being capable of receiving said first set of data and computing a set of metadata from said first set of data, said set of metadata comprising statistical data derived from said sequence of images over a plurality of different temporal intervals and said set of metadata representing a media fingerprint which identifies said set of images without altering said set of images; and

wherein

said metadata processor is capable of generating and storing a multi-scale version of said set of metadata, said multi-scale version including first metadata generated based on statistical attributes associated with said sequence of images over a first temporal interval and second metadata generated based on statistical attributes associated with said sequence of images over a second temporal interval;

said first temporal interval and said second temporal interval are associated with increasing levels of said multi-scale version of metadata; and

each increasing level of said multi-scale version of metadata is associated with an increasing confidence level in said media fingerprint;

said set of metadata further comprises at least one of statistical attributes derived directly from said first set of data and statistical attributes derived from image appearance correlates of said first set of data;

said media processor is further capable of performing display management (DM) processing of a set of image data to be rendered on a target display having a set of display specifications; and

to perform said DM processing, said media processor is operative to compute a first set of statistics from said set of image data over time,

compare a portion of said first set of statistics against said set of display specifications,

test for a match condition for said portion of said first set of statistics against said set of display specifications, and

render a portion of said set of image data upon said target display, said target display comprising said set of display specifications.

2. The media processor as recited in claim 1 wherein said statistical attributes are selected from the group consisting of mean, variance, skewness, kurtosis, statistical moments, histograms, image pyramids, edges, gradients, framerate, resolution and bit depth.

3. The media processor as recited in claim 1 wherein said multi-scale version comprises a subsampled set of said set of metadata wherein said subsampled set of said set of metadata is stored in a data structure.

4. The media processor as recited in claim 3 wherein said data structure comprises a pyramidal structure or a multi-scale structure.

5. The media processor as recited in claim 4 wherein said pyramidal structure comprises pyramids of data derived from temporal image information or pyramids derived from spatial-temporal image information.

6. The media processor as recited in claim 5 wherein said media processor comprises a DM processor, a digital rights management (DRM) processor, a compression processor, a tone mapping (TM) processor, a color appearance (CAM) processor, or a Gamut Mapping processor.

7. The media processor as recited in claim 5 wherein:

said media processor is further capable of performing digital rights management (DRM) processing on a set of test image data against a first pyramid structure derived from a second set of statistics compiled from a legacy set of image data; and

to perform said DRM processing, said media processor is further operative to compute a third set of statistics from said test image data over time,

derive a test pyramid structure from said third set of statistics from said test image data over time,

compare a portion of said test pyramid structure against said first pyramid structure,

test for a match condition from said comparison of said portion of said test pyramid structure against said first pyramid structure, and

return said match condition when said test determines a match within a given threshold.

8. The media processor as recited in claim 7 wherein said media processor is operative to determine the identity of a source film from which said test image data belongs.

9. The media processor as recited in claim 1 wherein to compute said first set of statistics said media processor is operative to compute dynamic range or color gamut of said set of image data over time.

10. The media processor as recited in claim 9 wherein said set of display specifications comprises the dynamic range of said target display or the color gamut of said target display.

11. The media processor as recited in claim 10 wherein said media processor is further operative to test for a match of the dynamic range of a scene of a video with the dynamic range of said target display or the color gamut of a scene of a video with the color gamut of said target display.

12. The media processor as recited in claim 11 wherein said scene at the dynamic range of said video is rendered upon the target display.

13. The media processor as recited in claim 1 wherein:

said media processor is further capable of restoring a master cut of a film, said film derived from a set of source reels and a degraded version; and

to restore said master cut, said media processor is operative to

compute a second set of statistics from said source reels,

compute a third set of statistics from said degraded version,

identify frames from said source reels that comprise frames from said degraded version, and

create a master cut of said film from the frames identified from said source reels.

14. The media processor as recited in claim 13 wherein to create said master cut of said film said media processor is further operative to create an edit decision list (EDL) for said master cut.

15. The media processor as recited in claim 1, wherein said first time interval and said second time interval overlap.

16. The media processor as recited in claim 1, wherein one of said first time interval and said second time interval is included within the other of said first time interval and said second time interval.

17. The media processor as recited in claim 1, wherein said first metadata is generated using values from at least two frames of image data from said sequence of images over said first temporal interval.

18. The media processor as recited in claim 1, wherein said media processor is a component of a consumer end-point device.

19. A media processor for the creation of metadata from a set of images, said media processor comprising:

an input for source data associated with said set of images, said set of images comprising a sequence of images; and

a metadata processor operative to

receive said source data,

generate first metadata based on statistics of said sequence of images associated with a first temporal interval,

generate second metadata based on statistics of said sequence of images associated with a second temporal interval, said second temporal interval being different than said first temporal interval such that said first metadata and said second metadata are derived based on different amounts of said source data, and

generate a multi-scale version of metadata based on said first metadata and said second metadata, said multi-scale version of said metadata representing a media fingerprint which identifies said set of images without altering said set of images; and

wherein

said first temporal interval and said second temporal interval are associated with increasing levels of said multi-scale version of metadata; and

each increasing level of said multi-scale version of metadata is associated with an increasing confidence level in said media fingerprint.

20. The media processor of claim 19, wherein said media processor is further operative to:

generate third metadata associated with a first spatial frequency and based on said source data; and

generate said multi-scale version of metadata based on said first metadata, said second metadata, and said third metadata.