Method and apparatus for generating compact transcoding hints metadata

• US 7,738,550 B2
• Filed: 03/13/2001
• Issued: 06/15/2010
• Est. Priority Date: 03/13/2000
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kuhn, Peter
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US10/009,119
• Publication Number:
  US 20020157112A1
• Time in Patent Office: 3,381 Days
• Field of Search: None
• US Class Current: 375/240.01
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/234336 by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

An audio/video (or audiovisual, “A/V”) signal processing apparatus and method for extracting a compact representation of a multimedia description and transcoding hints metadata for transcoding between different (e.g., MPEG) compressed content representations, manipulating (e.g., MPEG compressed) bitstream parameters such as frame rate, bit rate, session size, quantization parameters, and picture coding type structure (e.g., group of pictures, or “GOP”), classifying A/V content, and retrieving multimedia information.

Claims

1. A video signal processing method for processing video signals containing video material by a server, the server executing instructions to perform the method comprising:

separating video material from the video signals into segments, the video material comprising a plurality of frames;

calculating a number of new feature points per frame of the video material;

determining, for each segment, whether the number of new feature points exceeds a threshold value;

selecting, based on the determination, one of a plurality of transcoding hints states; and

transcoding the video material according to the selected one of a plurality of transcoding hints states.

2. A video signal processing method according to claim 1, wherein the step of separating the video material into segments comprises the steps of:

using feature points with associated motion vectors;

tracking the feature points; and

determining a new video segment for transcoding based on a number of the feature points that could not be tracked from one frame to a next frame.

3. A video signal processing method according to claim 1, wherein the step of transcoding the video material comprises the steps of:

receiving a first bitstream of compressed image data having a first GOP structure;

extracting transcoding hints metadata from the first bitstream;

utilizing the transcoding hints metadata associated to the first bitstream to facilitate transcoding; and

outputting a second bitstream.

4. A video signal processing method according to claim 3, wherein the step of transcoding the Video material further comprises the step of utilizing the transcoding hints metadata associated to temporal segments of the first bitstream to facilitate transcoding.

5. A video signal processing method according to claim 3, wherein the step of transcoding the video material further comprises the step of utilizing the transcoding hints metadata associated to spatial segments of the first bitstream to facilitate transcoding.

6. A video signal processing method according to claim 3, wherein the step of transcoding the video material further comprises the step of controlling a bit rate of the second bitstream so that a bit rate of the first bitstream is different from the bit rate of the second bit stream.

7. A video signal processing method according to claim 6, wherein the step of transcoding the Video material further comprises the step of adjusting a size of pictures represented by the first bitstream so that pictures represented by the second bitstream exhibits a size different from the size of the pictures represented by the first bitstream.

8. A video signal processing method according to claim 3, wherein the step of transcoding the video material further comprises the step of adjusting a size of pictures represented by the first bitstream so that pictures represented by the second bitstream exhibit a size different from the size of the pictures represented by the first bitstream.

9. A video signal processing method according to claim 8, wherein the step of transcoding the video material further comprises the step of encoding the pictures represented by the second bitstream as field pictures when the pictures represented by the first bitstream are encoded as frame pictures.

10. A video signal processing method according to claim 8, wherein the step of transcoding the video material further comprises the step of interlacing the pictures represented by the first bitstream when the pictures represented by the first bitstream are received as a progressive sequence so that the pictures represented by the second bitstream are output as an interlaced sequence.

11. A video signal processing method according to claim 3, wherein the step of transcoding the video material further comprises the step of encoding pictures represented by the second bitstream as field pictures when pictures represented by the first bitstream are encoded as frame pictures.

12. A video signal-processing method according to claim 3, wherein the step of transcoding the video material further comprises the step of interlacing pictures represented by the first bitstream when pictures represented by the first bitstream are received as a progressive sequence so that pictures represented by the second bitstream are output as an interlaced sequence.

13. A video signal processing method according to claim 1, further comprising the steps of:

describing transcoding target bitstream parameters;

extracting transcoding hints metadata; and

storing the transcoding hints metadata.

14. A video signal processing method according to claim 13, wherein the step of describing the transcoding target bitstream parameters comprises the steps of:

defining a bit rate of a second bitstream of compressed images;

defining a size of pictures of the second bitstream of compressed images;

defining a number of frames per second of the second bitstream of compressed images;

defining an aspect ratio of a pel of the second bitstream of compressed images;

defining a color depth of each of the pel of the second bitstream of compressed images;

defining whether progressive format is used for the second bitstream of compressed images;

defining whether interlaced format is used for the second bitstream of compressed images;

defining whether frame pictures are used for the second bitstream of compressed images;

defining whether field pictures are used for the second bitstream of compressed images; and

defining a compression method of the second bitstream of compressed images.

15. A video signal processing method according to claim 14, wherein the step of describing the transcoding target bitstream parameters further comprises the step of defining employed compression standards as defined by MPEG (Moving Pictures Expert Group).

16. A video signal processing method according to claim 14, wherein the step of describing the transcoding target bitstream parameters further comprises the step of defining employed compression standards as defined by ITU-T (International Telecommunications Union Technical Standards Group).

17. A video signal processing method according to claim 13, wherein the step of extracting the transcoding hints metadata comprises the steps of:

receiving a first bitstream of compressed image data having a first GOP structure;

obtaining first motion information from the first bitstream;

obtaining texture/edge information of a first segmentation;

obtaining feature points and associated motion information from the first bitstream; and

obtaining region of interest information from the first bitstream.

18. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of storing the first motion information as transcoding hints metadata.

19. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of representing motion-related transcoding hints metadata as parameters of a parametric motion model.

20. A video signal processing method according to claim 19, wherein the step of extracting the transcoding hints metadata further comprises the step of employing the parametric motion model to describe a global motion within subsequent rectangular video frames.

21. A video signal processing method according to claim 19, wherein the step of extracting the transcoding hints metadata further comprises the step of employing the parametric motion model to describe a motion within a defined region of arbitrary shape.

22. A video signal processing method according to claim 21, wherein the parametric motion model is employed to describe the motion within the defined region of arbitrary shape as used within MPEG-4.

23. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of representing motion-related transcoding hints metadata as an array of motion vectors contained in the first bitstream of the compressed image data.

24. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of representing motion-related transcoding hints metadata as an array of motion vectors derived from motion vectors contained in the first bitstream of the compressed image data.

25. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of representing motion-related transcoding hints metadata as a list of feature points with associated motion vectors, which are tracked within subsequent frames.

26. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of representing motion-related transcoding hints metadata as a list of feature points with associated motion vectors, which are tracked within arbitrarily shaped regions, within subsequent frames.

27. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of representing texture-related transcoding hints metadata as one of a list of DCT-coefficients and a measure (one of mean, minimum, maximum, variance, and standard deviation) derived thereof.

28. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of representing edge-related transcoding hints metadata as one of a list of DCT-coefficients and a measure derived thereof, the measure being one of mean, minimum, maximum, variance, or standard deviation.

29. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of representing the feature points and associated motion-related transcoding hints metadata as a list.

30. A video signal processing method according to claim 17, wherein the step of extracting the transcoding hints metadata further comprises the step of representing encoding-complexity-related transcoding hints metadata as a complexity metric derived from the feature points tracked within subsequent frames by using a number of lost and new feature points from one frame to a next frame.

31. A video signal processing method according to claim 13, wherein the step of storing the transcoding hints metadata comprises the step of maintaining a buffer containing transcoding hints metadata for several situations.

32. A video signal processing method according to claim 31, wherein the step of storing the transcoding hints metadata further comprises the step of storing individual general transcoding hints metadata for several target devices.

33. A video signal processing method according to claim 31, wherein the step of storing the transcoding hints metadata further comprises the step of storing general transcoding hints metadata for video segments of varying scene activity.

34. An apparatus for processing supplied video signals, comprising:

a segmenting unit for separating video material into segments, the video material comprising a plurality of frames;

a calculation unit for calculating a number of new features points per frame of the video material;

a determination unit for determining, for each segment, whether the number of new feature points exceeds a threshold value;

a selection unit for selecting, based on the determination, one of a plurality of transcoding hints states; and

a transcoding unit for transcoding the video material according to the selected one of a plurality of transcoding hints states.

35. The apparatus of claim 34, further comprising:

a target buffer for storing at least one description of transcoding target bitstream parameters;

an extraction unit for extracting transcoding hints metadata based on the at least one description;

a buffer for storing the transcoding hints metadata.