Moving picture editing method, moving picture editing system and storing medium with moving picture editing programs stored therein

• US 7,209,635 B2
• Filed: 11/07/2001
• Issued: 04/24/2007
• Est. Priority Date: 11/08/2000
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Hatabu, Atsushi
• Primary Examiner(s): Tran; Thai Q.
• Assistant Examiner(s): TOPGYAL, GELEK W
• Application Number:
  US09/986,015
• Publication Number:
  US 20020081094A1
• Time in Patent Office: 1,994 Days
• Field of Search: 37524012-24016, 37524026-24029, 386/4, 386/33, 386/52, 386111-112, 348/434.1, 348/435.1
• US Class Current: 386/283
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A moving picture editing method is disclosed, in which if one or two reference frames utilized for motion compensation of P or B frame, constituting a part of moving picture before editing and not deleted in edition so as to constitute moving picture after editing, is subject to error generation in processing although not subject to lack in edition, motion vectors in a range centered on each motion vector V before editing of the P or B frame, are searched for the motion vector V′ after editing corresponding to the minimum value of the difference between the motion compensated picture MC (X, V) before edition and motion compensated picture MC (X′, V′) after editing.

Claims

1. A moving picture editing method for editing compression coded moving pictures by utilizing inter-frame prediction based on motion compensation, the method comprising:

determining if reference frames which are utilized for motion compensation of a subject frame before editing and which are not deleted during editing are subject to error generation in processing; and

searching motion vectors in a range centered on each pre-edit motion vector of the subject frame to find a post-edit motion vector corresponding to a minimum value of a difference between a pre-edit motion compensated picture and a post-edit motion compensated picture.

6. The moving picture editing method as in any one of claims 1–

3 or 5, wherein candidates for each post-edit motion vector are limited to motion vectors present in a range centered on the pre-edit motion vector.

7. The moving picture editing method as in any one of claims 1–

3 or 5, wherein candidates for each post-edit motion vector are limited to motion vectors which are equal to the pre-edit motion vector or motion vectors present in a range centered on the pre-edit motion vector and having a non-integer coordinate value as a horizontal or vertical coordinate value.

15. A computer-readable storage medium, in which moving picture editing computer programs for realizing the functions set forth in any one of claims 1–

5 or 8–

12 in a personal computer are stored.

2. A moving picture editing method for editing compression coded moving pictures by utilizing inter-frame prediction based on motion compensation, the method comprising:

determining if reference frames which are utilized for motion compensation of a subject frame before editing and which are not deleted during editing are subject to error generation in processing;

searching motion vectors in a range centered on each pre-edit motion vector of the subject frame to find a post-edit motion vector corresponding to a minimum value of a difference between a pre-edit motion compensated picture and a post-edit motion compensated picture;

obtaining the post-edit motion compensated picture by motion compensation utilizing the post-edit motion vector;

coding a picture difference obtained as a result of subtraction of the post-edit motion compensated picture from pre-edit decoded data of the subject frame to obtain post-edit picture difference coded data; and

coding the post-edit picture difference coded data and the post-edit motion vectors to obtain post-edit coded data of the subject frame.

3. A moving picture editing method for editing compression coded moving pictures by utilizing inter-frame prediction based on motion compensation, the method comprising:

determining if reference frames which are utilized for motion compensation of a subject frame before editing and which are not deleted during editing are subject to error generation in processing;

determining if at least one of the reference frames meets at least a condition that error generation in the reference frame results from re-encoding during editing or a condition that coded data has changed and a motion vector has changed during editing such that a number of macroblocks having a difference between a pre-edit motion vector and a post-edit motion vector exceeds a threshold numbers;

searching motion vectors in a range centered on each pre-edit motion vector of the subject frame to find the post-edit motion vector corresponding to a minimum value of a difference between a pre-edit motion compensated picture and a post-edit motion compensated picture, and

counting the number of macroblocks having a difference between the pre-edit motion vector and the post-edit motion vector.

4. A moving picture editing method for editing compression coded moving pictures by utilizing inter-frame prediction based on motion compensation, the method comprising:

determining if reference frames which are utilized for motion compensation of a subject frame before editing and which are not deleted during editing are subject to error generation in processing;

determining if at least one of the reference frames meets at least a condition that error generation in the reference frame results from re-encoding during editing or a condition that coded data has changed and a picture difference has been re-encoded during editing and a number of macroblocks of post-edit picture difference coded data and a number of macroblocks of pre-edit picture difference coded data exceed a threshold number;

obtaining a post-edit motion compensated picture by motion compensation utilizing a post-edit motion vector of a subject frame;

subtracting the post-edit motion compensated picture from pre-edit decoded data of the subject frame to obtain the picture difference;

coding the picture difference to obtain the post-edit picture difference coded data;

coding the post-edit picture difference coded data and pre-edit motion vectors to obtain post-edit coded data of the subject frame and the number of macroblocks having a differenee between the post-edit picture difference coded data and the pre-edit picture difference coded data.

5. A moving picture editing method for editing compression coded moving pictures by utilizing inter-frame prediction based on motion compensation, the method comprising:

determining if reference frames which are utilized for motion compensation of a subject frame before editing and which are not deleted during editing are subject to error generation in processing during editing;

determining if at least one of the reference frames meets a condition that coded data and a motion vector are changed during editing such that a number of macroblocks having a difference between a pre-edit motion vector and a post-edit motion vector exceeds a threshold number, or meets at least a condition that the coded data is not changed during editing, a condition that a picture difference is not re-encoded during editing although the coded data is changed therein, or a condition that, although the coded data has changed and the picture difference has been re-encoded during editing, the number of macroblocks having a difference between post-edit picture difference coded data and pre-edit picture difference coded data is less than a threshold number,

searching motion vectors in a range centered on each pre-edit motion vector of the subject frame for the motion vector corresponding to a minimum value of a difference between a pre-edit motion compensated picture and a post-edit motion compensated picture;

counting the number of macroblocks having a difference between the pre-edit motion vector and the post-edit motion vector;

determining if the reference frames each meet at least a condition that the coded data has not changed during editing, a condition that the motion vectors are not changed during editing, or a condition that the number of macroblocks having a difference between the pre-edit motion vector and the post-edit motion vector is less than a threshold number;

determining if the reference frames meet a condition that the coded data has changed and the picture difference has been re-encoded during editing such that the number of macroblocks having a difference between the post-edit picture difference coded data and the pre-edit picture difference coded data exceeds a threshold number;

subtracting a post-edit motion compensated picture from pre-edit decoded data of the subject frame to obtain a picture difference;

coding the picture difference to obtain post-edit picture difference coded data;

coding the post-edit picture difference coded data and the post-edit motion vector of the subject frame;

counting the number of macroblocks having a difference between the post-edit picture difference coded data and the pre-edit picture difference coded data; and

determining if one of the reference frames meets at least a first condition that it is a frame subject to error generation due to re-encoding during editing, or a second condition that the coded data has changed and the picture difference has been re-encoded during editing such that the number of macroblocks having a difference between the pre-edit motion vector and the post-edit motion vector exceeds a threshold number and the number of macroblocks having a difference between the post-edit picture difference coded data and the pre-edit picture difference coded data exceeds a threshold number;

searching motion vectors in a range centered on the pre-edit motion vector of the subject frame to find the motion vector corresponding to a minimum value of a difference between a pre-edit motion compensated picture and a post-edit motion compensated picture;

counting the number of macroblocks having a difference between the pre-edit motion vector and the post-edit motion vector;

subtracting the post-edit motion compensated picture from the pre-edit decoded data of the subject frame to obtain post-edit picture difference coded data;

coding the post-edit picture difference coded data and the pre-edit motion vector to obtain post-edit coded data of the subject frame; and

counting the number of macroblocks having a difference between post-edit picture difference coded data and pre-edit picture difference coded data.

8. A moving picture editing system for editing compensation coded moving pictures by utilizing inter-frame prediction based on motion compensation, the system comprising:

a motion vector searcher which searches motion vectors in a range centered on each pre-edit motion vector to find a motion vector corresponding to a minimum value of a difference between a pre-edit motion compensated picture and a post-edit motion compensated picture; and

a controller which controls a motion vector searcher to search for the post-edit motion vector of a subject frame when reference frames which are utilized for motion compensation of a picture frame and which are not deleted during editing are subject to error generation in processing during editing.

9. The moving picture editing system according to claim 8, further comprising:

a motion compensator which performs motion compensation utilizing the post-edit motion vector to obtain a post-edit motion compensated picture;

a subtracter which subtracts the post-edit motion compensated picture from pre-edit decoded data of the subject frame to obtain a picture difference; and

a variable length coder which codes a post-edit picture difference coded data and the post-edit motion vector to obtain post-edit coded data of the subject frame.

10. A moving picture editing system for editing compensation coded moving pictures by utilizing inter-frame prediction based on motion compensation, the system comprising:

a motion vector searcher which searches motion vectors in a range centered on each pre-edit motion vector to find a motion vector corresponding to a minimum value of a difference between a pre-edit motion compensated picture and a post-edit motion compensated picture;

a counter which counts macroblocks having a difference between a pre-edit motion vector and a post-edit motion vector; and

a controller which controls the motion vector searcher and the counter,

wherein when reference frames which are utilized for motion compensation of a subject frame before editing and which are not deleted during editing are subject to error generation in processing, and if at least one of the reference frames meets at least a condition that error generation in the reference frame results from re-encoding during editing or a condition that coded data has changed and the motion vector has changed during editing such that a number of macroblocks having a difference between the pre-edit motion vector and the post-edit motion vector exceeds a threshold number, the controller controls the motion vector searcher to search for the post-edit motion vector the subject frame and the counter to count the number of macroblocks having a difference between the pre-edit motion vector and the post-edit motion vector.

13. The moving picture editing system as in any one of claims 8–

10 or 12, wherein the controller limits the range of search of motion vectors after editing in the motion vector searcher to motion vectors which are the same as the post-edit motion vector or motion vectors present in a range in the neighborhood of the post-edit motion vector.

14. The moving picture editing system as in any one of claims 8–

10 or 12, wherein the controller limits the range of search of motion vectors after editing in the motion vector searcher to motion vectors which are the same as the post-edit motion vector or motion vectors present in a range in the neighborhood of the post-edit motion vector and having non-integer coordinate values as horizontal or vertical coordinate values.

11. A moving picture editing system for editing compensation coded moving pictures by utilizing inter-frame prediction based on motion compensation, the system comprising:

a motion compensator which performs motion compensation utilizing a post-edit motion vector to obtain a post-edit motion compensated picture;

a subtracter which subtracts the post-edit motion compensated picture from pre-edit decoded data of a subject frame to obtain a picture difference;

a variable length coder which codes post-edit picture difference coded data and the post-edit motion vector to obtain post-edit coded data of the subject frame;

a counter which counts macroblocks having a difference between the post-edit picture difference coded data and pre-edit picture difference coded data; and

a controller which controls the motion compensator to obtain the post-edit motion vector of the subject frame, controls the subtracter to subtract the post-edit motion compensated picture from the decoded data before editing of the subject frame, controls the picture difference coder to code the post-edit picture difference coded data, controls the variable length coder to code the post-edit coded data of the subject frame, and controls the counter when reference frames which are utilized for motion compensation of a subject frame before editing and which are not deleted during editing are subject to error generation in processing during editing, and if at least one of the reference frames meets at least a condition that error generation in the reference frame results from re-encoding during editing or from a condition that coded data has changed during editing and the picture difference is re-encoded such that the number of macroblocks having a difference between post-edit picture difference coded data and pre-edit picture difference coded data exceeds a threshold number.

12. A moving picture editing system for editing compensation coded moving pictures by utilizing inter-frame prediction based on motion compensation, the system comprising:

a motion vector searcher which searches motion vectors in a range centered on each pre-edit motion vector to find a motion vector corresponding to a minimum value of a difference between a pre-edit motion compensated picture and a post-edit motion compensated picture;

a first counter which counts a number of macroblocks having a difference between a pre-edit motion vector and a post-edit motion vector;

a motion compensator which performs motion compensation utilizing the post-edit motion vector of a subiect frame to obtain a motion compensated picture;

a subtracter which subtracts the post-edit motion compensated picture from pre-edit decoded data of the subject frame;

a picture difference coder which codes post-edit picture difference coded data from the picture difference;

a variable length coder which codes post-edit coded data of the subject frame from the post-edit difference picture coded data and the post-edit motion vector;

a second counter which counts macroblocks having a difference between the post-edit picture difference coded data and pre-edit picture difference coded data; and

a controller,

wherein if reference frames which are utilized for motion compensation of a subject frame and which are not deleted during editing are subject to error generation in processing during editing, and if at least one of the reference frames meets a condition that coded data has changed and the motion vector has changed during editing, such that a number of macroblocks having a difference between a pre-edit motion vector and a post-edit motion vector exceeds a threshold number, or meets at least a condition that the coded data has not changed during editing, a condition that the picture difference is not re-encoded during editing although the coded data has changed therein, or a condition that, although the coded data has changed and the picture difference has been re-encoded during editing, the number of macroblocks having a difference between post-edit picture difference coded data and pre-edit picture difference coded data is less than a threshold number, the controller controls the motion vector searcher to search the post-edit motion vector of the subject frame and controls the first counter to count the number of macroblocks having a difference between the pre-edit motion vector of the subject frame and the post-edit motion vector thereof, and

wherein if the reference frames each meet at least a condition that the coded data has not changed during editing, a condition that motion vectors have not changed during editing, or a condition that a number of macroblocks having a difference between a pre-edit motion vector and a post-edit motion vector is less than a threshold number, or the reference frames meet a condition that the coded data has changed and the picture difference has been re-encoded during editing such that the number of macroblocks having a difference between post-edit picture difference coded data and pre-edit picture difference coded data exceeds a threshold number, the controller controls the motion compensator to obtain a post-edit motion compensated picture of the subject frame, controls the subtracter to subtract the post-edit motion compensated picture of the subject frame from post-edit decoded data of the subject frame so as to obtain a picture difference, controls the picture difference coder to execute coding to obtain post-edit picture difference coded data of the subject frame and controls the second counter to count macroblocks having a difference between the post-edit picture difference coded data of the subject frame and the pre-edit picture difference coded data thereof, and

at least one of the reference frames meets at least a first condition that it is a frame subject to error generation due to re-encoding during editing, or a second condition that the coded data has changed and the picture difference has been re-encoded during editing such that the number of macroblocks having a difference between a pre-edit motion vector and a post-edit motion vector exceeds a threshold number and the number of macroblocks having a difference between post-edit picture difference coded data and pre-edit picture difference coded data exceeds a threshold number, the controller controls the motion vector searcher to search for the post-edit motion compensated picture of the subject frame, controls the first counter to count macroblocks having a difference between a post-edit motion vector of the subject frame and a pre-edit motion vector thereof, controls the subtracter to subtract the post-edit motion vector of the subject frame from the pre-edit decoded data of the subject frame so as to obtain the a picture difference, controls the picture difference coder to execute coding so as to obtain the post-edit picture difference coded data of the subject frame, controls the variable length coder to execute coding so as to obtain post-edit coded data of the subject frame and controls the second counter to count macroblocks having a difference between the post-edit picture difference coded data of the subject frame and the pre-edit picture difference coded data thereof.