Moving picture editing method, moving picture editing system and storing medium with moving picture editing programs stored therein

• US 20020081094A1
• Filed: 11/07/2001
• Published: 06/27/2002
• Est. Priority Date: 11/08/2000
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Hatabu, Atsushi
• Granted Patent: US 7,209,635 B2
• Time in Patent Office: Days
• US Class Current: 386/52
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A moving picture editing method is disclosed, in which if one or two reference frames utilized for motion compensation of P or B frame, constituting a part of moving picture before editing and not deleted in edition so as to constitute moving picture after editing, is subject to error generation in processing although not subject to lack in edition, motion vectors in a range centered on each motion vector V before editing of the P or B frame, are searched for the motion vector V′ after editing corresponding to the minimum value of the difference between the motion compensated picture MC (X, V) before edition and motion compensated picture MC (X′, V′) after editing.

Claims

1. A moving picture editing method for editing compression coded moving picture by utilizing inter-frame prediction based on motion compensation, wherein:

if one or two reference frames utilized for motion compensation of a subject frame, constituting a part of the moving picture before editing and not deleted in edition so as to constitute a part of the moving picture after editing, is or are subject to error generation in processing although not subject to lack in edition, motion vectors in a range centered on each motion vector before editing of the subject frame are searched for the motion vector after editing corresponding to the minimum value of the difference between the motion compensated picture before editing and motion compensated picture after editing.

6. The moving picture editing method according to one of claims 1 to 3 and 5, wherein candidates for each motion vectors after editing are limited to motion vectors present in a predetermined range centered on the motion vector before editing.

7. The moving picture editing method according to one of claims 1 to 3 and 5, wherein candidates for each motion vectors after editing are limited to motion vectors, which are equal to motion vector before editing or motion vectors present in a predetermined range centered on the motion vector before editing and having non-integer coordinate value as horizontal or vertical coordinate value.

15. A storing medium, in which moving picture editing programs for realizing the functions set forth in one of claims 1 to 14 in a personal computer are stored.

2. A moving picture editing method for editing compression coded moving picture by utilizing inter-frame prediction based on motion compensation, wherein:

if one or two reference frames utilized for motion compensation of a subject frame, constituting a part of the moving picture before editing and not deleted in edition so as to constitute part of the moving picture after editing, is or are subject to error generation in processing although not subject to lack in edition, motion vectors in a range centered on each motion vector before editing of the subject frame are searched for the motion vector after editing corresponding to the minimum value of the difference between the motion compensated picture before editing and motion compensated picture after editing, and picture difference obtained as a result of subtraction of motion compensated picture after edition, obtained by motion compensation utilizing motion vector after editing, from decoded data before editing of the subject frame, is coded to obtain picture difference coded data after editing, and coding is performed from the picture difference coded data after editing thus obtained and the motion vectors after editing to obtain coded data after editing of the subject frame.

3. A moving picture editing method for editing compression coded moving picture by utilizing inter-frame prediction based on motion compensation, wherein:

if one or two reference frames utilized for motion compensation of a subject frame, constituting a part of the moving picture before editing and not deleted in edition so as to constitute part of the moving picture after editing, is subject to error generation in processing although not subject to lack in edition, and also if either one of the reference frames meets at least a condition that error generation in it takes place as a result of re-encoding in edition or a condition that the coded data and the motion vector are changed in edition such that the number of macroblocks having difference between motion vector before editing and motion vector after editing exceed a predetermined threshold number, motion vectors in a range centered on each motion vector before editing of the subject frame are searched for the motion vector after editing corresponding to the minimum value of the difference between the motion compensated picture before edition and motion compensated picture after editing, and also the number of macroblocks having difference between the motion vector before editing and the motion vector after editing are counted.

4. A moving picture editing method for editing compression coded moving picture by utilizing inter-frame prediction based on motion compensation, wherein:

if one or two reference frames utilized for motion compensation of a subject frame, constituting a part of the moving picture before editing and not deleted in edition so as to constitute part of the moving picture after editing, is subject to error generation in processing although not subject to lack in edition, and also if either one of the reference frames meets at least a condition that its error generation takes place as a result of re-encoding in edition or a condition that coded data is changed, picture difference is re-encoded in edition and the number of picture difference coded data after editing and picture difference coded data before editing exceed a predetermined threshold number, picture difference obtained as a result of subtraction of motion compensated picture after editing, obtained by motion compensation utilizing the motion vector after editing of the subject frame, from decoded data before editing of the subject frame is coded to obtain picture difference coded data after editing, also coding is performed from the picture difference coded data after editing and the motion vectors before editing to obtain coded data after editing of the subject frame, and the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing.

5. A moving picture editing method for editing compression coded moving picture by utilizing inter-frame prediction based on motion compensation, wherein:

if one or two reference frames utilized for motion compensation of a subject frame, constituting a part of the moving picture before editing and not deleted in edition so as to constitute part of the moving picture after editing, is subject to error generation in processing although not subject to lack in edition, and also if either one of the reference frames meets a condition that the coded data and motion vector are changed in edition, such that the number of macroblocks having difference between motion vector before editing and motion vector after editing exceed a predetermined threshold number, and also meets at least a condition that the coded data is not changed in edition, a condition that the picture difference is not re-encoded in edition, although the coded data is changed therein, or a condition that, although change in the coded data and re-encoding of the picture difference have taken place in edition, the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing is less than a predetermined threshold number, motion vectors in a range centered on each motion vector before editing of the subject frame are searched for the motion vector corresponding to the minimum value of the difference between the motion compensated picture before editing and motion compensated picture after editing, and the number of macroblocks having difference between motion vector before editing and motion vector after editing are counted;

if the one or two reference frames each meet at least a condition that the coded data is not changed in edition, a condition that the motion vectors are not changed in edition, or a condition that, the number of macroblocks having difference between motion vector before editing and motion vector after editing is less than a predetermined threshold number, while the one or two frames meet a condition that change in the coded data and re-encoding of the picture difference have taken place in edition such that the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing exceed a predetermined threshold number, picture difference obtained as a result of subtraction of motion compensated picture after editing, obtained by motion compensation utilizing the motion vector after editing of the subject frame, from decoded data before editing of the subject frame is coded to obtain picture difference coded data after editing, also coding is performed from the picture difference coded data after editing and the motion vector after editing of the subject frame, and the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing are counted; and

if one of the one or two reference frames meets at least a first condition that it is a frame subject to error generation due to re-encoding in edition, or a second condition that change in the coded data and re-encoding of picture difference have taken place in edition such that the number of macroblocks having difference between motion vector before editing and motion vector after editing exceed a predetermined threshold number and the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing exceed a predetermined threshold number, motion vector in a range centered on motion vector before editing of the subject frame are searched for the motion vector corresponding to the minimum value of the difference between the motion compensated picture before editing and motion compensated picture after editing, the number of macroblocks having difference between the motion vector before editing and the motion vector after editing are counted, picture difference obtained as a result of subtraction of motion compensated picture after editing, obtained by motion compensation utilizing the motion vector after editing of the subject frame, from decoded data before editing of the subject frame to obtain picture difference coded data after editing, coding is performed from the picture difference coded data after editing and the motion vector before editing to obtain coded data after editing of the subject frame, and the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing are counted.

8. A moving picture editing system for editing compensation coded moving picture by utilizing inter-frame prediction based on motion compensation, comprising:

a motion vector searcher for searching motion vectors in a range centered on each motion vector before editing for a motion vector corresponding to the minimum value of the difference motion compensated picture before editing and motion compensated picture after editing; and

a controller for controlling, if one or two reference frames utilized for motion compensation of a picture frame, constituting a part of the moving picture before editing and not deleted in edition so as to constitute part of the moving picture after edition, is subject to error generation in processing although not subject to lack in edition, the motion vector searcher to search for the motion vector after editing of the subject frame.

9. The moving picture editing system according to claim 8, further comprising:

a motion compensator for executing motion compensation utilizing motion vector after editing to obtain motion compensated picture after editing;

a subtracter for subtracting motion compensated picture after editing from decoded data before editing of the subject frame to obtain picture difference; and

a variable length coder for executing coding from the picture difference coded data after editing and the motion vector after editing to obtain coded data after editing of the subject frame.

10. A moving picture editing system for editing compensation coded moving picture by utilizing inter-frame prediction based on motion compensation, comprising:

a motion vector searcher for searching motion vectors in a range centered on each motion vector before editing for a motion vector corresponding to the minimum value of the difference motion compensated picture before editing and motion compensated picture after editing;

a counter for counting macroblocks having difference between motion vector before editing and motion vector after editing; and

a controller functioning such that, if one or two reference frames utilized for motion compensation of a subject frame, constituting a part of the moving picture before editing and not deleted in edition so as to constitute a part of the moving picture after editing, is subject to error generation in processing although not subject to lack in edition, and also if either one of the reference frames meets at least a condition that error generation in it takes place as a result of re-encoding in edition or a condition that the coded data and the motion vector are changed in edition such that the number of macroblocks having difference between motion vector before editing and motion vector after editing exceed a predetermined threshold number, it controls the motion vector searcher to search for the motion vector after editing of the subject frame and the counter to count the number of macroblocks having difference between motion vector before editing and motion vector after editing.

13. The moving picture editing system according to one of claims 8 to 10 and 12, wherein the controller limits the range of search of motion vectors after editing in the motion vector searcher to motion vectors, which are the same as the motion vector after editing or motion vectors present in a range in the neighborhood of the motion vector after editing.

14. The moving picture editing system according to one of claims 8 to 10 and 12, wherein the controller limits the range of search of motion vectors after editing in the motion vector searcher to motion vectors, which are the same as the motion vector after editing or motion vectors present in a range in the neighborhood of the motion vector after editing and having non-integer coordinate values as horizontal or vertical coordinate values.

11. A moving picture editing system for editing compensation coded moving picture by utilizing inter-frame prediction based on motion compensation, comprising:

a motion compensator for executing motion compensation utilizing motion vector after editing to obtain motion compensated picture after editing;

a subtracter for subtracting motion compensated picture after editing from decoded data before editing of the subject frame to obtain picture difference;

a variable length coder for executing coding from the picture difference coded data after editing and the motion vector after editing to obtain coded data after editing of the subject frame;

a counter for counting macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing; and

a controller functioning such that, if one or two reference frames utilized for motion compensation of a subject frame, constituting a part of the moving picture before editing and not deleted in edition so as to constitute a part of the moving picture after editing, is subject to error generation in processing although not subject to lack in edition, and also if either one of the reference frames meets at least a condition that error generation in it takes place as a result of re-encoding in edition or a condition that the coded data is changed in edition and the picture difference is re-encoded such that the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing exceed a predetermined threshold number, it controls the motion compensator to obtain the motion vector after editing of the subject frame, the subtracter to subtract the motion compensated picture after editing from the decoded data before editing of the subject frame, the picture difference coder to code the picture difference coded data after editing, variable length coder to code the coded data after editing of the subject frame, and the counter to count the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing.

12. A moving picture editing system for editing compensation coded moving picture by utilizing inter-frame prediction based on motion compensation, comprising:

a motion vector searcher for searching motion vectors in a range centered on each motion vector before editing for a motion vector corresponding to the minimum value of the difference motion compensated picture before editing and motion compensated picture after editing;

a first counter for counting the number of macroblocks having difference between motion vector before editing and motion vector after editing;

a motion compensator for performing motion compensation utilizing the motion vector of the frame after editing to obtain motion compensated picture;

a subtracter for subtracting the motion compensated picture after editing from the decoded data before editing of the subject frame;

a picture difference coder for coding the picture difference coded data after editing from the picture difference;

a variable length coder for coding the coded data after editing of the subject frame from the diffrence picture coded data after editing and the motion vector after editing;

a second counter for counting macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing; and

a controller functioning such that;

if one or two reference frames utilized for motion compensation of a subject frame, constituting a part of the moving picture before editing and not deleted in edition so as to constitute a part of the moving picture after editing, is subject to error generation in processing although not subject to lack in edition, and also if either one of the reference frames meets a condition that the coded data and the motion vector are changed in edition, such that the number of macroblocks having difference between motion vector before editing and motion vector after editing exceed a predetermined threshold number, and also meets at least a condition that the coded data is not changed in edition, a condition that the picture difference is not re-encoded in edition, although the coded data is changed therein, or a condition that, although change in the coded data and re-encoding of the picture difference have taken place in edition, the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing is less than a predetermined threshold number, it controls the motion vector searcher to search the motion vector after editing of the subject frame and also control the first counter to count the number of macroblocks having difference between the motion vector before editing of the subject frame and the motion vector after editing thereof,

if the one or two reference frames each meet at least a condition that the coded data is not changed in edition, a condition that motion vectors are not changed in edition, or a condition that the number of macroblocks having difference between motion vector before editing and motion vector after editing is less than a predetermined threshold number, while the one or two reference frames meet a condition that change in the coded data and re-encoding of the picture difference have taken place in edition such that the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing exceed a predetermined threshold number, it controls the motion compensator to obtain motion compensated picture after editing of the subject frame, controls the subtracter to subtract the motion compensated picture after editing of the subject frame from decoded data after editing of the subject frame so as to obtain picture difference, controls the picture difference coder to execute coding to obtain picture difference coded data after editing of the subject frame and controls the second counter to count macroblocks having difference between the picture difference coded data after editing of the subject frame and the picture difference coded data before editing thereof, and

if one of the one or two reference frames meets at least a first condition that it is a frame subject to error generation due to re-encoding in edition, or a second condition that change in the coded data and re-encoding of the picture difference have taken place in edition such that the number of macroblocks having difference between motion vector before editing and motion vector after editing exceed a predetermined threshold number and the number of macroblocks having difference between picture difference coded data after editing and picture difference coded data before editing exceed a predetermined threshold number, it controls the motion vector searcher to search for the motion picture after editing of the subject frame, controls the first counter to count macroblocks having difference between motion vector before editing of the subject frame and motion vector before editing thereof, controls the subtracter to subtract the motion vector after editing of the subject frame from the decoded data before editing of the subject frame so as to obtain the picture difference, controls the picture difference coder to execute coding so as to obtain the picture difference coded data after editing of the subject frame, controls the variable length coder to execute coding so as to obtain coded data after editing of the subject frame and controls the second counter to count macroblocks having difference between the picture difference coded data after editing of the subject frame and the picture difference coded data after editing thereof.