Method of and apparatus for generating an output frame

• US 10,194,156 B2
• Filed: 07/08/2015
• Issued: 01/29/2019
• Est. Priority Date: 07/15/2014
• Status: Expired due to Fees

• Current Assignee: Arm Ltd. (United Kingdom) (SoftBank Group Corp.)
• Original Assignee: Arm Ltd. (United Kingdom) (SoftBank Group Corp.)
• Inventors: Croxford, Daren, Ellis, Sean, James, Ben
• Primary Examiner(s): Haque, Md N
• Application Number:
  US14/793,907
• Publication Number:
  US 20160021384A1
• Time in Patent Office: 1,301 Days
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

A method and an apparatus are provided for generating an output frame from an input frame, in which the input frame is processed when generating the output frame. A region of a current input frame is compared with a region of a preceding input frame to determine if the region of the current input frame is similar to the region of the preceding input frame. When the region of the current input frame is determined to be similar to the region of the preceding input frame, information relating to processing performed on the region of the preceding input frame when generating a region of a preceding output frame is read, wherein the information is generated during the processing on the region of the preceding input frame. When the information indicates that the processing is unnecessary, a part or all of the processing of the region of the current input frame can be bypassed or eliminated.

Claims

1. A method of generating output frames from input frames, in which input frames are processed when generating output frames, the method comprising:

when generating a region of a first output frame from a region of a first input frame;

processing the region of the first input frame;

generating information relating to the processing performed on the region of the first input frame; and

storing the information relating to the processing performed on the region of the first input frame; and

when generating a region of a second output frame from a region of a second input frame;

comparing the region of the second input frame with the region of the first input frame to determine if the region of the second input frame is similar to the region of the first input frame; and

when the region of the second input frame is determined to be similar to the region of the first input frame;

reading the stored information relating to the processing performed on the region of the first input frame when generating the region of the first output frame;

determining whether the read information relating to the processing performed on the region of the first input frame when generating the region of the first output frame indicates that a part or all of the processing of the region of the second input frame is not required; and

when it is not determined that the read information relating to the processing performed on the region of the first input frame when generating the region of the first output frame indicates that a part or all of the processing of the region of the second input frame is not required;

processing the region of the second input frame; and

when it is determined that the read information relating to the processing performed on the region of the first input frame when generating the region of the first output frame indicates that a part or all of the processing of the region of the second input frame is not required;

omitting the part or all of the processing of the region of the second input frame.

2. The method of claim 1, further comprising:

comparing the region of the first input frame with the region of the first output frame to determine if the region of the first output frame is similar to the region of the first input frame, wherein the first output frame is obtained by performing the processing on the first input frame; and

storing the result of the comparison between the region of the first input frame and the region of the first output frame as part of the information relating to the processing performed on the region of the first input frame, by indicating that the processing is not required when the region of the first output frame is determined to be similar to the region of the first input frame, or indicating that the processing is necessary when the region of the first output frame is determined to be not similar to the region of the first input frame.

3. The method of claim 2, further comprising using the region of the second input frame as a corresponding region of a second output frame when the information indicates that the processing is not required.

4. The method of claim 2, further comprising processing the region of the second input frame to obtain a corresponding region of a second output frame when the information indicates that the processing is necessary.

5. The method of claim 1, further comprising assessing a likelihood of the region of the second input frame being similar to the region of the first input frame, and only performing the step of storing the information relating to the processing performed on the region of the first input frame when the region of the second input frame is assessed to be likely to be similar to the region of the first input frame.

6. The method of claim 1, further comprising:

performing analysis on the region of the first input frame; and

storing the result of the analysis as part of the information relating to the processing performed on the region of the first input frame.

7. The method of claim 6, wherein performing analysis comprises performing edge detection on the region of the first input frame and/or generating statistics on the region of the first input frame for use in one or more algorithms to be applied to process the region of the first input frame, and wherein the result of the analysis comprises the result of edge detection of whether one or more edges are detected, and/or an indication of one or more algorithms to be applied to process the region of the first input frame based on the result of edge detection, and/or the statistics on the region of the first input frame.

8. The method of claim 1, wherein the step of comparing comprises comparing one or more signatures representative of the content of the region in question with one or more signatures representative of the content of the frame region that the frame region in question is to be compared with.

9. The method of claim 1, wherein the processing performed on the first input frame comprises a plurality of processing stages that together generate a final first output frame for use, and the first output frame is an output frame generated during an intermediate processing stage of the plurality of processing stages, the first input frame is an input frame of the intermediate processing stage, and the second input frame is an input frame of the intermediate processing stage.

10. The method of claim 1, wherein the processing performed on the first input frame comprises a plurality of processing stages that together generate a final first output frame for use, and further comprising using the method for more than one of the plurality of processing stages.

11. An apparatus for generating output frames from input frames, comprising processing circuitry capable of processing input frames when generating output frames and control circuitry capable of:

when generating a region of a first output frame from a region of a first input frame;

processing the region of the first input frame;

generating information relating to the processing performed on the region of the first input frame; and

storing the information relating to the processing performed on the region of the first input frame; and

when generating a region of a second output frame from a region of a second input frame;

comparing the region of the second input frame with the region of the first input frame to determine if the region of the second input frame is similar to the region of the first input frame; and

when the region of the second input frame is determined to be similar to the region of the first input frame;

reading the stored information relating to the processing performed on the region of the first input frame when generating the region of the first output frame;

determining whether the read information relating to the processing performed on the region of the first input frame when generating the region of the first output frame indicates that a part or all of the processing of the region of the second input frame is not required; and

when it is not determined that the read information relating to the processing performed on the region of the first input frame when generating the region of the first output frame indicates that a part or all of the processing of the region of the second input frame is unnecessary not required;

processing the region of the second input frame; and

when it is determined that the read information relating to the processing performed on the region of the first input frame when generating the region of the first output frame indicates that a part or all of the processing of the region of the second input frame is not required;

omitting the part or all of the processing of the region of the second input frame.

12. The apparatus of claim 11, wherein the control circuitry is further capable of:

comparing the region of the first input frame with the region of the first output frame to determine if the region of the first output frame is similar to the region of the first input frame, wherein the first output frame is obtained by performing the processing on the first input frame; and

storing the result of the comparison between the region of the first input frame and the region of the first output frame as part of the information relating to the processing performed on the region of the first input frame, wherein the control circuitry is capable of storing the result of the comparison by indicating that the processing is not required when the region of the first output frame is determined to be similar to the region of the first input frame, or indicating that the processing is necessary when the region of the first output frame is determined to be not similar to the region of the first input frame.

13. The apparatus of claim 12, wherein the processing circuitry is capable of using the region of the second input frame as a corresponding region of a second output frame when the information indicates that the processing is not required.

14. The apparatus of claim 12, wherein the processing circuitry is capable of processing the region of the second input frame to obtain a corresponding region of a second output frame when the information indicates that the processing is necessary.

15. The apparatus of claim 11, wherein the control circuitry is further capable of assessing a likelihood of the region of the second input frame being similar to the region of the first input frame, and only stores the information relating to the processing performed on the region of the first input frame when the region of the second input frame is assessed to be likely to be similar to the region of the first input frame.

16. The apparatus of claim 11, wherein the control circuitry is further capable of:

performing analysis on the region of the first input frame; and

storing the result of the analysis as part of the information relating to the processing performed on the region of the first input frame.

17. The apparatus of claim 16, wherein the control circuitry is capable of performing analysis by performing edge detection on the region of the first input frame and/or generating statistics on the region of the first input frame for use in one or more algorithms to be applied to process the region of the first input frame, and wherein the result of the analysis comprises the result of edge detection of whether one or more edges are detected, and/or an indication of one or more algorithms to be applied to process the region of the first input frame based on the result of edge detection, and/or the statistics on the region of the first input frame.

18. The apparatus of claim 11, wherein the control circuitry is capable of performing the comparing by comparing one or more signatures representative of the content of the region in question with one or more signatures representative of the content of the frame region that the frame region in question is to be compared with.

19. The apparatus of claim 11, wherein the processing performed on the first input frame comprises a plurality of processing stages that together generate a final first output frame for use, and the first output frame is an output frame generated during an intermediate processing stage of the plurality of processing stages, the first input frame is an input frame of the intermediate processing stage, and the second input frame is an input frame of the intermediate processing stage.

20. The apparatus of claim 11, wherein the processing performed on the first input frame comprises a plurality of processing stages that together generate a final first output frame for use, and the apparatus is configured to perform the functions for more than one of the plurality of processing stages.

21. A non-transitory storage medium containing a computer program which, when run on a data processor, causes the data processor to perform a method of generating output frames from input frames, in which input frames are processed when generating output frames, the method comprising:

when generating a region of a first output frame from a region of a first input frame;

processing the region of the first input frame;

generating information relating to the processing performed on the region of the first input frame; and

storing the information relating to the processing performed on the region of the first input frame; and

when generating a region of a second output frame from a region of a second input frame;

comparing the region of the second input frame with the region of the first input frame to determine if the region of the second input frame is similar to the region of the first input frame; and

when the region of the second input frame is determined to be similar to the region of the first input frame;

reading the stored information relating to the processing performed on the region of the first input frame when generating the region of the first output frame;

determining whether the read information relating to the processing performed on the region of the first input frame when generating the region of the first output frame indicates that a part or all of the processing of the region of the second input frame is not required; and

when it is not determined that the read information relating to the processing performed on the region of the first input frame when generating the region of the first output frame indicates that a part or all of the processing of the region of the second input frame is not required;

processing the region of the second input frame; and

when it is determined that the read information relating to the processing performed on the region of the first input frame when generating the region of the first output frame indicates that a part or all of the processing of the region of the second input frame is not required;

omitting the part or all of the processing of the region of the second input frame.