MULTI-WINDOW IMAGE PROCESSING AND MOTION COMPENSATION

• US 20150138403A1
• Filed: 06/16/2014
• Published: 05/21/2015
• Est. Priority Date: 06/17/2013
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Grunberg, Daniel B., Chin, Douglas M., Besen, Peter D., Prithviraj, Anantharanga
• Granted Patent: US 9,811,914 B2
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G11B 31/006 with video camera or receiver
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 9/7908 Suppression of interfering signals at the reproducing side, e.g. noise

Abstract

An image processing system receives a sequence of frames including a current input frame and a next input frame (the next input frame is captured subsequent in time with respect to capturing of the current input frame). The image processing system stores a previously outputted output frame. The previously outputted output frame is derived from previously processed input frames in the sequence. The image processing modifies the current input frame based on detected first motion and second motion. The first motion is detected based on an analysis of the current input frame with respect to the next input frame. The second motion is detected based on an analysis of the current input frame with respect to the previously outputted output frame. According to one configuration, the image processing system implements multi-sized analyzer windows to more precisely detect the first motion and second motion.

Claims

1. A method comprising:

via computer processor hardware;

applying a first-sized analyzer window to correlate a rendition of an object in a first frame of display elements to a rendition of the object in a second frame of display elements;

applying a second-sized analyzer window to correlate the rendition of the object in the first frame of display elements to a rendition of the object in the second frame of display elements; and

deriving motion vector information for the rendered object depending on results of applying the first-sized analyzer window and the second-sized analyzer window.

2. The method as in claim 1, wherein the motion vector information indicates a degree of movement of the rendered object from a location in the second frame with respect to a location in the first frame.

3. The method as in claim 1, wherein the first-sized analyzer window is sized to analyze a first number of display elements;

and

wherein the second-sized analyzer window is sized to analyze a second number of display elements, the second number of display elements being substantially less than the first number of display elements.

4. The method as in claim 1, wherein the rendition of the object is defined by substantially fewer display elements than a number of display elements captured by the first-sized analyzer window;

wherein the rendition of the object is defined by substantially fewer display elements than a number of display elements captured by the second-sized analyzer window; and

wherein the second-sized analyzer window is substantially smaller than the first-sized analyzer window.

5. The method as in claim 2 further comprising:

producing a first array of correlation results based on application of the first-sized analyzer window, the first array of correlation results indicating different degrees of matching of the first-sized analyzer window of display elements surrounding the rendered object at the location in the first frame to candidate locations in the second frame; and

producing a second array of correlation results based on application of the second-sized analyzer window, the second array of correlation results indicating different degrees of matching of the second-sized analyzer window of display elements surrounding the rendered object at the location in the first frame to candidate locations in the second frame.

6. The method as in claim 5, wherein deriving the motion vector information for the rendered object includes:

responsive to detecting that a particular candidate location as indicated by the first array of correlation results provides a better match for the rendered object than the candidate locations as indicated by the second array of correlation results, selecting the particular candidate location to produce the motion vector information.

7. The method as in claim 5, wherein deriving the motion vector information for the rendered object includes:

responsive to detecting that a particular candidate location as indicated by the second array of correlation results provides a better match for the rendered object than any candidate location as indicated by the first array of correlation results, selecting the candidate location from the second array of correlation results to produce the motion vector information.

8. The method as in claim 1, wherein applying the first-sized analyzer window includes calculating a likeness of a first-sized region of display elements surrounding the rendered object in the first frame to different candidate regions in the second frame;

and

wherein applying the second-sized analyzer window includes calculating a likeness of a second-sized region of display elements surrounding the rendered object in the first frame to the different candidate regions in the second frame.

9. The method as in claim 2 further comprising:

utilizing the motion vector information to map a region in the second frame to a region in the first frame, the region in the second frame and the region in the first frame both representing the rendered object, the rendered object detected as moving from the location in the second frame to the location in the first frame; and

producing an output frame to include a rendition of the object, the output frame including a rendition of the object, the rendition of the object derived from a combination of settings of the rendition of the object in the first frame to a rendition of the object in the second frame.

10. The method as in claim 9, wherein the rendered object is a moving object and the combination reduces noise in the rendition of the object in the output frame.

11. A computer system comprising:

computer processor hardware; and

a hardware storage resource coupled to the computer processor hardware, the hardware storage resource storing instructions that, when executed by the computer processor hardware, causes the computer processor hardware to perform operations of;

applying a first-sized analyzer window to correlate a rendition of an object in a first frame of display elements to a rendition of the object in a second frame of display elements;

applying a second-sized analyzer window to correlate the rendition of the object in the first frame of display elements to a rendition of the object in the second frame of display elements; and

deriving motion vector information for the rendered object depending on results of applying the first-sized analyzer window and the second-sized analyzer window.

12. The computer system as in claim 11, wherein the motion vector information indicates a degree of movement of the rendered object from a location in the second frame with respect to a location in the first frame.

13. The computer system as in claim 11, wherein the first-sized analyzer window is sized to analyze a first number of display elements;

and

wherein the second-sized analyzer window is sized to analyze a second number of display elements, the second number of display elements being substantially less than the first number of display elements.

14. The computer system as in claim 11, wherein the rendition of the object is defined by substantially fewer display elements than a number of display elements captured by the first-sized analyzer window;

wherein the rendition of the object is defined by substantially fewer display elements than a number of display elements captured by the second-sized analyzer window; and

wherein the second-sized analyzer window is substantially smaller than the first-sized analyzer window.

15. The computer system as in claim 12, wherein the computer processor hardware further performs operations of:

producing a first array of correlation results based on application of the first-sized analyzer window, the first array of correlation results indicating different degrees of matching of the first-sized analyzer window of display elements surrounding the rendered object at the location in the first frame to candidate locations in the second frame; and

producing a second array of correlation results based on application of the second-sized analyzer window, the second array of correlation results indicating different degrees of matching of the second-sized analyzer window of display elements surrounding the rendered object at the location in the first frame to candidate locations in the second frame.

16. The computer system as in claim 15, wherein deriving the motion vector information for the rendered object includes:

responsive to detecting that a particular candidate location as indicated by the first array of correlation results provides a better match for the rendered object than the candidate locations as indicated by the second array of correlation results, selecting the particular candidate location to produce the motion vector information.

17. The computer system as in claim 15, wherein deriving the motion vector information for the rendered object includes:

responsive to detecting that a particular candidate location as indicated by the second array of correlation results provides a better match for the rendered object than any candidate location as indicated by the first array of correlation results, selecting the candidate location from the second array of correlation results to produce the motion vector information.

18. The computer system as in claim 11, wherein applying the first-sized analyzer window includes calculating a likeness of a first-sized region of display elements surrounding the rendered object in the first frame to different candidate regions in the second frame;

and

wherein applying the second-sized analyzer window includes calculating a likeness of a second-sized region of display elements surrounding the rendered object in the first frame to the different candidate regions in the second frame.

19. The computer system as in claim 11, wherein the computer processor hardware further performs operations of:

utilizing the motion vector information to map a region in the second frame to a region in the first frame, the region in the second frame and the region in the first frame both representing the rendered object, the rendered object detected as moving from the location in the second frame to the location in the first frame; and

producing an output frame to include a rendition of the object, the output frame including a rendition of the object, the rendition of the object derived from a combination of settings of the rendition of the object in the first frame to a rendition of the object in the second frame.

20. The computer system as in claim 11, wherein the rendered object is a moving object and the combination reduces noise in the rendition of the object in the output frame.

21. Computer-readable hardware storage having instructions stored thereon, the instructions, when carried out by computer processor hardware, causes the computer processor hardware to perform operations of:

applying a first-sized analyzer window to correlate a rendition of an object in a first frame of display elements to a rendition of the object in a second frame of display elements;

applying a second-sized analyzer window to correlate the rendition of the object in the first frame of display elements to a rendition of the object in the second frame of display elements; and

deriving motion vector information for the rendered object depending on results of applying the first-sized analyzer window and the second-sized analyzer window.