Motion Adaptive Cropping for Video Stabilization

• US 20140307112A1
• Filed: 04/16/2013
• Published: 10/16/2014
• Est. Priority Date: 04/16/2013
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Sokeila, Tomi, Gren, Juuso
• Granted Patent: US 8,994,838 B2
• Time in Patent Office: Days
• US Class Current: 348/208.6
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Abstract

Determining an amount of motion in a second frame of video with reference to a first frame of video. Adaptively predicting a cropping factor for the second frame in response to the determined amount of motion. The cropping factor specifies a portion of the second frame that is to be excluded from the second frame. The predicted cropping factor is adaptively changed for a successive plurality of frames by buffering the first frame to predict a desired cropping factor for the second frame, wherein the first frame occurs prior to the second frame.

Claims

1. A method comprising:

determining an amount of motion in a second frame of video with reference to a first frame of video, and

adaptively predicting a cropping factor for the second frame in response to the determined amount of motion, wherein the cropping factor specifies a portion of the second frame that is to be excluded from the second frame.

2. The method of claim 1 wherein the predicted cropping factor is adaptively changed for a successive plurality of frames by buffering the first frame to predict a desired cropping factor for the second frame, wherein the first frame occurs prior to the second frame.

3. The method of claim 1 further comprising detecting a face or other object of interest in at least one of the received image frames, and adjusting the cropping factor so as not to exclude any portion of the face or other object of interest.

4. The method of claim 3 wherein the cropping factor specifies performing cropping non-centrically or non-symmetrically with respect to one or more image frame boundaries.

5. A method comprising:

receiving at an apparatus a plurality of image frames of a scene;

for each of respective image frames of the plurality of image frames, determining at the apparatus a corresponding cropping factor based upon detecting an amount of motion in the respective image frame relative to at least one other image frame of the plurality of image frames to provide image stabilization for the respective frame.

6. The method of claim 5 further comprising:

storing in a buffer memory the corresponding cropping factors for the respective image frames, and using at least one of the stored corresponding cropping factors to predict at least one applied cropping factor to be applied to one or more of the respective image frames.

7. The method of claim 5, wherein the at least one applied cropping factor that is applied to an image frame of the one or more respective image frames is different from the corresponding cropping factor for the image frame that is stored in the buffer memory.

8. The method of claim 6, wherein predicting the at least one applied cropping factor is performed to provide a smooth change of the cropping factor from a first image frame to a second image frame in the plurality of image frames.

9. The method of claim 5, wherein the plurality of image frames are temporally consecutive frames.

10. The method of claim 5, wherein one of the received image frames is a first received image frame out of the plurality of the image frames.

11. The method of claim 6, wherein the predicting of the at least one applied cropping factor is based on increasing or decreasing of a motion of one or more objects in the scene within the plurality of image frames to provide a smooth adaptation of the applied cropping factor within the plurality of image frames.

12. The method of claim 11, wherein the predicting of the at least one applied cropping factor is based upon a sudden increase or decrease of a motion of one or more objects in the scene within the plurality of image frames to provide a smooth adaptation of the at least one applied cropping factor such that a that a user does not detect a zooming effect.

13. The method of claim 6, wherein the predicting of the at least one applied cropping factor is further based on a content of the plurality of image frames.

14. The method of claim 6, wherein the predicting of the at least one applied cropping factor for the at least one of the received image frames is based on analysis of image frames received only before the at least one of the received image frames.

15. The method of claim 6, wherein the predicting of the at least one applied cropping factor for the at least one of the received image frames is based on analysis of image frames received before and after the at least one of the received image frames.

16. The method of claim 5, wherein receiving the plurality of image frames comprises receiving by the apparatus the plurality of image frames externally through a wireless or wired link.

17. The method of claim 5, wherein receiving the plurality of image frames comprises acquiring by the apparatus the plurality of image frames.

18. An apparatus comprising at least one processor and a memory storing a set of computer instructions, in which the processor and the memory storing the computer instructions are configured to cause the apparatus to perform:

determining an amount of motion in a second frame of video with reference to a first frame of video, and

adaptively predicting a cropping factor for the second frame in response to the determined amount of motion, wherein the cropping factor specifies a portion of the second frame that is to be excluded from the second frame.

19. The apparatus of claim 18 wherein the predicted cropping factor is adaptively changed for a successive plurality of frames by buffering the first frame to predict a desired cropping factor for the second frame, wherein the first frame occurs prior to the second frame.

20. An apparatus comprising at least one processor and a memory storing a set of computer instructions, in which the processor and the memory storing the computer instructions are configured to cause the apparatus to perform:

receiving at an apparatus a plurality of image frames of a scene;

for each of respective image frames of the plurality of image frames, determining at the apparatus a corresponding cropping factor based upon detecting an amount of motion in the respective image frame relative to at least one other image frame of the plurality of image frames to provide image stabilization for the respective frame.

21. The apparatus of claim 20 wherein the computer instructions are further configured to perform:

storing in a buffer memory the corresponding cropping factors for the respective image frames, and using at least one of the stored corresponding cropping factors to predict at least one applied cropping factor to be applied to one or more of the respective image frames.

22. A non-transitory computer readable memory encoded with a computer program comprising computer readable instructions recorded thereon for execution of a method comprising:

determining an amount of motion in a second frame of video with reference to a first frame of video, and

adaptively predicting a cropping factor for the second frame in response to the determined amount of motion, wherein the cropping factor specifies a portion of the second frame that is to be excluded from the second frame.