Method and apparatus for cropping and stabilization of video images

• US 8,743,222 B2
• Filed: 02/14/2012
• Issued: 06/03/2014
• Est. Priority Date: 02/14/2012
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Hamalainen, Matti Sakari
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): Haliyur, Padma
• Application Number:
  US13/372,635
• Publication Number:
  US 20130208134A1
• Time in Patent Office: 840 Days
• Field of Search: 348/208.99, 348/222.1
• US Class Current: 348/208.99
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

A method including receiving cropping information for a current video frame; capturing at a camera sensor the current video frame; performing a first crop of the current video frame using a first cropping window to form a cropped video frame, wherein the first cropping window has been dynamically adjusted in response to the received cropping information; and sending the cropped video frame for digital image stabilization.

Claims

1. A method comprising:

receiving cropping information, at a camera module for cropping a current video frame, from a host apparatus;

capturing at a camera sensor of the camera module the current video frame;

performing a first crop of the current video frame using a first cropping window to form a cropped video frame, wherein the first cropping window is dynamically adjusted in response to the received cropping information; and

sending, via an interface, the cropped video frame to the host apparatus for digital image stabilization;

using processing performed by the host apparatus to identify a second cropping window for the cropped video frame to at least partially generate cropping information for a subsequent current video frame.

2. A method as claimed in claim 1, wherein the method is performed at the camera module comprising:

the interface, which comprises an input interface configured to receive cropping information for the current video frame from the host apparatus;

the camera sensor configured to capture the current video frame;

cropping circuitry configured to crop the current video frame using the first cropping window to form a cropped video frame, wherein the first cropping window is dynamically adjusted in size in response to the received cropping information; and

the interface also comprises an output interface configured to send the cropped video frame for digital image stabilization in the host apparatus.

3. A method as claimed in claim 1, wherein the received cropping information is dependent upon one or more preceding video frames but not the current video frame.

4. A method as claimed in claim 1, wherein the cropping information defines a variable size of the first cropping window that is sufficiently larger than a region of interest to enable digital image stabilization of the region of interest.

5. A method as claimed in claim 1, wherein the cropping information defines a size of the first cropping window that is sufficiently larger than a region of interest to enable metadata to be displayed adjacent the region of interest.

6. A method as claimed in claim 1, further comprising:

performing, as part of digital image stabilization, a second crop of the cropped video frame using the second cropping window to form a double-cropped video frame.

7. A method as claimed in claim 6, further comprising:

determining the second crop by analyzing the current video frame.

8. A method as claimed in claim 1, wherein the cropping information identifies a cropping window used in digital image stabilization of a video frame preceding the current video frame.

9. A method as claimed in claim 1, wherein the cropping information comprises a variable size and location constraints for the first cropping window.

10. A method as claimed in claim 6, comprising introducing metadata into the double-cropped image.

11. A method as claimed in claim 1, comprising:

for each video frame in a sequence of video frames;

capturing a current video frame;

receiving cropping information for the current video frame;

dynamically adjusting a first cropping window in response to the cropping information;

performing a first crop of the current video frame using the first cropping window to form a cropped video frame; and

sending via a physical interface the cropped video frame for digital image stabilization.

21. The method as claimed in claim 1, further comprising:

performing a second crop of a subsequent video frame using the second cropping window to form a double cropped video frame, wherein the second cropping window is dynamically adjusted in size relative to the first cropping window.

12. An apparatus comprising:

a camera sensor configured to capture a current video frame;

cropping circuitry configured;

to receive cropping information for cropping the current video frame;

to define a variable size of a first cropping window in response to the cropping information;

to crop the current video frame using the first cropping window to form a cropped video frame; and

to send the cropped video frame for digital image stabilization at a host apparatus, the host apparatus is configured to re-use processing performed to identify a second cropping window for the cropped video frame to at least partially generate cropping information for a subsequent current video frame.

13. An apparatus as claimed in claim 12, wherein the apparatus is a camera module for use with a host apparatus comprising:

an input interface configured to receive cropping information for a current video frame from the host apparatus;

the camera sensor configured to capture the current video frame; and

cropping circuitry configured to crop the current video frame using a first cropping window to form a cropped video frame, wherein the first cropping window has been dynamically adjusted in size in response to the received cropping information; and

an output interface configured to send the cropped video frame for digital image stabilization in the host apparatus.

14. An apparatus as claimed in claim 12, wherein the cropping information is dependent upon one or more preceding video frames but not the current video frame.

15. A system comprising an apparatus as claimed in claim 12 and a host apparatus configured to perform, as part of digital image stabilization, a crop of the cropped video frame using the second cropping window to form a double-cropped video frame.

16. A system as claimed in claim 15, wherein the host apparatus is configured to determine the second crop by analyzing the current video frame.

17. A system as claimed in claim 15, wherein the cropping information identifies a cropping window used in digital image stabilization of a video frame preceding the current video frame.

18. A system as claimed in claim 15, wherein the host apparatus is configured to introduce metadata into the double-cropped image.

19. A system as claimed in claim 15, wherein the system is a wearable video camera system.

20. An apparatus comprising:

at least one processor; and

at least one non-transitory memory including computer program code, the at least one non-transitory memory and the computer program code configured to, with the at least one processor, cause a camera module at least to perform for each video frame in a series of video frames;

receiving cropping information for cropping a current video frame;

dynamically adjusting a first cropping window in response to the cropping information;

performing a first crop of the current video frame using the first cropping window to form a cropped video frame; and

sending via an interface the cropped video frame to a host apparatus for digital image stabilization, the host apparatus is configured to re-use processing performed to identify a second cropping window for the cropped video frame to at least partially generate cropping information for a subsequent current video frame.