Method and device for determining an image shift in an image sequence

• US 7,053,962 B1
• Filed: 01/15/2000
• Issued: 05/30/2006
• Est. Priority Date: 03/05/1999
• Status: Expired due to Fees

• Current Assignee: Robert Bosch GmbH
• Original Assignee: Robert Bosch GmbH
• Inventors: Engelsberg, Andreas
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US09/914,977
• Time in Patent Office: 2,327 Days
• Field of Search: 348/699, 348/700, 348/207, 348/522, 348/333, 348/272, 348/222, 348/208, 396/52, 396/53, 396/54
• US Class Current: 348/699
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

A method and a device for determining an image shift in an image sequence may be used, for example, to compensate for a camera movement or other image source movement. A plurality of image zones of images are available to determine the shift, with the zone shift of any given image zone from a plurality of image zones being determined from the image data of, for example, consecutive images within any given image zone. A block matching method or a method using block shift information from a block-based coding method is used to determine the zone shift and to determine a reliability for the zone shift determination.

Claims

1. A method for determining an image shift in an image sequence to compensate for an image source movement, a plurality of image zones of images being available to determine the image shift, each of the plurality of image zones being at a specific position in the images and each having predefined dimensions of predefined numbers of pixels in different image directions, the method comprising:

determining the image shift from one of (i) first image data of the first image and second image data of a second image, and (ii) the first image data of the first image and input image data of an input image for use in correcting the first image in the image sequence;

determining a zone shift of any given image zone of the plurality of image zones from one of (i) the first image data of the first image and the second image data of the second image within the given image zone, and (ii) the first image data of the first image and the input image data of the input image within the given image zone, including determining the zone shift in two image zones and determining a reliability for the zone shift determination by;

forming the zone shift and a correlation quotient for each of the two image zones;

determining a threshold value function as a function of a corresponding value of determined zone shifts in the two image zones;

comparing the correlation quotient of one of the two image zones to a comparison value obtained from the threshold value function for a zone shift of another of the two image zones;

determining that the reliability of the zone shift determination is adequate for the one image zone of the two image zones if the correlation quotient determined for the one image zone is greater than the comparison value; and

using the zone shift of one image zone of the plurality of image zones as the image shift as a function of the reliability for the zone shift determination of the one image zone.

2. The method of claim 1, wherein the correlation quotient for one of the plurality of image zones is determined by:

determining shift correlation values for multiple possible zone shifts using block-matching;

determining the zone shift of the plurality of image zones to be a zone shift associated with a maximum of the shift correlation values; and

forming the correlation quotient by dividing the maximum of the shift correlation values by an average of the shift correlation values.

3. The method of claim 1, wherein:

the threshold value function assumes a predefined second threshold value for a given zone shift value less than a predefined first threshold value;

the threshold value function assumes a value that is the predefined second threshold value minus a product for the given zone shift value greater than the predefined first threshold value;

the product includes as factors a predefined gradient parameter and a difference; and

the difference is formed from the given zone shift and the predefined first threshold value.

4. The method of claim 1, further comprising:

separating an image movement produced by the image source movement from an additional movement superimposed on the image movement in at least one of the image zones of the image to be corrected by;

determining that a probability that the image movement will occur without the additional movement at different image positions; and

determining the position and dimensions of a given image zone, and permanently specifying as a function of the probability that the image movement will occur without the additional movement within the given image zone.

5. The method of claim 4, wherein the image source is a camera.

6. The method of claim 4, wherein at least one first image zone is used to determine the image shift.

7. The method of claim 1, further comprising:

selecting a position and dimensions of at least one first image zone so that the at least one first image zone of the images to be corrected is largely filled by an image background.

8. The method of claim 7, wherein the at least one first image zone and at least one second image zone are available to determine the image shift.

9. The method of claim 7, wherein two first image zones and a single second image zone are available for correcting the image, and the image shift is determined, in descending order of priority, from one of:

an average of zone shifts of the first two image zones if the reliability of the zone shift determination of the first two image zones is determined to be adequate;

the zone shift of one of the two first image zones for which the reliability of the zone shift determination is determined to be adequate; and

a zone shift of the single second image zone.

10. The method of claim 9, wherein the source image movement is a camera movement and the method is used for a head-and-shoulder shot, further comprising:

selecting the first two image zones in a side area to the left and right of a vertical center axis of a predefined rectangular image; and

selecting the single second image zone in a center of the image with respect to the vertical center axis of the rectangular image;

wherein;

a first bottom distance of the first two image zones from a bottom of the image is greater than a first top distance of the first two image zones from a top of the image; and

a second top distance of the single second image zone from the top of the image is greater than the second bottom distance of the single second image zone from the bottom of the image.

11. The method of claim 10, wherein:

the first two image zones is selected in a side area symmetric to the vertical center axis of the predefined rectangular image; and

the single second image zone is selected in a center of the image symmetric to the vertical center axis of the rectangular image.

12. The method of claim 1, further comprising:

selecting position and dimensions of at least one second image zone so that the at least one second image zone of the images to be corrected is largely filled with an image foreground.

13. The method of claim 12, wherein at least one first image zone and the at least one second image zone are available to determine the image shift.

14. The method of claim 1, wherein the image source is a camera.

15. The method of claim 1, wherein one of the following is satisfied:

the second image data of the second image directly precedes the first image in the image sequence; and

the input image data of the input image directly precedes the first image in the image sequence.

16. The method of claim 1, further comprising:

selecting a position and dimensions of at least one first image zone so that the at least one first image zone of the images to be corrected is largely filled by an image background; and

selecting position and dimensions of at least one second image zone so that the at least one second image zone of the images to be corrected is largely filled with an image foreground.

17. The method of claim 16, wherein the at least one first image zone and the at least one second image zone are available to determine the image shift.

18. A method for determining an image shift in an image sequence of a plurality of images to compensate for a camera movement, at least one image zone of the plurality of images being available to determine the image shift, the at least one image zone being at a predefined position in the images and having predefined dimensions of predefined numbers of pixels in different image directions, the method comprising:

determining the image shift from one of (i) first image data of a first image and second image data of a second image, and (ii) the first image data of the first image and input image data of an input image by using a zone shift of the at least one image zone as the image shift; and

determining the zone shift using block shift information from block-based coding used for the at least one image zone, wherein;

image blocks located in at least one image zone are reflected in the block shift information of the at least one image zone,

the at least one image zone is used as a function of a reliability of a zone shift determination, and

the zone shift for the at least one image zone, the zone shift including a horizontal component and a vertical component, and the reliability of a zone shift determination are determined by;

establishing a first frequency distribution of frequencies of different values for a horizontal component of the block shift information to determine the horizontal component of the zone shift, the horizontal component of the zone shift corresponding to a horizontal component value of the block shift information for which the first frequency distribution assumes its primary maximum;

establishing a second frequency distribution of frequencies of different values for a vertical component of the block shift information to determine the vertical component of the zone shift, the vertical component of the zone shift corresponding to a vertical component value of the of the block shift information for which the second frequency distribution assumes its primary maximum;

determining that the reliability of the zone shift is adequate when the following conditions are met;

an absolute value of a difference in position of values corresponding to the primary maximum and a secondary maximum of the first frequency distribution of the horizontal component of the block shift information is less than a predefined first difference threshold;

the absolute value of the difference in position of the values corresponding to the primary maximum and a secondary maximum of the second frequency distribution of the vertical component of the block shift information is less than a predefined second difference threshold;

the primary maximum of the first frequency distribution is greater than a first frequency threshold; and

the primary maximum of the second frequency distribution is greater than a second frequency threshold.

19. A device for determining an image shift, comprising:

an image shift detecting arrangement, including a zone shift detector, an image storage device, and a microcomputer, wherein the shift detecting arrangement determines the image shift; and

an enlarging arrangement;

wherein the device is operable to determine the image shift in an image sequence to compensate for an image source movement, a plurality of image zones of images being available to determine the image shift, each of the plurality of image zones being at a specific position in the images and each having predefined dimensions of predefined numbers of pixels in different image directions, by;

determining the image shift from one of (i) first image data of the first image and second image data of a second image, and (ii) the first image data of the first image and input image data of an input image for use in correcting the first image in the image sequence;

determining a zone shift of any given image zone of the plurality of image zones from one of (i) the first image data of the first image and the second image data of the second image within the given image zone, and (ii) the first image data of the first image and the input image data of the input image within the given image zone, including determining the zone shift in two image zones and determining a reliability for the zone shift determination by;

forming the zone shift and a correlation quotient for each of the two image zones;

determining a threshold value function as a function of a corresponding value of determined zone shifts in the two image zones;

comparing the correlation quotient of one of the two image zones to a comparison value obtained from the threshold value function for a zone shift of another of the two image zones; and

determining that the reliability of the zone shift determination is adequate for the one image zone of the two image zones if the correlation quotient determined for the one image zone is greater than the comparison value; and

using the zone shift of one image zone of the plurality of image zones as the image shift as a function of the reliability for the zone shift determination of the one image zone.