METHOD AND APPARATUS FOR DETERMINING THE MIS-ALIGNMENT IN IMAGES

• US 20110058749A1
• Filed: 08/09/2010
• Published: 03/10/2011
• Est. Priority Date: 09/04/2009
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: COOPER, Richard Jared
• Granted Patent: US 8,526,762 B2
• Time in Patent Office: Days
• US Class Current: 382/218
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation

Abstract

A method of determining the amount of mis-alignment in one direction of an image of a scene, the image captured by a first camera is described. The method comprising the steps of:

• • defining a line between a first point in the image and a second point in the image, the line extending in the direction of mis-alignment;
  • defining a reference section in the scene to which the image is to be aligned, the reference section of the scene having a reference image characteristic;
  • identifying an image characteristic at different points along the line in the image; and
  • comparing the image characteristic at each point along the line with the reference image characteristic, and determining the amount of mis-alignment by determining the distance between the position of the reference image characteristic of the reference section captured in the image and the defined reference section in the scene.

Claims

1. A method of determining the amount of mis-alignment in one direction of an image of a scene, the image captured by a first camera and the method comprising:

defining a line between a first point in the image and a second point in the image, the line extending in the direction of mis-alignment;

defining a reference section in the scene to which the image is to be aligned, the reference section of the scene having a reference image characteristic;

identifying an image characteristic at different points along the line in the image;

comparing the image characteristic at each point along the line with the reference image characteristic, and determining the amount of mis-alignment by determining the distance between the position of the reference image characteristic of the reference section captured in the image and the defined reference section in the scene; and

defining at least one of a circular sampling path and a radial sampling path in the image, wherein the circular sampling path defines a circular sampling path between the first and second point and the radial sampling path has a centre point coincident with a corresponding point in said image and extending along a radial line therefrom, the line between the first and second points in the image being defined by one or more of the circular sampling path or the radial sampling path.

2. A method according to claim 1, further comprising:

capturing a second image using a second camera, whereby the first and second image have an area of overlap and the reference section in the scene is located in the area of overlap;

defining a second line between a first point in the second image to a second point in the second image the second line being drawn in the direction of mis-alignment and intersecting with the reference section, the reference section having a second reference image characteristic;

identifying an image characteristic at different points along the second line in the second image; and

comparing the image characteristic at each point along the line in the first image with the image characteristic at each point along the second line in the second image, whereby the amount of mis-alignment is determined in accordance with the difference in position between the position of the reference image characteristic in the first image and the second reference image characteristic in the second image.

3. A method according to either one of claim 1, wherein the image characteristic in the first or second image is formed by interpolation of a plurality of pixel values.

4. A method according to claim 3, wherein the interpolation is bilinear interpolation of the nearest four pixel values.

5. A method according to claim 1, comprising displaying a polar and/or a radial template in the first image, the polar template having a circular template line and a radial template line having a centre point coincident with a corresponding point in each respective image and extending along a radial line therefrom, the line between the first and second points in the first image being defined by one or more of the circular template line and/or the radial template line.

6. A method according to claim 1, comprising determining the image characteristics at positions in an area surrounding the first or second line, the positions being a substantially similar distance from the optical axis of the camera capturing the image.

7. A method according to claim 1, wherein the centre point through which the radial line passes is the optical axis of the camera capturing the image.

8. A method according to claim 1, wherein the centre of the circular sampling path corresponds to the optical axis of the camera capturing the image.

9. A method according to claim 1, comprising plotting the value of an image property at each sampling point along the sampling path against the position of the sampling point along the sampling path.

10. A method of aligning an image, comprising:

a method of determining the non-alignment as claimed in any preceding claim; and

transforming the first image until the distance between the image characteristic of the captured section and the reference section is within a predetermined threshold.

11. A method according to claim 10, wherein the first image is transformed along the line between the first and second point.

12. A method according to claim 11, wherein the line in the first image and the line in the second image is a circular line, and the first image has a roll transformation applied thereto.

13. A method according to claim 11, wherein the line in the first image and the line in the second image is a radial line and the second image has a transformation applied along the length of the radial line.

14. A computer program comprising computer readable instructions which, when loaded onto a computer, configures the computer to perform a method according to claim 1.

15. A computer readable storage medium configured to store the computer program of claim 14 therein or thereon.

16. An apparatus for determining the amount of mis-alignment in one direction of an image of a scene, the image captured by a first camera and the apparatus comprising:

a definer operable to define a line between a first point in the image and a second point in the image, the line in the direction of mis-alignment;

the definer being further operable to define a reference section in the scene to which the image is to be aligned, the reference section of the scene having a reference image characteristic;

an identifier operable to identify an image characteristic at different points along the line in the image;

a comparator operable to comparing the image characteristic at each point along the line with the reference image characteristic, and determining the amount of mis-alignment by determining the distance between the position of the reference image characteristic of the reference section captured in the image and the defined reference section in the scene; and

a definer operable to define at least one of a circular or radial sampling path in the image, the circular sampling path defining a circular sampling path between the first and second point and a radial sampling path having a centre point coincident with a corresponding point in said image and extending along a radial line therefrom, the line between the first and second points in the image being defined by one or more of the circular sampling path or the radial sampling path.

17. An apparatus according to claim 16, further comprising:

an input connectable to a second camera which is operable to capture a second image, whereby the first and second image have an area of overlap and the reference section in the scene is located in the area of overlap;

the definer being further operable to define a second line extending between a first point in the second image to a second point in the second image the second line extending in the direction of mis-alignment and intersecting with the reference section, the reference section having a second reference image characteristic;

the identifier operable to identify an image characteristic at different points along the second line in the second image; and

the comparator operable to compare the image characteristic at each point along the line in the first image with the image characteristic at each point along the second line in the second image, whereby the amount of mis-alignment is determined in accordance with the difference in position between the position of the reference image characteristic in the first image and the second reference image characteristic in the second image.

18. An apparatus according to claim 16, wherein the image characteristic in the first or second image is formed by interpolation of a plurality of pixel values.

19. An apparatus according to claim 18, wherein the interpolation is bilinear interpolation of the nearest four pixel values.

20. An apparatus according to claim 16, comprising a display operable to display at least one of a polar or a radial template in the first image, the polar template having a circular template line and a radial template line having a centre point coincident with a corresponding point in each respective image and extending along a radial line therefrom, the line between the first and second points in the first image being defined by one or more of the circular template line or the radial template line.

21. An apparatus according to claim 16, wherein the determiner is operable to determine the image characteristics at positions in an area surrounding the first or second line, the positions being a substantially similar distance from the optical axis of the camera capturing the image.

22. An apparatus according to claim 16, wherein the point from which the radial line radiates is the optical axis of the camera capturing the image.

23. An apparatus according to claim 16, wherein the centre of the circular sampling path corresponds to the optical axis of the camera.

24. An apparatus according to claim 16, further comprising a plotter operable to plot the value of an image property at each sampling point along the sampling path against the position of the sampling point along the sampling path.

25. A device for aligning an image, comprising:

an apparatus for determining the mis-alignment as claimed in claim 16; and

a transformer operable to transform the first image until the distance between the image characteristic of the captured section and the reference section is below a threshold distance.

26. A device according to claim 25, wherein the first image is transformed along the line extending between the first and second point.

27. A device according to claim 25, wherein the line in the first image and the line in the first image is a circular line, and the first image has a roll transformation applied thereto.

28. A device according to claim 26, wherein the line in the first image and the line in the second image is a radial line and the first image has a transformation applied along the length of the radial line.

29. A system comprising a first and second camera connected to an apparatus according to claim 16.