Estimating vanishing points in images

• US 8,396,285 B2
• Filed: 04/20/2009
• Issued: 03/12/2013
• Est. Priority Date: 04/20/2009
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Maurer, Ron, Kogan, Hadas, Keshet, Renato
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US12/426,386
• Publication Number:
  US 20100266212A1
• Time in Patent Office: 1,422 Days
• Field of Search: None
• US Class Current: 382/154
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

A digital image is processed to provide an estimation of the position in the image plane of a vanishing point. The processing includes detecting pairs of similar image patches and identifying a concurrent set of straight virtual lines that substantially converge at a point on the image plane, each line passing through a pair of similar image patches within the image.

Claims

1. An automated image processor arranged to receive original digital image data and estimate the position of a vanishing point in an image plane of a respective original digital image, including by detecting pairs of similar image patches within an image and identifying a concurrent set of straight virtual lines that substantially converge at a point on the image plane, each line passing through a pair of similar image patches within the image.

2. The image processor of claim 1, wherein pairs of similar patches are detected by matching structural image features characterising the patches.

3. The image processor of claim 1, arranged to estimate the position of a vanishing point of an image by identifying a concurrent set of straight virtual lines that substantially converge at a point in the image plane, including outside of the image boundaries.

4. The image processor of claim 1, arranged to locate one or more pairs of image strips in the image and, for each pair of image strips, identify a candidate vanishing point by determining at least one set of concurrent straight virtual lines, each passing through an image patch in a first of the image strips and a similar image patch in a second of the image strips.

5. The image processor of claim 4, arranged to locate a plurality of pairs of image strips and a respective plurality of candidate vanishing points, and estimate the position of a vanishing point on the image plane by applying a clustering process to the plurality of vanishing point candidates.

6. The image processor of claim 4, arranged for each pair of image strips to estimate a one-dimensional affine transformation that maximizes the similarity between the image strips under an image strip similarity measure.

7. The image processor of claim 4, arranged for each pair of image strips to compare each image location within the first image strip with each image location within the second image strip to measure the structural similarity between the corresponding image patches.

8. The image processor of claim 7, arranged to generate for each image location to be compared a feature vector and attribute a similarity measure for each pair of image locations by computing a structural correlation evidence (SCORE) measure between the corresponding feature vectors.

9. The image processor of claim 8, wherein a SCORE measure attributes a high similarity score only to pairs of feature vectors that each have large magnitudes and are substantially aligned.

10. The image processor of claim 9, wherein the SCORE measures are collected into a two-dimensional SCORE matrix, in which each dimension represents the image locations of one image strip, and a straight line is fitted with respect to locations in the matrix representing SCORE measures having relatively high similarity values.

11. The image processor of claim 10, wherein slope and offset parameters of the fitted line determine the scaling and translation parameters of the best similarity transformation between respective image strips, in order to identify a candidate vanishing point for the respective pair of image strips.

12. The image processor of claim 11, wherein the line is fitted with respect to matrix locations having relatively high similarity values including by solving a maximisation problem, in which locations that are relatively more spatially distant in the matrix have a relatively increased significance in the maximisation problem.

13. The image processor of claim 6, wherein, for each pair of image strips, a vanishing point candidate is computed in the pencil slope coordinates from the affine transformation parameters according to Equations 5-7.

14. The image processor of claim 13, wherein the vanishing point candidates are clustered in a finite pencil-slope domain with reference to two different reference points, to estimate vanishing points located anywhere in the infinite image plane.

15. The image processor of claim 4, wherein the image strips have a relatively narrow elongated rectangular shape.

16. The image processor of claim 4, wherein the image strips in a pair are substantially parallel and spaced apart in the image.

17. The image processor of claim 5, wherein there are plural pairs of substantially vertical image strips.

18. The image processor of claim 5, wherein there are plural pairs of substantially horizontal image strips.

19. The image processor of claim 1, wherein the original digital image is first downscaled to generate a relatively low resolution version thereof, on which subsequent processing steps are carried out, whereby a vanishing point of the original digital image is estimated by first estimating a vanishing point of the low resolution version.

23. The image processor of claim 1, wherein the detecting is based on comparisons of feature vectors respective characterizing the image patches.

20. An image processor comprising:

a. an input to receive digital data representing a digital image;

b. a data store to store a received digital image; and

c. a vanishing point estimator, which is adapted to estimate the position of a vanishing point in an image plane of a digital image, including by detecting pairs of similar image patches within an image and identifying a concurrent set of straight virtual lines that substantially converge at a point on the image plane, each line passing through a pair of similar image patches within the image.

21. An automated method of processing an image by estimating the position of a vanishing point in an image plane of a respective original digital image, including by detecting pairs of similar image patches within an image and identifying a concurrent set of straight virtual lines that substantially converge at a point on the image plane, each line passing through a pair of similar image patches within the image.

22. A non-transitory computer readable medium having stored thereon processing instructions which, when executed, enact an image processing method as claimed in claim 21.