Automatic perspective control using vanishing points

• US 10,354,364 B2
• Filed: 09/14/2015
• Issued: 07/16/2019
• Est. Priority Date: 09/14/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Pyo, Sean, Lee, Sangyup, Park, Minje
• Primary Examiner(s): Motsinger, Sean T
• Application Number:
  US14/853,272
• Publication Number:
  US 20170076434A1
• Time in Patent Office: 1,401 Days
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/13 Edge detection

Abstract

Techniques related to automatic perspective control of images using vanishing points are discussed. Such techniques may include determining a perspective control vanishing point associated with the image based on lines detected within the image, rotating the image based on the perspective control vanishing point to generate an aligned image, and warping the aligned image based on aligning two lines of the detected lines that meet at the perspective control vanishing point.

Claims

1. A method for providing automatic perspective control for an image comprising:

determining a perspective control vanishing point associated with the image based on a plurality of lines detected within the image by dividing an image space associated with the image into a plurality of bins, inserting individual intersection points of the plurality of lines into their respective bins of the plurality of bins, and determining a first vanishing point associated with a bin of the plurality of bins having a largest number of intersection points;

rotating the image based on the perspective control vanishing point to generate an aligned image; and

warping the aligned image based on at least partially aligning two lines of the plurality of lines that meet at the perspective control vanishing point to generate a perspective controlled image.

2. The method of claim 1, further comprising:

detecting edges within the image to generate an edge image; and

generating the plurality of lines based on applying a line detection to the edge image.

3. The method of claim 2, wherein applying the line detection comprises:

applying a transform to the edge image to generate a transform map; and

including a line with a largest number of associated edge pixels in the transform map in the plurality of lines.

4. The method of claim 3, further comprising:

applying a decremental transform to the transform map to generate a second transform map; and

including a second line with a largest number of associated edge pixels in the second transform map in the plurality of lines.

5. The method of claim 1, wherein the bins comprise 2-dimensional angular and radial bins.

6. The method of claim 1, wherein determining the first vanishing point comprises assigning the first vanishing point only if the first vanishing point is external to a frame of the image.

7. The method of claim 1, further comprising:

determining at least a second vanishing point; and

selecting the perspective control vanishing point from the first or second vanishing points as the vanishing point from the first or second vanishing points that is closest to a vertical axis associated with the image.

8. The method of claim 1, wherein the perspective control vanishing point comprises the first vanishing point and the method further comprises associating lines that meet at the perspective vanishing point as perspective control lines, wherein the two lines comprise two outer lines of the perspective control lines.

9. The method of claim 1, wherein rotating the image comprises rotating the image about its center to move the perspective control vanishing point to a vertical axis of the image.

10. The method of claim 1, wherein warping the aligned image comprises:

generating a homographic mapping based on aligning the two lines to parallel; and

applying the homographic mapping to the aligned image.

11. A system for providing automatic perspective control for an image comprising:

a memory configured to store an image; and

a processor coupled to the memory, the processor to;

determine a perspective control vanishing point associated with the image based on a plurality of lines detected within the image, wherein the processor to determine the perspective control vanishing point comprises the processor to divide an image space associated with the image into a plurality of bins, insert individual intersection points of the plurality of lines into their respective bins of the plurality of bins, and determine a first vanishing point associated with a bin of the plurality of bins having a largest number of intersection points,

rotate the image based on the perspective control vanishing point to generate an aligned image, and

warp the aligned image based on at least partially aligning two lines of the plurality of lines that meet at the perspective control vanishing point to generate a perspective controlled image.

12. The system of claim 11, wherein the processor is further to detect edges within the image to generate an edge image and generate the plurality of lines based on an application of a line detector to the edge image.

13. The system of claim 12, wherein the application of the line detector comprises the processor to apply a transform to the edge image to generate a transform map and include a line with a largest number of associated edge pixels in the transform map in the plurality of lines.

14. The system of claim 11, wherein the processor to determine the first vanishing point comprises the processor to assign the first vanishing point only if the first vanishing point is external to a frame of the image.

15. The system of claim 11, wherein the processor is further to determine at least a second vanishing point and to select the perspective control vanishing point from the first or second vanishing points as the vanishing point from the first or second vanishing points that is closest to a vertical axis of the image.

16. The system of claim 11, wherein the processor to rotate the image comprises the processor to rotate the image about its center to move the perspective control vanishing point to a vertical axis of the image.

17. The system of claim 11, wherein the processor to warp the aligned image comprises the processor to generate a homographic mapping based on aligning the two lines to parallel and to apply the homographic mapping to the aligned image.

25. The system of claim 11, wherein the processor is further to apply a decremental transform to the transform map to generate a second transform map and include a second line with a largest number of associated edge pixels in the second transform map in the plurality of lines.

18. At least one non-transitory machine readable medium comprising a plurality of instructions that, in response to being executed on a device, cause the device to provide automatic perspective control for an image by:

determining a perspective control vanishing point associated with the image based on a plurality of lines detected within the image by dividing an image space associated with the image into a plurality of bins, inserting individual intersection points of the plurality of lines into their respective bins of the plurality of bins, and determining a first vanishing point associated with a bin of the plurality of bins having a largest number of intersection points;

rotating the image based on the perspective control vanishing point to provide an aligned image; and

warping the aligned image based on at least partially aligning two lines of the plurality of lines that meet at the perspective control vanishing point to generate a perspective controlled image.

19. The non-transitory machine readable medium of claim 18, the machine readable medium comprising further instructions that, in response to being executed on the device, cause the device to provide automatic perspective control for the image by:

detecting edges within the image to generate an edge image; and

generating the plurality of lines based on applying a line detection to the edge image.

20. The non-transitory machine readable medium of claim 19, wherein applying the line detection comprises:

applying a transform to the edge image to generate a transform map; and

including a line with a largest number of associated edge pixels in the transform map in the plurality of lines.

21. The non-transitory machine readable medium of claim 18, the machine readable medium comprising further instructions that, in response to being executed on the device, cause the device to provide automatic perspective control for the image by:

determining at least a second vanishing point; and

selecting the perspective control vanishing point from the first or second vanishing points as the vanishing point from the first or second vanishing points that is closest to a vertical axis of the image.

22. The non-transitory machine readable medium of claim 18, wherein warping the aligned image comprises:

generating a homographic mapping based on aligning the two lines to parallel; and

applying the homographic mapping to the aligned image.

23. The non-transitory machine readable medium of claim 20, the machine readable medium comprising further instructions that, in response to being executed on the device, cause the device to provide automatic perspective control for the image by:

applying a decremental transform to the transform map to generate a second transform map; and

including a second line with a largest number of associated edge pixels in the second transform map in the plurality of lines.

24. The non-transitory machine readable medium of claim 18, wherein determining the first vanishing point comprises assigning the first vanishing point only if the first vanishing point is external to a frame of the image.