Image processing method, recording medium carrying image processing program, and image processing apparatus

• US 8,588,469 B2
• Filed: 07/06/2012
• Issued: 11/19/2013
• Est. Priority Date: 02/15/2008
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Hayashi, Kazunori, Kawakami, Itaru, Sako, Yoichiro, Kon, Takayasu, Kamada, Yasunori
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US13/543,329
• Publication Number:
  US 20120275720A1
• Time in Patent Office: 501 Days
• Field of Search: None
• US Class Current: 382/104
• CPC Class Codes: B60R 2300/306 using a re-scaling of images
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

An image processing method includes the steps of extracting edges from each of a plurality of original images, thereby obtaining edge images; extracting straight lines from the edge images, thereby obtaining line images; assuming extensions of the straight lines constituting the line images and setting vanishing points on the basis of the extensions; and trimming the plurality of original images so that positions of the vanishing points in the respective original images mutually match, thereby obtaining final images.

Claims

1. An image processing method comprising the steps of:

extracting edges from each original image of a plurality of original images, thereby obtaining an edge image corresponding to the original image;

extracting straight lines from the edge image, thereby obtaining a line image;

determining extensions of the straight lines constituting the line image and setting a vanishing point based at least in part on the extensions; and

obtaining a final image corresponding to each original image by processing each original image of the plurality of original images so that a position of the vanishing point of each original image of the plurality of original images matches with a position of a vanishing point of each other original image of the plurality of original images wherein, when the original image is processed so that the position of the vanishing point of the original image matches with the position of the vanishing point of each other original image, inclination of the original image is corrected so that the straight lines in the line image obtained from the original image match straight lines in a line image obtained from each other original image of the plurality of original images.

2. The image processing method according to claim 1, further comprising:

if a non-image area exists in the original image when the original image is processed so that the position of the vanishing point of each original image matches with the position of the vanishing point of each other original image of a plurality of original images, extending the original image to fill the non-image area.

3. The image processing method according to claim 1,

wherein, when the final image is obtained, the final image has a size based on the original image.

4. The image processing method according to claim 1,

wherein the final image is stored in a storage device.

7. The image processing method according to claim 1, wherein the processing each original image of the plurality of original images comprises trimming each original image of the plurality of original images so that the position of the vanishing point of each original image of the plurality of original images matches with the position of the vanishing point of each other original image of the plurality of original images, thereby obtaining the final image corresponding to each original image.

10. The image processing method according to claim 1, further comprising:

capturing the plurality of original images.

11. The image processing method according to claim 10,wherein the plurality of original images are captured automatically by an image acquisition device carried by a user.

5. A computer-readable storage device storing computer-executable instructions comprising an image processing program that, when executed by a computer, cause the computer to execute the steps of:

extracting edges from each original image of a plurality of original images, thereby obtaining an edge image corresponding to the original image;

extracting straight lines from the edge image, thereby obtaining a line image;

determining extensions of the straight lines constituting the line image and setting a vanishing point based at least in part on the extensions; and

obtaining a final image corresponding to each original image by processing each original image of the plurality of original images so that a position of the vanishing point of each original image of the plurality of original images matches with a position of a vanishing point of each other original image of the plurality of original images wherein, when the original image is processed so that the position of the vanishing point of the original image matches with the position of the vanishing point of each other original image, inclination of the original image is corrected so that the straight lines in the line image obtained from the original image match straight lines in a line image obtained from each other original image of the plurality of original images.

8. The computer-readable storage device according to claim 5, wherein the computer-executable instructions, when executed by the computer, cause the computer to process each original image of the plurality of original images by trimming each original image of the plurality of original images so that the position of the vanishing point of each original image of the plurality of original images matches with the position of the vanishing point of each other original image of the plurality of original images, thereby obtaining the final image corresponding to each original image.

12. The computer-readable storage device according to claim 5, wherein the plurality of original images are captured automatically by an image acquisition device carried by a user.

6. An image processing apparatus comprising:

an edge extracting circuit configured to extract edges from each original image of a plurality of original images, thereby obtaining an edge image corresponding to the original image;

a line image circuit configured to extract straight lines from the edge image, thereby obtaining a line image;

a vanishing point setting circuit configured to determine extensions of the straight lines constituting the line image and set a vanishing point based at least in part on the extensions; and

a circuit configured to obtain a final image corresponding to each original image by processing each original image of the plurality of original images so that a position of the

vanishing point of each original image of the plurality of original images matches with a position of a vanishing point of each other original image of the plurality of original images wherein, when the original image is processed so that the position of the vanishing point of the original image matches with the position of the vanishing point of each other original image, inclination of the original image is corrected so that the straight lines in the line image obtained from the original image match straight lines in a line image obtained from each other original image of the plurality of original images.

9. The image processing apparatus according to claim 6, wherein the circuit is configured to process each original image of the plurality of original images by trimming each original image of the plurality of original images so that the position of the vanishing point of each original image of the plurality of original images matches with the position of the vanishing point of each other original image of the plurality of original images, thereby obtaining the final image corresponding to each original image.

13. The image processing apparatus according to claim 6, further comprising:

an image acquisition device configured to capture the plurality of original images.

14. The image processing apparatus according to claim 13, wherein:

the image acquisition device is configured to capture the plurality of original images automatically.