Image processing apparatus and image processing method

• US 10,510,163 B2
• Filed: 08/25/2017
• Issued: 12/17/2019
• Est. Priority Date: 01/13/2017
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Oki, Yutaka
• Primary Examiner(s): Woldemariam, Aklilu K
• Application Number:
  US15/686,442
• Publication Number:
  US 20180204351A1
• Time in Patent Office: 844 Days
• Field of Search: 382106, 382154, 382254, 348 7, 348 42, 348 51, 348135, 348 43, 348 44, 348 47
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

According to an embodiment, an image processing apparatus includes a matching unit configured to perform stereo matching processing on two images obtained through imaging using two cameras, an error map creating unit configured to create an error map indicating a displacement amount of the two images in a direction orthogonal to a search line for the two images subjected to the stereo matching processing, and a parallelization parameter error determining unit configured to determine whether or not there is an error of rectification which parallelizes the two images, based on the error map.

Claims

1. An image processing apparatus comprising:

a rectification circuit configured to parallelize a first image and a second image based on parameter information for performing at least one of rotation and parallel movement on one or both of the first image and the second image;

a matching circuit configured to calculate and output y disparity by performing stereo matching processing on the first image and the second image outputted from the rectification circuit, the stereo matching processing being performed by calculating a degree of matching of each pixel in the first image while moving a window of a predetermined size in an X direction in the second image, and when the movement in the X direction is finished, changing from a line to a next line and moving the window in the X direction;

a displacement amount information creating circuit configured to create, by using the y disparity, displacement amount information of the images in a direction orthogonal to a search line for the first and second images subjected to the stereo matching processing; and

a rectification error determining circuit configured to determine whether or not there is an error of rectification which parallelizes the first and second images in the rectification circuit, based on the displacement amount information.

2. The image processing apparatus according to claim 1, further comprising:

a parameter correcting circuit configured to generate a correction signal for correcting the error of the rectification based on the displacement amount information.

3. The image processing apparatus according to claim 1, further comprising:

a parameter correcting circuit configured to correct a parameter of a rectification circuit which parallelizes the images, based on the displacement amount information.

8. The image processing apparatus according to claim 1, wherein the matching circuit is further configured to calculate and output x disparity, and wherein the displacement amount information creating circuit is configured to create, as the displacement amount information, an error map created by disposing data of the y disparity at a pixel position of the x disparity.

4. An image processing method comprising:

performing rectification for parallelizing a first image and a second image obtained through imaging using cameras, based on parameter information for performing at least one of rotation and parallel movement on one or both of the first and second images;

calculating y disparity by performing stereo matching processing on the first and second images subjected to the rectification, the stereo matching processing being performed by calculating a degree of matching of each pixel in the first image while moving a window of a predetermined size in an X direction in the second image, and when the movement in the X direction is finished, changing from a line to a next line and moving the window in the X direction;

creating, by using the y disparity, displacement amount information between the images in a direction orthogonal to a search line for the first and second images obtained through the stereo matching processing; and

determining whether or not there is an error of rectification which parallelizes the first and second images in the rectification, based on the displacement amount information.

5. The image processing method according to claim 4, further comprising:

generating a correction signal for correcting the error of the rectification based on the displacement amount information.

6. The image processing method according to claim 4, further comprising:

parallelizing the first and second images.

7. The image processing method according to claim 4, further comprising:

correcting a parameter of a rectification circuit which parallelizes the first and second images, based on the displacement amount information.

9. The image processing method according to claim 4, further comprising:

performing the stereo matching processing to calculate x disparity, wherein the displacement amount information includes an error map created by disposing data of the y disparity at a pixel position of the x disparity.