Image processor, image processing method and program, and imaging system

• US 10,666,860 B2
• Filed: 09/10/2013
• Issued: 05/26/2020
• Est. Priority Date: 09/11/2012
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Takenaka, Hirokazu, Terao, Noriyuki, Irino, Yoshiaki, Tanaka, Tomonori, Imae, Nozomi, Harada, Toru, Yamamoto, Hideaki, Masuda, Kensuke, Ito, Yoichi, Sawaguchi, Satoshi, Bessho, Daisuke, Satoh, Hiroyuki, Shohara, Makoto, Takasu, Shusaku
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Fereja, Samuel D
• Application Number:
  US14/022,987
• Publication Number:
  US 20140071227A1
• Time in Patent Office: 2,450 Days
• Field of Search: None
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

An image processor includes a first converter to convert input images into images in a different coordinate system from that of the input images according to first conversion data based on a projection model, a position detector to detect a connecting position of the images converted by the converter, a corrector to correct the first conversion data on the basis of a result of the detection by the position detector, and a data generator to generate second conversion data for image synthesis from the conversion data corrected by the corrector on the basis of coordinate conversion, the second conversion data defining the conversion of the input images.

Claims

1. An image processor, comprising:

circuitry configured to;

convert input images, having a first coordinate system, into converted images in a second coordinate system different from the first coordinate system according to first conversion data based on a projection model, the first conversion data including arrays for a conversion of coordinate values from the first coordinate system to the second coordinate system;

for each respective pixel of an overlapping area of a first image and a second image of the converted images;

set a pixel block of predetermined dimensions around the respective pixel in the overlapping area of the first image as a pattern image; and

perform pattern matching, according to a correlation coefficient, on the second image at the respective pixel while moving the pattern image vertically and horizontally to find a pixel connecting position in the second image that correspond to the respective pixel in the first image;

detect, according to the pattern matching, connecting points of the overlapping area between the first image and the second image;

correct the arrays of the first conversion data according to the connecting points to produce corrected first conversion data; and

generate second conversion data for image synthesis, the image synthesis to be performed on the input images, the second conversion data being generated by performing a rotational coordinate conversion on the corrected first conversion data,

wherein the first conversion data and the second conversion data include respective conversion tables including information such that an angle of one pixel is at least 1/10 of a degree in both a θ

direction with θ

being from 1 to 360 degrees, and a ϕ

direction with ϕ

being from 0 to 180 degrees, and

wherein the conversion tables include information indicating at least 3,600*1,800 relations for the first image, and at least 3,600*1,800 relations for the second image.

2. The image processor according to claim 1, wherein an overlapping area between the input images is projected near an equatorial line of a spherical surface.

3. The image processor according to claim 1, wherein the circuitry is further configured to:

convert the input images into new converted images in the second coordinate system according to the second conversion data; and

synthesize the new converted images to generate a synthetic image represented in the second coordinate system.

4. The image processor according to claim 3, wherein

the input images are captured with the lens system, the lens system having an angle of view of larger than 180 degrees, and

the synthetic image is an omnidirectional image represented in a coordinate with at least two arguments.

5. The image processor according to claim 1, wherein the first conversion data is used to define a projection from an input image represented in a plane coordinate system to an image represented in a polar coordinate system with at least two arguments.

6. The image processor according to claim 1, wherein

the input images are captured with different lens systems, and

the first conversion data includes sets of data for correcting a distortion of the input images captured with the different lens systems for projection.

7. The image processor according to claim 1, wherein

the input images are represented in a plane coordinate system, and

the converted images are represented in a spherical coordinate system.

8. The image processor according to claim 1, wherein the rotational coordinate conversion corresponds to a 90 degree rotational coordinate conversion.

9. The image processor according to claim 1, wherein

the overlapping area is defined by horizontal angles of 0 to 360 degrees, and

vertical angles that are start and end points of the vertical angle of an overlapping area preset in accordance with total angle of view of a lens system generating the input images.

10. An image processing method for a computer having circuitry, the method comprising:

receiving, by the circuitry, input images having a first coordinate system;

converting the input images into converted images in a second coordinate system different from the first coordinate system according to first conversion data based on a projection model, the first conversion data including arrays for a conversion of coordinate values from the first coordinate system to the second coordinate system;

for each respective pixel of an overlapping area of a first image and a second image of the converted images;

setting a pixel block of predetermined dimensions around the respective pixel in the overlapping area of the first image as a pattern image; and

performing pattern matching, according to a correlation coefficient, on the second image at the respective pixel while moving the pattern image vertically and horizontally to find a pixel connecting position in the second image that correspond to the respective pixel in the first image;

detecting, according to the pattern matching, connecting points of the overlapping area between the first image and the second image;

correcting the arrays of the first conversion data according to the connecting points to produce corrected first conversion data; and

generating second conversion data for image synthesis, the image synthesis to be performed on the input images, the second conversion data being generated by performing a rotational coordinate conversion on the corrected first conversion data,

wherein the first conversion data and the second conversion data include respective conversion tables including information such that an angle of one pixel is at least 1/10 of a degree in both a θ

direction with θ

being from 1 to 360 degrees, and a ϕ

direction with ϕ

being from 0 to 180 degrees, and

wherein the conversion tables include information indicating at least 3,600*1,800 relations for the first image, and at least 3,600*1,800 relations for the second image.

11. The image processing method according to claim 10, wherein the generating includes

finding, by the rotational coordinate conversion, a coordinate value of the second coordinate system of each of the converted images in association with a coordinate value of a coordinate system of a synthetic image, and

acquiring a coordinate value of an input image associated with the coordinate value.

12. The image processing method according to claim 10, further comprising:

converting the input images into new converted images in the second coordinate system according to the second conversion data; and

synthesizing the new converted images to generate a synthetic image represented in the second coordinate system from that of the input images.

13. The image processing method according to claim 10, wherein

the input images are represented in a plane coordinate system, and

the converted images are represented in a spherical coordinate system.

14. The image processing method according to claim 10, wherein the rotational coordinate conversion corresponds to a 90 degree rotational coordinate conversion.

15. A non-transitory computer-readable storage medium storing a program for causing a computer to execute an image processing method, the image processing method comprising:

receiving input images having a first coordinate system;

converting the input images into converted images in a second coordinate system different from the first coordinate system according to first conversion data based on a projection model, the first conversion data including arrays for a conversion of coordinate values from the first coordinate system to the second coordinate system;

for each respective pixel of an overlapping area of a first image and a second image of the converted images;

setting a pixel block of predetermined dimensions around the respective pixel in the overlapping area of the first image as a pattern image; and

performing pattern matching, according to a correlation coefficient, on the second image at the respective pixel while moving the pattern image vertically and horizontally to find a pixel connecting position in the second image that correspond to the respective pixel in the first image;

detecting, according to the pattern matching, connecting points of the overlapping area between the first image and the second image;

correcting the arrays of the first conversion data according to the connecting points to produce corrected first conversion data; and

generating second conversion data for image synthesis, the image synthesis to be performed on the input images, the second conversion data being generated by performing a rotational coordinate conversion on the corrected first conversion data,

wherein the first conversion data and the second conversion data include respective conversion tables including information such that an angle of one pixel is at least 1/10 of a degree in both a θ

direction with θ

being from 1 to 360 degrees, and a ϕ

direction with ϕ

being from 0 to 180 degrees, and

wherein the conversion tables include information indicating at least 3,600*1,800 relations for the first image, and at least 3,600*1,800 relations for the second image.

16. The non-transitory computer-readable storage medium according to claim 15, wherein

the input images are represented in a plane coordinate system, and

the converted images are represented in a spherical coordinate system.

17. The non-transitory computer-readable storage medium according to claim 15, wherein the rotational coordinate conversion corresponds to a 90 degree rotational coordinate conversion.

18. An imaging system comprising:

an imaging device; and

circuitry configured to;

convert input images, captured in different directions by the imaging device and having a first coordinate system, into images in a second coordinate system different from the first coordinate system according to first conversion data based on a projection model, the first conversion data including arrays for a conversion of coordinate values from the first coordinate system to the second coordinate system;

for each respective pixel of an overlapping area of a first image and a second image of the converted images;

set a pixel block of predetermined dimensions around the respective pixel in the overlapping area of the first image as a pattern image; and

perform pattern matching, according to a correlation coefficient, on the second image at the respective pixel while moving the pattern image vertically and horizontally to find a pixel connecting position in the second image that correspond to the respective pixel in the first image;

detect, according to the pattern matching, connecting points of the overlapping area between the first image and the second image;

correct the arrays of the first conversion data according to the connecting points to produce corrected first conversion data; and

generate second conversion data for image synthesis, the image synthesis to be performed on the input images, the second conversion data being generated by performing a rotational coordinate conversion on the corrected first conversion data,

wherein the first conversion data and the second conversion data include respective conversion tables including information such that an angle of one pixel is at least 1/10 of a degree in both a θ

direction with θ

being from 1 to 360 degrees, and a ϕ

direction with ϕ

being from 0 to 180 degrees, and

wherein the conversion tables include information indicating at least 3,600*1,800 relations for the first image, and at least 3,600*1,800 relations for the second image.

19. The imaging system according to claim 18, wherein

the input images are represented in a plane coordinate system, and

the converted images are represented in a spherical coordinate system.