INFORMATION PROCESSING APPARATUS AND METHOD, IMAGE PROCESSING APPARATUS AND METHOD, AND PROGRAM

• US 20140375762A1
• Filed: 02/01/2013
• Published: 12/25/2014
• Est. Priority Date: 02/17/2012
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ohki, Mitsuharu
• Application Number:
  US14/377,221
• Publication Number:
  US 20140375762A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G06T 3/073 Transforming surfaces of revolution to planar images, e.g. cylindrical surfaces to planar images
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

The present technology relates to information processing apparatus and method, image processing apparatus and method, and a program which enable acquisition of a panoramic image with higher quality.

According to the image processing apparatus, homogeneous transformation matrixes H′_s,s+1 between adjacent captured images are acquired under a more generous condition, and homogeneous transformation matrixes H″_s,s+1 between the adjacent captured images are acquired under a more strict condition, for N captured images which are successively captured. Furthermore, the homogeneous transformation matrixes H′_s,s+1 and the homogeneous transformation matrixes H″_s,s+1 are accumulated to acquire a homogeneous transformation matrix H′_1,s between first and s-th captured images, and the homogeneous transformation matrixes H″_s,s+1 are accumulated to acquire a homogeneous transformation matrix H″_1,s between the first and the s-th captured images. The respective captured images are connected based on a homogeneous transformation matrix acquired by performing weighted addition of the homogeneous transformation matrix H′_1,s and the homogeneous transformation matrix H″_1,s, and a panoramic image is generated. The present technology can be applied to the image processing apparatus.

Claims

1. An information processing apparatus which generates a single data item by connecting a plurality of data arranged in an order, the apparatus comprising:

a first map calculation unit which calculates a map H1 representing a correlation between mutually adjacent data items under a first condition with a higher degree of freedom;

a second map calculation unit which calculates a map H2 representing the correlation between the mutually adjacent data items under a second condition with a lower degree of freedom as compared to the first condition; and

a data generation unit which acquires a map H3 and generates the single data item based on the map H3, the map H3 being configured such that the correlation between target data as the data item and adjacent data adjacent to the target data becomes a relationship closer to the correlation represented by the map H1 than to the correlation represented by the map H2 at a position in the target data close to the adjacent data and becomes a relationship closer to the relationship represented by the map H2 than to the correlation represented by the map H1 at a position in the target data far from the adjacent data, based on the map H1 and the map H2.

2. The information processing apparatus according to claim 1,

wherein the map H3 is a map configured such that the correlation at each position in the target data is a relationship acquired by prorating the correlation represented by the map H1 and the correlation represented by the map H2 in accordance with the position in the target data.

3. The information processing apparatus according to claim 2,

wherein the map H3 is a map configured such that the correlation between the target data and the adjacent data becomes the correlation represented by the map H1 at a first position in the target data in the vicinity of the adjacent data and becomes the correlation represented by the map H2 at a second position in the target data far from the adjacent data.

4. The information processing apparatus according to claim 1,

wherein the plurality of data items are a plurality of captured images arranged in an order, and

wherein the data generation unit generates a panoramic image as the single data by acquiring a homogeneous transformation matrixes representing positional relationships between the captured images as the map H3 and connecting the captured images based on the homogeneous transformation matrix.

5. The information processing apparatus according to claim 4,

wherein the first map calculation unit calculates a homogeneous transformation matrixes Q1 which represent positional relationships between the mutually adjacent captured images as the map H1,

wherein the second map calculation unit calculates homogeneous transformation matrixes Q2 which represent positional relationships between the mutually adjacent captured images as the map H2 under a condition that the map H2 is an orthogonal matrix,

wherein the information processing apparatus further comprises

a first homogeneous transformation matrix calculation unit which calculates a homogeneous transformation matrix Q1_1s representing a positional relationship between a reference first captured image and an s-th captured image by accumulating the homogeneous transformation matrixes Q2 acquired for the first captured image to an s−

1-th captured image in the captured images arranged in an order and multiplying the accumulated homogeneous transformation matrixes Q2 by the homogeneous transformation matrix Q1 of the s-th captured image, and

a second homogeneous transformation matrix calculation unit which calculates a homogeneous transformation matrix Q2_1s representing a positional relationship between the first and the s-th captured images by accumulating the homogeneous transformation matrixes Q2 acquired for the first to the s-th captured images, and

wherein the data generation unit calculates a homogeneous transformation matrix Q3_1s as the map H3 representing a positional relationship between the first and the s-th captured images based on the homogeneous transformation matrix Q1_1s, and the homogeneous transformation matrix Q2_1s.

6. The information processing apparatus according to claim 5,

wherein the data generation unit acquires the homogeneous transformation matrix Q3_1s at each position on the s-th captured image by performing weighted addition of the homogeneous transformation matrix Q1_1s and the homogeneous transformation matrix Q2_1s with a weight in accordance with the position on the s-th captured image.

7. The information processing apparatus according to claim 1,

wherein the plurality of data items are a plurality of captured images arranged in an order, and

wherein the data generation unit generates a panoramic image as the single data item by acquiring gain values of the respective color components between the captured images as the map H3 and connecting the captured images after gain adjustment based on the gain values.

8. The information processing apparatus according to claim 7,

wherein the first map calculation unit calculates gain values G1 of the respective color components between the mutually adjacent captured images as the map H1 under a condition that the gain values of the respective color components are independent,

wherein the second map calculation unit calculates gain values G2 of the respective color components between the mutually adjacent captured images as the map H2 under a condition that the gain values of the respective color components are the same,

wherein the information processing apparatus further comprises

a first accumulated gain value calculation unit which calculates a gain value G1_1s between a reference first captured image and an s-th captured image by accumulating the gain values G2 acquired for the first captured image to an s−

1-th captured image in the captured images arranged in an order and multiplying the accumulated gain values G2 by the gain value G1 of the s-th captured image, and

a second accumulated gain value calculation unit which calculates a gain value G2_1s between the first and the s-th captured images by accumulating the gain values G2 acquired for the first to the s-th captured images, and

wherein the data generation unit calculates a gain value G3_1s between the first and the s-th captured images as the map H3 based on the gain value G1_1s and the gain value G2_1s.

9. The information processing apparatus according to claim 8,

wherein the data generation unit acquires the gain value G3_1s at each position on the s-th captured image by performing weighted addition of the gain value G1_1s and the gain value G2_1s with a weight in accordance with the position on the s-th captured image.

10. An information processing method for generating a single data item by connecting a plurality of data items arranged in an order, the method comprising the steps of:

calculating a map H1 which represents a correlation between mutually adjacent data items under a first condition with a higher degree of freedom;

calculating a map H2 which represents the correlation between the mutually adjacent data items under a second condition with a lower degree of freedom as compared to the first condition; and

acquiring a map H3 and generating the single data item based on the map H3, the map H3 being configured such that the correlation between target data as the data item and adjacent data adjacent to the target data becomes a relationship closer to the correlation represented by the map H1 than to the correlation represented by the map H2 at a position in the target data close to the adjacent data and becomes a relationship closer to the relationship represented by the map H2 than to the correlation represented by the map H1 at a position in the target data far from the adjacent data, based on the map H1 and the map H2.

11. A program for information processing, which is for generating a single data item by connecting a plurality of data items arranged in an order, and which causes a computer to execute processing comprising the steps of:

calculating a map H1 which represents a correlation between mutually adjacent data items under a first condition with a higher degree of freedom;

calculating a map H2 which represents the correlation between the mutually adjacent data items under a second condition with a lower degree of freedom as compared to the first condition; and

acquiring a map H3 and generating the single data item based on the map H3, the map H3 being configured such that the correlation between target data as the data item and adjacent data adjacent to the target data becomes a relationship closer to the correlation represented by the map H1 than to the correlation represented by the map H2 at a position in the target data close to the adjacent data and becomes a relationship closer to the relationship represented by the map H2 than to the correlation represented by the map H1 at a position in the target data far from the adjacent data, based on the map H1 and the map H2.

12. An image processing apparatus comprising:

a forward direction calculation unit which calculates a homogeneous transformation matrix Q1 representing a positional relationship between a reference first captured image and an s-th captured image by accumulating, in ascending order from the first captured image to an s-th captured image, homogeneous transformation matrixes H representing positional relationships between mutually adjacent captured images acquired for the N respective captured images that an imaging device captures while being turned;

a backward direction calculation unit which calculates a homogeneous transformation matrix Q2 representing a positional relationship between the first and the s-th captured images by accumulating inverse matrixes of the homogeneous transformation matrixes H in descending order from the N-th captured image to the s-th captured image; and

a homogeneous transformation matrix calculation unit which calculates a homogeneous transformation matrix Q3 representing a positional relationship between the first and the s-th captured images by prorating the homogeneous transformation matrix Q1 and the homogeneous transformation matrix Q2.

13. The image processing apparatus according to claim 12,

wherein the homogeneous transformation matrix calculation unit prorates the homogeneous transformation matrix Q1 and the homogeneous transformation matrix Q2 such that a proportion of the proration of the homogeneous transformation matrix Q1 becomes greater as a difference in imaging orders between the first and the s-th captured images is smaller.

14. The image processing apparatus according to claim 13,

wherein the homogeneous transformation matrix calculation unit prorates the homogeneous transformation matrix Q1 and the homogeneous transformation matrix Q2 such that a difference between a proportion of the proration of the homogeneous transformation matrix Q1 for the s−

1-th captured image and a proportion of the proration of the homogeneous transformation matrix Q1 for the s-th captured image becomes greater as an angle between a direction of the s−

1-th captured image and a direction of the s-th captured image is larger.

15. The image processing apparatus according to claim 12,

wherein the homogeneous transformation matrix calculation unit prorates the homogeneous transformation matrix Q1 and the homogeneous transformation matrix Q2 by performing weighted addition of a direction acquired by transforming a predetermined direction with reference to the s-th captured image by the homogeneous transformation matrix Q1 and a direction acquired by transforming the predetermined direction by the homogeneous transformation matrix Q2.

16. The image processing apparatus according to claim 15, further comprising:

a panoramic image generation unit which generates a panoramic image by connecting the captured images based on the homogeneous transformation matrix Q3.

17. An image processing method comprising the steps of:

calculating a homogeneous transformation matrix Q1 representing a positional relationship between a reference first captured image and an s-th captured image by accumulating, in ascending order from the first captured image to an s-th captured image, homogeneous transformation matrixes H representing positional relationships between mutually adjacent captured images acquired for the N respective captured images that an imaging device captures while being turned;

calculating a homogeneous transformation matrix Q2 representing a positional relationship between the first and the s-th captured images by accumulating inverse matrixes of the homogeneous transformation matrixes H in descending order from the N-th captured image to the s-th captured image; and

calculating a homogeneous transformation matrix Q3 representing a positional relationship between the first and the s-th captured images by prorating the homogeneous transformation matrix Q1 and the homogeneous transformation matrix Q2.

18. A program which causes a computer to execute processing comprising the steps of:

calculating a homogeneous transformation matrix Q1 representing a positional relationship between a reference first captured image and an s-th captured image by accumulating, in ascending order from the first captured image to the s-th captured image, homogeneous transformation matrixes H representing positional relationships between mutually adjacent captured images acquired for the N respective captured images that an imaging device captures while being turned;

calculating a homogeneous transformation matrix Q2 representing a positional relationship between the first and the s-th captured images by accumulating inverse matrixes of the homogeneous transformation matrixes H in descending order from the N-th captured image to the s-th captured image; and

calculating a homogeneous transformation matrix Q3 representing a positional relationship between the first and the s-th captured images by prorating the homogeneous transformation matrix Q1 and the homogeneous transformation matrix Q2.