IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM

• US 20130287304A1
• Filed: 04/17/2013
• Published: 10/31/2013
• Est. Priority Date: 04/26/2012
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kimura, Atsushi
• Application Number:
  US13/864,451
• Publication Number:
  US 20130287304A1
• Time in Patent Office: Days
• US Class Current: 382/199
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: H04N 23/662 by using master/slave camera arrangements for affecting the control of camera image capture, e.g. placing the camera in a desirable condition to capture a desired image
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

There is provided an image processing device, including a seam reference determination processing unit that determines a seam reference to serve as a reference line for determining a seam between adjacent two pieces of frame image data of a series of n pieces (n is a natural number equal to or greater than 2) of frame image data to be used for panoramic image generation using object information of the adjacent two pieces of frame image data, and a seam determination processing unit that determines a seam between the adjacent two pieces of frame image data only within a determining region that is set based on the seam reference determined by the seam reference determination processing unit using the object information.

Claims

1. An image processing device, comprising:

a seam reference determination processing unit that determines a seam reference to serve as a reference line for determining a seam between adjacent two pieces of frame image data of a series of n pieces (n is a natural number equal to or greater than

     2) of frame image data to be used for panoramic image generation using object information of the adjacent two pieces of frame image data; and

a seam determination processing unit that determines a seam between the adjacent two pieces of frame image data only within a determining region that is set based on the seam reference determined by the seam reference determination processing unit using the object information.

2. The image processing device according to claim 1,

wherein the seam reference determined by the seam reference determination processing unit serves as a linear reference line that is perpendicular to a sweep direction at image capturing for a series of frame image data.

3. The image processing device according to claim 1,

wherein the seam reference determination processing unit determines the seam reference by optimizing a one-dimensional cost function that reflects the object information.

4. The image processing device according to claim 1,

wherein the seam determination processing unit determines the seam by optimizing a two-dimensional cost function that reflects the object information.

5. The image processing device according to claim 4,

wherein the seam is an indeterminate shaped line.

6. The image processing device according to claim 1,

wherein the determining region is a region that, of an overlapping region of adjacent two pieces of frame image data, falls within a predetermined distance from a line serving as the seam reference in a sweep direction at image capturing for a series of frame image data and in an opposite direction thereof, respectively.

7. The image processing device according to claim 6,

wherein the predetermined distance to set the determining region is set variably.

8. The image processing device according to claim 1, further comprising:

an image combining unit that combines each frame image data based on each seam determined by the seam determination processing unit to generate panoramic image data that uses the n pieces of frame image data.

9. The image processing device according to claim 8,

wherein the image combining unit carries out blend processing on a range of adjacent two pieces of frame image data that falls within a predetermined distance from a line serving as the seam in a sweep direction at image capturing for a series of frame image data and in an opposite direction thereof, respectively, to combine the two pieces of frame image data.

10. The image processing device according to claim 1,

wherein the object information detecting unit detects object information of frame image data in an input process of a series of n pieces of frame image data to be used in panoramic image generation, and

wherein the seam reference determination processing unit sequentially carries out, in the input process, processing to determine m or less seam references for each group of (m+1) pieces (m<

n) of frame image data.

11. The image processing device according to claim 10,

wherein the seam determination processing unit carries out processing to determine the each seam within a determining region that is set based on each determined seam reference each time processing to determine the m or less seam references is carried out by the seam reference determination processing unit.

12. The image processing device according to claim 1, further comprising:

an object information detecting unit that detects object information of a series of n pieces of frame image data to be used in panoramic image generation.

13. The image processing device according to claim 1,

wherein the seam reference determination processing unit determines the seam reference using image data of a first resolution, and

the seam determination processing unit determines the seam using image data that are higher in resolution than the image data of the first resolution.

14. The image processing device according to claim 12,

wherein the object information detecting unit includes a low resolution object information detecting unit that detects object information of low resolution image data and a high resolution object information detecting unit that detects object information of high resolution image data,

wherein the seam reference determination processing unit determine the seam reference, with respect to frame image data, using object information obtained by the low resolution object information detecting unit, and

the seam determination processing unit determines the seam, with respect to at least an image within the determining region of frame image data, using object information obtained by the high resolution object information detecting unit.

15. An image processing method, comprising:

seam reference determination processing that determines a seam reference to serve as a reference line for determining a seam between adjacent two pieces of frame image data of a series of n pieces (n is a natural number equal to or greater than

     2) of frame image data to be used for panoramic image generation using object information of the adjacent two pieces of frame image data; and

seam determination processing that determines a seam between the adjacent two pieces of frame image data only within a determining region that is set based on the seam reference determined in the seam reference determination processing using the object information.

16. A program to cause an arithmetic processing device to execute:

seam reference determination processing that determines a seam reference to serve as a reference line for determining a seam between adjacent two pieces of frame image data of a series of n pieces (n is a natural number equal to or greater than

     2) of frame image data to be used for panoramic image generation using object information of the adjacent two pieces of frame image data; and

seam determination processing that determines a seam between the adjacent two pieces of frame image data only within a determining region that is set based on the seam reference determined in the seam reference determination processing using the object information.