Image processing apparatus, image processing method, and image processing program

• US 8,643,788 B2
• Filed: 04/25/2003
• Issued: 02/04/2014
• Est. Priority Date: 04/25/2002
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Tetsujiro, Kikuchi, Daisuke
• Primary Examiner(s): VO, TUNG T
• Application Number:
  US10/481,910
• Publication Number:
  US 20050041156A1
• Time in Patent Office: 3,938 Days
• Field of Search: 348/699, 348/700, 348/701, 348/43, 348/169, 382/232, 382/154, 345/428
• US Class Current: 348/700
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 5/147 Scene change detection
• CPC Class Codes: H04N 9/12 Picture reproducers H04N9/11 takes precedence
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026

Abstract

An image processing apparatus and method for generating a wide angle image from received images. Using an image signal of a received image, a scene change detection unit detects scene change of the received images. A movement detection unit detects a movement in each of the received images. A display position determination unit determines a display position of each of the received images based on the movement in each of the received images detected in a period of continuous scene. A wide field angle image generation unit superimposes the received images in said period of said continuous scene, a display position of which has been determined, to generate an image signal of the wide field angle image. A received image slide unit generates image signal wherein the received images are slid to the display positions. An image superimposition unit superimposes the received images whose display positions have been slid on the wide field angle image, to generate an image signal of a wide field angle display image in which the display position of each of the received images is slid in response to the movement in each of the received images.

Claims

1. An image processing apparatus comprising:

scene change detection means including correlation value calculation means for calculating a correlation coefficient between mutually adjacent frames in said received images; and

determination means for determining whether scene change has occurred based on said correlation coefficient for detecting scene change of one or more received images;

movement detection means for detecting a movement in each of said received images; and

display image generation means for determining a period of each continuous scene based on the scene change detected by said scene change detection means and superimposing said received images of the period of said continuous scene based on the movement in each of said images detected by said movement detection means in the period of said continuous scene, and generating a display image which has a wider field angle than said received images and in which a position of each of said received images is slid in response to said movement in each, of said images,

wherein the display image generation means generates the display image by sequentially superimposing each of the received images of the period in a reverse order of time and supplies signals of the display image to a plurality of projectors, each projecting a portion of the display image.

2. The image processing apparatus according to claim 1, wherein said correlation value calculation means comprises:

difference average value calculation means for calculating an average value of difference values between mutually adjacent frames as a difference average value;

pixel average value calculation means for calculating an average value of pixel values of one frame as a pixel average value; and

normalization means for normalizing said difference average value using said pixel average value to output said difference average value thus normalized as said correlation value.

3. The image processing apparatus according to claim 1, wherein said display image generation means comprises:

display position determination means for determining a display position of each of said received images based on the movement in each of the images detected by said movement detection means in said period of said continuous scene;

wide field angle image generation means for superimposing the received images in said period of said continuous scene, display positions of said images having been determined by said display position determination means, and for generating a wide field angle image having a wider field angle than said received images;

received image slide means for sliding said received images to the display position determined by said display position determination means; and

image superimposition means for superimposing the received images whose display position has been slid by said received image slide means on said wide field angle image, to generate said display image.

4. The image processing apparatus according to claim 3, wherein said wide field angle image generation means sequentially superimposes the received images in reverse order of time in said period of said continuous scene, display positions of said received images having been determined by said display position determination means, to generate said wide field angle image.

5. The image processing apparatus according to claim 3,

wherein said wide field angle image generation means slides a display position of said wide field angle image used on the movement in each of the images detected by said movement detection means in said period of said continuous scene, to prevent a portion of said received images in said display image from going out of an image display region in which said display image is displayed;

and

wherein said received image slide means corrects the display position determined by said display position determination means with said wide field angle image sliding in said wide field angle image generation means.

6. The image processing apparatus according to claim 3,

wherein said wide field angle image generation means slides said display position thereof based on the movement in each of the images detected by said movement detection means in said period of said continuous scene, in accordance with sizes of said wide field angle image and said image display region;

and

wherein said received image slide means corrects the display position determined by said display position determination means with said wide field angle image sliding in said wide field angle image generation means.

7. The image processing apparatus according to claim 1, further comprising image division means for dividing said display image,

wherein, when an image display region for displaying said display image is comprised of multiple display regions, said image division means divides said display image corresponding to said multiple display regions.

8. An image processing apparatus for changing a display position of an image in response to a movement in a received image, said apparatus comprising:

movement detection means for detecting said movement in the received image;

display position determination means for determining a display position of said received image based on information of multiple frames on the movement in the image detected by said movement detection means;

received image slide means for sliding said received image to said display position determined by said display position determination means; and

superimposing means for sequentially superimposing each of the slide images in a reverse order of time, the slide images being consecutive images and supplying signals of the display image to a plurality of projectors, each projecting a portion of the display image.

9. An image processing method executed by an image processing apparatus for processing image signals, comprising the steps of:

detecting scene change of received images and a movement in each of said received images by calculating a correlation coefficient between mutually adjacent frames in said received images and determining whether scene change has occurred based on said correlation coefficient;

determining a period of each continuous scene based on the scene change thus detected and superimposing said received images in said period of said continuous scene based on the movement in each of the images detected in said period of said continuous scene; and

generating a display image which has a wider field angle than said received images and in which a position of each of said received images is slid in response to said movement in each of said images,

wherein the generating step generates the display image by sequentially superimposing each of the received images of the period in a reverse order of time and supplies signals of the display image to a plurality of projectors, each projecting a portion of the display image.

10. The image processing method according to claim 9, further comprising the steps of:

calculating an average value of difference values between mutually adjacent frames, as a difference average value;

calculating an average value of pixel values of one frame as a pixel average value; and

normalizing said difference average value using said pixel average value, to use said normalized difference average value as said correlation value.

11. The image processing method according to claim 9, further comprising the steps of:

determining a display position of each of said received images based on the movement in each of the images detected in said period of said continuous scene;

superimposing the received images in said period of said continuous scene, display positions of said images having been determined, and generating a wide field angle image having a larger wide field angle than said received images;

sliding said received images to said display position; and

superimposing said received images whose display position has been slid on said wide field angle image to thereby generate said display image.

12. The image processing, method according to claim 11, wherein received images in said period of said continuous scene whose display positions have been determined are superimposed in reverse order of time to generate said wide field angle image.

13. The image processing method according to claim 11, further comprising the steps of:

sliding a display position of said wide field angle image based on the movement in each of the images detected in said period of said continuous scene, to prevent a portion of said received images in said display image from going out of an image display region in which said display image is displayed; and

correcting a display position with respect to said received image in said period of said continuous scene with said wide field angle image sliding.

14. The image processing method according to claim 9, further comprising the steps of:

when an image display region for displaying said display image is comprised of multiple display regions, dividing said display image corresponding to said multiple display regions.

15. An image processing method executed by an image processing apparatus for changing a display position of an image in response to a movement in a received image, said method comprising the steps of:

detecting said movement in said received image;

determining a display position of said received image based on information of multiple frames on said movement in the image thus detected;

sliding said received image to said determined display position; and

sequentially superimposing each of the slide images in a reverse order of time, the slide images being consecutive images and supplying signals of the display image to a plurality of projectors, each projecting a portion of the display image.

16. A non-transitory computer-readable storing medium encoded with an executable program that, when executed, causes a computer to execute:

a procedure for detecting scene change of received images by calculating a correlation coefficient between mutually adjacent frames in said received images and determining whether scene change has occurred based on said correlation coefficient;

a procedure for detecting a movement in each of said received images;

a procedure for determining a period of each continuous scene based on the scene change thus detected and superimposing said received images in said period of said continuous scene based on the movement in each of the images detected in said period of said continuous scene; and

a procedure of generating a display image which has a wider field angle than said received images and in which a position of each of said received images is slid in response to said movement in each of said images;

wherein the generating procedure generates the display image by sequentially superimposing each of the received images of the period in a reverse order of time and supplies signals of the display image to a plurality of projectors, each projecting a portion of the display image.