Image processing to obtain high-quality loop moving image

• US 10,277,812 B2
• Filed: 05/31/2016
• Issued: 04/30/2019
• Est. Priority Date: 03/18/2011
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yamada, Hideshi, Ohki, Mitsuharu
• Primary Examiner(s): Dang, Hung Q
• Application Number:
  US15/168,445
• Publication Number:
  US 20160277677A1
• Time in Patent Office: 1,064 Days
• Field of Search: 386278-290
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

The present technology relates to an image processing device and method, and program, whereby a high-quality loop moving image can be obtained more easily. Upon continuous shot images of multiple frames serving as material of a loop moving image being input, the continuous shot images are divided into multiple segments including motion segments including a moving subject and a still segment including a subject with no motion. An image of the motion segment region of the continuous shot images is formed into a loop moving image, and a segment loop moving image is generated. Also, an image of a still segment region of the continuous shot images is clipped out, and a segment image which is a still image is generated. Further, these segment loop moving image and segment image are integrated, to form one loop moving image. The present invention can be applied to an image processing device.

Claims

1. An image processing device, comprising:

a loop moving image generator configured to;

clip out a subject with motion from a plurality of shot images shot at different points in time to generate a motion segment, wherein the motion segment includes the subject with motion;

detect a first image and a second image, of a plurality of continuous images of a plurality of the motion segments, having a degree of similarity that is highest,

wherein the degree of similarity is calculated based on a sum of absolute difference of pixels, of the first image and the second image, that corresponds to a sum of absolute values of difference in a luminance value for each of the pixels at same position in the first image and the second image; and

generate, from the plurality of shot images, a plurality of first images playable as a loop moving image, based on the detected first image and the second image, the motion segment, and at least a still segment,

wherein the motion segment and the still segment are obtained from the plurality of shot images in a spatial direction.

2. The image processing device according to claim 1,

wherein, of the plurality of continuous images, the loop moving image is repeatedly displayed in a first order between the first image and the second image of the plurality of first images.

3. The image processing device according to claim 2,

wherein the loop moving image generator is further configured to generate a segment map to identify a position of each of the plurality of the motion segments or a plurality of the still segments present on the plurality of shot images.

4. The image processing device according to claim 1,

wherein the loop moving image generator is further configured to generate a still segment image which is a still image based on a plurality of the still segments.

5. The image processing device according to claim 4,

wherein the loop moving image generator is further configured to integrate the plurality of first images playable as the loop moving image and the still segment image.

6. The image processing device according to claim 1,

wherein the loop moving image generator is further configured to generate the still segment, based on an average image of a plurality of images of the still segments in the plurality of shot images.

7. The image processing device according to claim 1,

wherein the loop moving image generator is further configured to position the plurality of shot images to rectify shake in the plurality of shot images due to motion of an image capturing device at a time of capture of the plurality of shot images.

8. The image processing device according to claim 1,

wherein the loop moving image generator is further configured to generate a still segment image from one image of the plurality of continuous images of the plurality of the motion segments, based on a lack of two images with the degree of similarity in the plurality of continuous images.

9. The image processing device according to claim 1, further comprising:

a selector configured to switch between a plurality of shooting modes;

wherein, based on a selection of a first mode from the plurality of shooting modes by the selector, the plurality of shot images and a high-resolution image are shot.

10. The image processing device according to claim 9,

wherein the plurality of shot images are shot before the high-resolution image is shot.

11. The image processing device according to claim 10,

wherein, based on the detection of the first image and the second image, the high-resolution image is shot and the loop moving image is generated.

12. An image processing method, comprising:

clipping out a subject with motion from a plurality of shot images shot at different points in time to generate a motion segment, wherein the motion segment includes the subject with motion;

detecting a first image and a second image, of a plurality of continuous images of a plurality of the motion segments, having a degree of similarity that is highest,

wherein the degree of similarity is calculated based on a sum of absolute difference of pixels, of the first image and the second image, that corresponds to a sum of absolute values of difference in a luminance value for each of the pixels at same position in the first image and the second image; and

generating, from the plurality of shot images, a plurality of first images playable as a loop moving image, based on the detected first image and the second image, the motion segment, and at least a still segment,

wherein the motion segment and the still segment are obtained from the plurality of shot images in a spatial direction.

13. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a processor of a computer, cause the processor to execute operations, the operations comprising:

clipping out a subject with motion from a plurality of shot images shot at different points in time to generate a motion segment, wherein the motion segment includes the subject with motion;

detecting a first image and a second image, of a plurality of continuous images of a plurality of the motion segments, having a degree of similarity that is highest,

wherein the degree of similarity is calculated based on a sum of absolute difference of pixels, of the first image and the second image, that corresponds to a sum of absolute values of difference in a luminance value for each of the pixels at same position in the first image and the second image; and

generating, from the plurality of shot images, a plurality of first images playable as a loop moving image, based on the detected first image and the second image, the motion segment, and at least a still segment,

wherein the motion segment and the still segment are obtained from the plurality of shot images in a spatial direction.