Method, apparatus, and non-transitory medium for generating a synthetic image from a series of captured images

• US 9,407,804 B2
• Filed: 10/21/2010
• Issued: 08/02/2016
• Est. Priority Date: 11/20/2009
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kawai, Tetsuro, Nakao, Daisuke
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): Segura, Cynthia
• Application Number:
  US12/925,422
• Publication Number:
  US 20110122275A1
• Time in Patent Office: 2,112 Days
• Field of Search: 348/222.1, 348/211.11
• US Class Current: 1/1
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

An image processing apparatus includes: a specific sound detecting section detecting a specific sound generated during motion in a specific motion of a target object included in an image group including a plurality of images which continues in a time-series manner; a valid sound range setting section setting, as a valid sound range, a range in the time axis for determination of whether the specific sound is valid on the basis of a user manipulation; and a selecting section selecting a predetermined number of motion transition images representing transitions of the specific motion from a selection range of the image group in the time axis, with reference to a detection position of the specific sound in the time axis, in a case where the specific sound is detected in the set valid sound range.

Claims

1. An image processing apparatus for generating a number of synthetic images representative of motion transitions, said apparatus comprising:

a memory device having stored thereon a program; and

a computer which upon execution of the program from the memory performs as

(i) a valid sound range setting section to set according to a user manipulation a first valid range indicative of a first time range with regard to received images arranged in a time-series manner, the first valid range having a first plurality of images associated therein in which a number of images in the first plurality of images is less than that in the received images;

(ii) a sound determination section to determine whether a specific sound is included in the first valid range; and

(iii) an image selection section to select a second plurality of images that continue in a time-series manner in a second valid range indicative of a second time range in a case the specific sound is determined to be included in the first valid range, the selected plurality of images are for use in generating the number of synthetic images,

the second valid range includes (i) a before portion which precedes a detection position of the specific sound detected in the set first valid range and (ii) an after portion which is after the detection position of the specific sound, said before portion being larger than said after portion such that a number of images in the before portion is larger than that in the second portion.

2. The apparatus according to claim 1,

wherein the user manipulation relates to an image capturing operation.

3. The apparatus according to claim 1, in which the computer upon execution of the program further performs as

a specific change detecting section to detect a specific change in time between the images.

4. The apparatus according to claim 3,

wherein the specific change detecting section detects the specific change using at least one of a characteristic amount extracted from each image and a characteristic amount extracted on the basis of a sound relating to each image.

5. The image processing apparatus of claim 1, in which the computer upon execution of the program further performs as

a synthetic image generating section to generate the number of synthetic images using the selected plurality of images along with layer images obtained therefrom.

6. The apparatus according to claim 5, wherein the synthetic image generating section is configured to arrange and synthesize the layer images in a time-series manner.

7. The image processing apparatus of claim 5, wherein the synthetic image generating section comprises:

a layer separating section configured to obtain a plurality of layer images by separating a target object from a background of the images.

8. The image processing apparatus of claim 1, further comprising a display configured to display a plurality of types of motion selectable by a user to designate a selected type.

9. The image processing apparatus of claim 1, in which the first time range is shorter than the second time range such that the first valid range set by the valid sound range setting section is shorter than the second valid range and the number of images in the first plurality of images is less than that of the second plurality of images.

10. An image processing apparatus for generating a number of synthetic images representative of motion transitions, said apparatus comprising:

a memory device having stored thereon a program; and

a computer which upon execution of the program from the memory performs as

(i) a valid sound range setting section to set a first valid range indicative of a first time range according to a user manipulation;

(ii) a sound determination section to determine whether a specific sound is included in the first valid range; and

(iii) an image selection section to select a plurality of images that continue in a time-series manner in a second valid range indicative of a second time range in a case the specific sound is determined to be included in the first valid range, the selected plurality of images are for use in generating the number of synthetic images,

in which the specific sound is contained in the second valid range and the second valid range is overlapped with the first valid range,

wherein the user manipulation is a specific manipulation received when motion of a target object is terminated or a timer setting manipulation for such a setting that a termination time of the motion of the target object becomes a termination time of a time counter, and

wherein the valid sound range setting section sets the first valid range with reference to a position in time where the specific manipulation is received in a case where the user manipulation is the specific manipulation, and sets the first valid range with reference to the termination time of the time counter by means of the timer setting manipulation in a case where the user manipulation is the timer setting manipulation.

11. The apparatus according to claim 10,

wherein the valid sound range setting section narrows the first valid range set in the case where the user manipulation is the specific manipulation compared with the first valid range set in the case where the user manipulation is the timer setting manipulation.

12. An image processing apparatus for generating a number of synthetic images representative of motion transitions, said apparatus comprising:

a memory device having stored thereon a program; and

a computer which upon execution of the program from the memory performs as

(i) a valid sound range setting section to set a first valid range indicative of a first time range according to a user manipulation;

(ii) a sound determination section to determine whether a specific sound is included in the first valid range; and

(iii) an image selection section to select a plurality of images that continue in a time-series manner in a second valid range indicative of a second time range in a case the specific sound is determined to be included in the first valid range, the selected plurality of images are for use in generating the number of synthetic images,

in which the specific sound is contained in the second valid range and the second valid range is overlapped with the first valid range,

said computer upon execution of the program further performs as

a storage section to store specific sounds and first valid ranges relating to a plurality of types of specific motions; and

a manipulation receiving section to receive a designating manipulation for designating a desired specific motion from among the plurality of types of specific motions in which the specific sounds and the first valid ranges are stored,

wherein the sound determination section determines the specific sound relating to the designated specific motion, and

wherein the valid sound range setting section sets the first valid range relating to the designated specific motion according to the user manipulation.

13. An image processing method for generating a number of synthetic images representative of motion transitions, said method comprising the steps of:

setting according to a user manipulation a first valid range indicative of a first time range with regard to received images arranged in a time-series manner, the first valid range having a first plurality of images associated therein in which a number of images in the first plurality of images is less than that in the received images;

determining whether a specific sound is included in the first valid range;

selecting a plurality of images that continue in a time-series manner in a second valid range indicative of a second time range in a case the specific sound is determined to be included in the first valid range; and

using the selected plurality of images in the generating of the number of synthetic images,

the second valid range includes (i) a before portion which precedes a detection position of the specific sound detected in the set first valid range and (ii) an after portion which is after the detection position of the specific sound, said before portion being larger than said after portion such that a number of images in the before portion is larger than that in the second portion.

14. A non-transitory medium comprising a program which allows a procedure to be executed on a computer, the procedure for generating a number of synthetic images representative of motion transitions, the procedure comprising the steps of:

setting according to a user manipulation a first valid range indicative of a first time range with regard to received images arranged in a time-series manner, the first valid range having a first plurality of images associated therein in which a number of images in the first plurality of images is less than that in the received images;

determining whether a specific sound is included in the first valid range;

selecting a plurality of images that continue in a time-series manner in a second valid range indicative of a second time range in a case the specific sound is determined to be included in the first valid range; and

using the selected plurality of images in the generating of the number of synthetic images,

the second valid range includes (i) a before portion which precedes a detection position of the specific sound detected in the set first valid range and (ii) an after portion which is after the detection position of the specific sound, said before portion being larger than said after portion such that a number of images in the before portion is larger than that in the second portion.

15. An image processing apparatus for generating a number of synthetic images representative of motion transitions, said apparatus comprising:

a memory device having stored thereon a program; and

a computer which upon execution of the program from the memory performs as

(i) a valid sound range setting section to set a first valid range indicative of a first time range according to a user manipulation;

(ii) a sound determination section to determine whether a specific sound is included in the first valid range; and

(iii) an image selection section to select a plurality of images that continue in a time-series manner in a second valid range indicative of a second time range in a case the specific sound is determined to be included in the first valid range, the selected plurality of images are for use in generating the number of synthetic images,

wherein the first valid range is set based on one of the user manipulation;

(i) a specific manipulation received in a case motion of a target object, generated by the plurality of images, is terminated, or (ii) a timer setting manipulation for such a setting that a termination time of the motion of the target object is met.