TAIL THE MOTION METHOD OF GENERATING SIMULATED STROBE MOTION VIDEOS AND PICTURES USING IMAGE CLONING

• US 20120002112A1
• Filed: 07/02/2010
• Published: 01/05/2012
• Est. Priority Date: 07/02/2010
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Robertson, Mark, Liu, Ming-Chang, Huang, Kuang-Man
• Application Number:
  US12/829,716
• Publication Number:
  US 20120002112A1
• Time in Patent Office: Days
• US Class Current: 348/579
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188

Abstract

The apparatus generates simulated strobe effects in the form of video or still image output in response to receipt of a video stream, and without the need of additional strobe hardware. Videos of a moving target object are categorized into one of multiple categories, from which a strobe generation process is selected. In one mode, the two categories comprise target objects with either small motion or large motions in relation to the frame size. Interoperation between image registration and cloning are utilized to produce simulated strobe motion videos or pictures. Motion segmentation is applied to the foreground object in each image frame, and a foreground mask is updated as each checkpoint is reached along the object trajectory, such as in response to time differences between checkpoints. Potential applications include special features for camcorders, digital cameras, or computer software.

Claims

1. An apparatus for generating simulated strobe effects, comprising:

(a) a computer configured for receiving video having a plurality of frames;

(b) a memory coupled to said computer; and

(c) programming executable on said computer for,

receiving a video input of a target object in motion within a received video sequence,

determining whether the camera is capturing target object motion within the received video sequence in response to a static positioning or in response to a non-static positioning,

selecting a strobe effect generation process, from multiple strobe effect generation processes, in response to determining said static positioning or said non-static positioning, and

generating a simulated strobe effect output in which one or more foreground elements are extracted from prior video frames and combined into a current frame in response to registering and cloning of images within the video input.

2. The apparatus recited in claim 1, wherein said programming executable on said computer for generating a simulated strobe effect output is configured for:

applying motion segmentation to detect a foreground object in each image frame of the received video sequence;

selecting at least one checkpoint image based on time differences of each image frame within the received video sequence to attain a desired interval between checkpoint images; and

updating an overall foreground mask and pasting an overall foreground area on future images as each said checkpoint image is reached.

3. The apparatus recited in claim 2, further comprising programming executable on said computer for generating a background model for applying said motion segmentation if the relative motion of the target object is large in relation to the frame size.

4. The apparatus recited in claim 1, further comprising programming executable on said computer for selecting between motion tracking for large motions or image differencing for small motion when determining a region of interest (ROI) within the received video sequence.

5. The apparatus recited in claim 1, further comprising programming executable on said computer for determining image differences as a basis of segmenting the region of interest within the received video sequence.

6. The apparatus recited in claim 1:

wherein said multiple strobe effect generation process comprises a first process and a second process within programming executable on said computer;

wherein said first process is selected in response to detection of commencement of target object motion;

wherein if a large motion is detected in response to accumulated motion exceeding a threshold, then a switch is made within programming executable on said computer from said first process to said second process; and

wherein if no large motion is detected, then generation of simulated strobe effect output continues according to said first method for small motion.

7. The apparatus recited in claim 1, wherein said simulated strobe motion output contains multiple foreground images of a target object, representing different time periods along a trajectory captured in the received video sequence, over a single background image.

8. The apparatus recited in claim 1, wherein said apparatus is selected from the group of devices configured for processing received video sequences consisting of camcorders, digital cameras, video recorders, image processing applications, televisions, display systems, computer software, video/image editing software, and/or combinations thereof.

9. The apparatus recited in claim 1, wherein said simulated strobe effect output comprises a video.

10. The apparatus recited in claim 1, wherein said simulated strobe effect output comprises a still image.

11. The apparatus recited in claim 1, wherein said simulated strobe effect output is a still image, generated in response to programming executable on said computer configured for,

dividing an image area which overlaps between each pair of adjacent images in response to,

forcing a cutting line to pass through a middle point of centroids of an identified moving object in each pair of adjacent images using a cost function, and

increasing the cost function within the image area of the identified moving object to prevent cutting through the identified moving object.

12. An apparatus for generating simulated strobe effects, comprising:

(a) a computer configured for receiving a video input having a plurality of frames;

(b) a memory coupled to said computer; and

(c) programming executable on said computer for,

receiving the video input of a target object in motion within a received video sequence,

determining whether the received video sequence is capturing small or large target object motion,

generating or updating a background model in response to detection of large target object motion,

applying motion segmentation,

selecting checkpoint images, and

generating a simulated strobe effect output in which one or more foreground elements are extracted from prior video frames and combined into a current frame in response to registering and cloning of images within the video input.

13. The apparatus recited in claim 12, further comprising programming executable on said computer for determining image difference as a basis for segmenting a region of interest within the video sequence.

14. The apparatus recited in claim 12, wherein said simulated strobe motion output contains multiple foreground images of the target object, representing different time periods along a trajectory captured in the received video sequence, over a single background image.

15. The apparatus recited in claim 12, wherein said apparatus is selected from a group of devices configured for processing received video consisting of camcorders, digital cameras, video recorders, image processing applications, televisions, display systems, computer software, video/image editing software, and/or combinations thereof.

16. The apparatus recited in claim 12, wherein said simulated strobe effect output comprises a video.

17. The apparatus recited in claim 12, wherein said simulated strobe effect output comprises a still image.

18. The apparatus recited in claim 12, wherein said simulated strobe effect output is a still image, generated in response to programming executable on said computer for,

dividing an overlapping area between each pair of adjacent images in response to,

forcing a cutting line to pass through a middle point of centroids of the target object, as represented in the adjacent images, using a cost function, and

increasing said cost function within the overlapping area, between the pair of adjacent images, to prevent cutting through representations of the target object in either of the pair of adjacent images.

19. A method of generating simulated strobe effects, comprising:

(a) receiving video input of a target object in motion within a received video sequence;

(b) determining whether the received video sequence depicts capturing target object motion within the received video sequence in response to a static positioning or in response to a non-static positioning;

(c) selecting a strobe effect generation method, from multiple strobe effect generation methods, in response to determining said static positioning or said non-static positioning; and

(d) generating a simulated strobe effect output in which one or more foreground elements are extracted from prior video frames and combined into a current frame in response to registering and cloning of images within the video input.

20. The method recited in claim 19, wherein said simulated strobe effect output comprises a video or a still image.