Moving-image processing device, moving-image processing method, and information recording medium

• US 9,723,225 B2
• Filed: 10/01/2012
• Issued: 08/01/2017
• Est. Priority Date: 07/20/2012
• Status: Active Grant

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Rakuten, Inc. (Rakuten Group, Inc.)
• Inventors: Onai, Rikio, Okabe, Makoto, Kurihara, Tatsuya, Sanjo, Masahiro, Hirano, Hiromi
• Primary Examiner(s): Aynalem, Nathanael
• Application Number:
  US14/361,477
• Publication Number:
  US 20140340477A1
• Time in Patent Office: 1,765 Days
• Field of Search: 345473, 345474, 345633, 345958, 345419, 340436, 701301
• CPC Class Codes: G06T 13/00 Animation
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2210/21 Collision detection, intersection
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A moving-image processing device (101) synthesizes a first moving image and a second moving image. In the first moving image, a first object is drawn, and accompanied by the anterior-surface depth of the first object. In the second moving image, a second object is drawn, and accompanied by the anterior-surface depth of the second object. A rear-surface-depth acquirer (102) acquires the rear-surface depth of the first object and the rear-surface depth of the second object. A clash determiner (103) functioning as an interference determiner refers to the anterior-surface depth and rear-surface depth of the first object to obtain an occupancy space which the first object can occupy, and refers to the anterior-surface depth and the rear-surface depth of the second object to determine whether the occupancy space and the second object drawn in certain frames of the second moving image satisfy interference conditions. A corrector (106) functioning as a range setter sets, on the basis of a determination result from the interference determiner, the possible range in which the second object can be positioned without interfering with the occupancy space.

Claims

1. A moving-image processing device for synthesizing a first moving image and a second moving image, wherein a first object is depicted in the first moving image and is accompanied by anterior-surface depth information of the first object and a second object is depicted in the second moving image and is accompanied by anterior-surface depth information of the second object, the moving-image processing device comprising:

at least one non-transitory memory operable to store program code;

at least one processor operable to read said program code and operate as instructed by said program code, said program code including;

acquiring code configured to cause said at least one processor to acquire rear-surface depth information of the first object, and rear-surface depth information of the second object;

interference determining code configured to cause said at least one processor to find an occupancy zone that can be occupied by the first object depicted in the first moving image, referencing the anterior-surface depth information and the rear-surface depth information of the first object, and to determine whether or not the occupancy zone and the second object depicted in a given frame of the second moving image satisfy an interference condition, by referencing the anterior-surface depth information and the rear-surface depth information of the second object; and

range setting code configured to cause said at least one processor to set a possible range where the second object can be positioned without interfering with the occupancy zone, from the determination result.

2. The moving-image processing device according to claim 1, wherein the acquiring code causes said at least one processor to acquire first depth length associated in advance with an attribute of the first object, from a database, and acquires second depth length associated in advance with an attribute of the second object, from the database, and acquires rear-surface depth information of the first object from the anterior-surface depth information of the first object and the first depth length, and acquires rear-surface depth information of the second object from the anterior-surface depth information of the second object and the second depth length.

3. The moving-image processing device according to claim 1, wherein the range setting code causes said at least one processor to acquire a positioning condition for the second object associated in advance with an attribute of the second object, from a database, and sets the possible range so that the positioning condition is satisfied.

4. The moving-image processing device according to claim 1, wherein the range setting code causes said at least one processor to set the possible range so that the occupancy zone and the second object do not satisfy the interference condition, for all frames of the second moving image.

5. The moving-image processing device according to claim 1, wherein:

in one among the first moving image and the second moving image, a non-deforming transformation indicating translation within a three-dimensional space is applied; and

the possible range is expressed by a range of values that can be taken by a

transformation parameter indicating an amount of translation relating to the non-deforming transformation.

6. The moving-image processing device according to claim 5, wherein:

the non-deforming transformation is a translation with shift amounts in a horizontal direction and a vertical direction as the transformation parameter;

the one among the first moving image and the second moving image is the second moving image; and

a possible range of shift amounts in the horizontal direction and the vertical direction set by the range setting code, and a given frame of the first moving image, are displayed on a screen so as to prompt a user to correct the shift amounts in the horizontal direction and the vertical direction.

7. The moving-image processing device according to claim 5, wherein:

the non-deforming transformation is a translation with shift amounts in a horizontal direction, a vertical direction and a depth direction as the transformation parameter;

the one among the first moving image and the second moving image is the second moving image; and

a possible range of shift amounts in the horizontal direction and the vertical direction set with respect to a shift amount in a current depth direction by the range setting code, and a given frame of the first moving image, are displayed on a screen to prompt a user to correct the shift amounts in the horizontal direction, the vertical direction and the depth direction.

8. The moving-image processing device according to claim 5, wherein said program code further includes:

correcting code configured to cause said at least one processor to correct the one of the moving images by selecting any of the transformation parameters from the possible range and applying the non-deforming transformation; and

moving image generating code configured to cause said at least one processor to generate a third moving image synthesizing the first moving image and the second moving image.

9. The moving-image processing device according to claim 8, wherein said program code further includes:

initial value setting code configured to cause said at least one processor to set an initial value of the transformation parameters;

wherein the correcting code causes said at least one processor to select transformation parameters closest to the set initial value, from the possible range, if the set initial value is not contained in the possible range.

10. A moving-image processing method, executed by a moving-image processing device including at least one processor, for synthesizing a first moving image and a second moving image, wherein a first object is depicted in the first moving image and is accompanied by anterior-surface depth information of the first object and a second object is depicted in the second moving image and is accompanied by anterior-surface depth information of the second object, the moving-image processing method comprising:

acquiring, using said at least one processor, rear-surface depth information of the first object, and rear-surface depth information of the second object;

finding, using said at least one processor, an occupancy zone that can be occupied by the first object depicted in the first moving image, referencing, using said at least one processor, the anterior-surface depth information and the rear-surface depth information of the first object, and determining, using said at least one processor, whether or not the occupancy zone and the second object depicted in a given frame of the second moving image satisfy an interference condition, by referencing the anterior-surface depth information and the rear-surface depth information of the second object; and

setting, using said at least one processor, a possible range where the second object can be positioned without interfering with the occupancy zone, from the determination result.

11. A non-transitory computer-readable information recording medium on which is recorded a program for synthesizing a first moving image and a second moving image, wherein a first object is depicted in the first moving image and is accompanied by anterior-surface depth information of the first object and a second object is depicted in the second moving image and is accompanied by anterior-surface depth information of the second object, the program causing a computer to:

acquire rear-surface depth information of the first object, and rear-surface depth information of the second object;

find an occupancy zone that can be occupied by the first object depicted in the first moving image, referencing the anterior-surface depth information and the rear-surface depth information of the first object, and determining whether or not the occupancy zone and the second object depicted in a given frame of the second moving image satisfy an interference condition, by referencing the anterior-surface depth information and the rear-surface depth information of the second object; and

set a possible range where the second object can be positioned without interfering with the occupancy zone, from the determination result from the interference determiner.