Image processing device and image processing method

• US 10,216,987 B2
• Filed: 02/19/2016
• Issued: 02/26/2019
• Est. Priority Date: 12/24/2008
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Ikenoue, Shoichi
• Primary Examiner(s): Pringle-Parker, Jason
• Application Number:
  US15/048,063
• Publication Number:
  US 20160171296A1
• Time in Patent Office: 1,103 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 10/752 Contour matching
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

First, the data of a moving image that is captured is read for each frame, and whether to start tracking is determined based on the presence or absence of a target object (S20, S22). An edge image of the image frame is created after it is determined to start tracking (S24). Meanwhile, a particle is distributed in a space of a coefficient set for each control point sequence when the control point sequence of a B-spline curve representing the shape of the target object is represented in a linear combination of control point sequence of a B-spline curve representing a plurality of reference shapes that are made available in advance (S26). A particle is also distributed in the space of a shape-space vector (S28), the likelihood of each particle is observed, and the probability density distribution is acquired (S30). A curve obtained by weighting parameters by the probability density distribution and then averaging the weighted parameters is created as a tracking result (S32).

Claims

1. An image processing device comprising:

a contour line acquisition unit configured to acquire a contour line of an object from an image frame that constitutes a moving image;

a target part identification unit configured to identify the position of a predetermined part of the object based on the contour line of the object acquired by the contour line acquisition unit;

a processing unit configured to create a processed image in which predetermined processing is performed on an area, which is determined based on the position of the predetermined part identified by the target part identification unit, in the image frame;

an output unit configured to output data of the moving image having, as the image frame, the processed image generated by the processing unit; and

a storage unit configured to store data of a three-dimensional graphics image of a fingernail with nail art, wherein

the contour line acquisition unit acquires the contour line of a human hand, which is the object,

the target part identification unit identifies an area of the fingernail based on the position of a fingertip in the contour line, and

the processing unit combines the image of the fingernail with nail art such that the image of the fingernail with nail art selected by a user is overlapped with the area of the fingernail, and

the contour line acquisition unit acquires, by adjusting through, matching score computation, a set of coefficients occurring when a parameter of a B-spline curve that defines the contour line of the hand is represented in a linear combination of a parameter that defines the contour line of a plurality of reference shapes defined in advance.

2. The image processing device according to claim 1, wherein

the target part identification unit acquires the time change of the inclination of the predetermined part from the time change of the contour line, and

the processing unit creates the processed image so that a result of the processing is time-variable in accordance with the time change of the inclination of the predetermined part.

3. The image processing device according to claim 1, wherein the contour line acquisition unit represents the parameter of the B-spline curve in a linear combination of a parameter of respective B-spline curves of five reference shapes, in each of which any one of the five fingers is extended.

4. The image processing device according to claim 1, wherein the processing unit combines, in an area entirely outside the contour of the object determined based on the position of the predetermined part identified by the target part identification unit, an image displaying information regarding the object.

5. The image processing device according to claim 1, wherein the processing unit does not combine an information tag having a size below a lower limit or above an upper limit, which are set for the size of the information tag in advance.

6. The image processing device according to claim 1, wherein

the target part identification unit detects, based on the time change of the contour line of the object, that at least a part of a first part is hidden behind a second part, and

the processing unit performs, when combining images corresponding to the first part and the second part, respectively, as the process, a hidden-surface removal process so that at least a part of the image corresponding to the first part is hidden by the second part.

7. An image processing method comprising:

reading an image frame constituting a moving image stored in a memory and acquiring a contour line of a human hand;

reading, from the memory, data of a three-dimensional graphics image of a fingernail with nail art;

adjusting, through matching score computation, a set of coefficients occurring when a parameter of a B-spline curve that defines the contour line of the human hand is represented in a linear combination of a parameter that defines the contour line of a plurality of reference shapes defined in advance,

wherein the plurality of reference shapes are possible known different configurations of the human hand;

identifying an area of a fingernail of the human hand in the image frame based on the contour line; and

creating a processed image in which the three-dimensional graphics image is overlapped with the area of the fingernail.

8. A non-transitory computer-readable recording medium having embodied thereon a computer program product comprising modules of:

reading an image frame constituting a moving image stored in a memory and acquiring a contour line of a human hand;

reading, from the memory, data of a three-dimensional graphics image of a fingernail with nail art;

adjusting, through matching score computation, a set of coefficients occurring when a parameter of a B-spline curve that defines the contour line of the human hand is represented in a linear combination of a parameter that defines the contour line of a plurality of reference shapes defined in advance,

wherein the plurality of reference shapes are possible known different configurations of the human hand;

identifying an area of a fingernail of the human hand in the image frame based on the contour line; and

creating a processed image in which the three-dimensional graphics image is overlapped with the area of the fingernail.