Phantom image data bank (3D)

• US 9,235,320 B2
• Filed: 03/30/2012
• Issued: 01/12/2016
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: Landeskriminalamt Rheinland-Pfalz
• Original Assignee: Landeskriminalamt Rheinland-Pfalz
• Inventors: Kinn, Uwe
• Primary Examiner(s): Nabi, Reza
• Application Number:
  US14/008,370
• Publication Number:
  US 20140026102A1
• Time in Patent Office: 1,383 Days
• Field of Search: 715/852
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

The invention relates to the generation of phantom images on the basis of an image data bank (10), comprising a plurality of first image data elements, wherein each first image data element displays a facial section of a human face in a first view/perspective, and wherein the image data bank (10) comprises a corresponding second image data element for each first image data element such that the second image data element displays, at least in part, the same facial section of the same human face in a second view/perspective. The user thereby selects a first image data element (26a) of a first facial section of a human face and a first image data element (26b) of a second facial section of a human face. The selected first image data elements are graphically displayed in a first display region (22) of a graphic user interface. Furthermore, the second image data elements (28a, 28b) corresponding to the selected first image data elements are identified, and are graphically displayed in a second display region (24) of the graphic user interface.

Claims

1. A computer-implemented method of constructing facial composites, comprising:

providing an image database comprising a plurality of first image data elements, wherein each first image data element represents a part of a human face in a first perspective, and wherein for every first image data element the image database comprises a corresponding second image data element such that the second image data element represents at least partially the same part of the same human face in a second perspective;

electronically acquiring a user input for selection of a first image data element of a first part of the human face;

electronically acquiring a user input for selection of a first image data element of a second part the human face;

graphically representing the selected first image data elements in a first display area of a graphical user interface;

electronically determining the corresponding second image data elements corresponding to the selected first image data elements; and

graphically representing the determined second image data elements in a second display area of the graphical user interface,

wherein at least one of the first and second perspectives represents a partial profile view, and the other of the first and second perspectives represents a front view, and wherein the method comprises;

determining a first editing area of the partial profile view, wherein the first editing area represents a first side of the face in the partial profile view, and a corresponding first editing area of the front view, wherein the first editing area represents the first side of the face in the front view;

determining a first scaling factor;

acquiring a user input for graphically editing the representation in the first editing area of the partial profile view or the front view, which comprises shifting at least one pixel of at least one of the representation in the first editing area of the partial profile view or the front view and a primary editing tool represented in the first editing area of the partial profile view or the front view by a primary position change stipulated by the user input, which includes a primary horizontal and/or a primary vertical component of the position change; and

shifting at least one pixel of at least one of the representation in the first editing area of the front view or partial profile view and a secondary editing tool represented in the first editing area of the front view or partial profile view by a secondary position change determined depending on the user input and the first scaling factor, which includes at least one of a secondary horizontal and a secondary vertical component, such that the secondary horizontal component corresponds to the primary horizontal component scaled by the first scaling factor and the secondary vertical component corresponds to the primary vertical component, wherein the scaling factor is based on a ratio of the distances of marked pixels from a center line of the human face.

2. The method according to claim 1, further comprising:

acquiring a user input for at least one of positioning, orienting, and dimensioning the representation of the first image data element of the second part of the face in the first display area relative to the representation of the first image data element of the first part of the face in the first display area; and

adapting at least one of a position, an orientation, and a dimension of the representation of the corresponding second image data element of the second part of the face in the second display area relative to the representation of the second image data element of the first part of the face in the second display area depending on the acquired user input for the at least one of positioning, orienting, and dimensioning in the first display area.

3. The method according to claim 2, further comprising activating a two-dimensional (2D) editing mode in which during the acquisition of user inputs for the at least one of selecting, positioning, orienting, and dimensioning first image data elements, the first display area, but not the second display area, is represented graphically.

4. The method according to claim 2 further comprising activating a three-dimensional (3D) editing mode in which during the acquisition of user inputs for selecting, positioning, orienting, and dimensioning of the first image data elements, the first display area and the second display area are represented graphically at the same time.

5. The method according to claim 4, further comprising:

acquiring a user input for at least one of positioning, orienting, and dimensioning the representation of the second image data element of the second part of the face in the second display area relative to the representation of the second image data element of the first part of the face in the second display area; and

adapting at least one of a position, orientation, and dimension of the representation of the corresponding first image data element of the second part of the face in the first display area relative to the representation of the first image data element of the first part of the face in the first display area depending on the acquired user input for the at least one of positioning, orienting, and dimensioning in the second display area.

6. The method according to claim 1, wherein the image database comprises a plurality of image area data sets, each of which comprises a plurality of image data elements such that image data elements included in different image area data sets represent different parts of human faces, while different image data elements within one image area data set represent substantially the same part of the face of different human faces.

7. The method according to claim 1, wherein the database is provided such that for each first image data element the database comprises a corresponding third image data element, such that the third image data element represents at least partially the same part of the face of the same human face in a third perspective, wherein the method comprises determining the corresponding third image data elements corresponding to the selected first image data elements.

8. The method according to claim 7, wherein the image database is provided such that at least one of the first, second, and third perspective(s) comprises at least one of a front view, a profile view, and a semiprofile view of the respective human face.

9. The method according to claim 8, further comprising:

constructing a 3D model of a composite human face from the selected first image data elements and at least one of the determined corresponding second and third image data elements; and

graphically representing the 3D-modeled human face in a further perspective different from the first and second perspectives.

10. The method of claim 1, further comprising:

constructing at least one of a two-dimensional (2D) model or three dimensional (3D) model of a composite human face from the selected first image data elements and corresponding second image data elements.

11. A computer-implemented method of constructing facial composites, comprising:

providing an image database comprising a plurality of first image data elements, wherein each first image data element represents a part of a human face in a first perspective, and wherein for every first image data element the image database comprises a corresponding second image data element such that the second image data element represents at least partially the same part of the same human face in a second perspective;

electronically acquiring a user input for selection of a first image data element of a first part of a human face;

electronically acquiring a user input for selection of a first image data element of a second part of a human face;

graphically representing the selected first image data elements in a first display area of a graphical user interface;

electronically determining the corresponding second image data elements corresponding to the selected first image data elements; and

graphically representing the determined second image data elements in a second display area of the graphical user interface,

wherein at least one of the first and second perspectives represents a partial profile view, and wherein the method comprises;

determining a first editing area of the first or second display area, wherein the first editing area represents a side of the face partially averted in a viewing direction in the first or second perspective, and a second editing area of the first or second display area, wherein the second editing area represents a side of the face facing toward the viewing direction;

determining a scaling factor;

acquiring a user input for graphically editing the representation in the first editing area, which comprises shifting at least one pixel of at least one of the representation in the first or second editing area and a primary editing tool represented in the first or second editing area by a primary position change stipulated by the user input, which includes one of a primary horizontal and a primary vertical component of the position change; and

shifting at least one pixel of at least one of the representation in the second or first editing area and a secondary editing tool represented in the second or first editing area by a secondary position change determined depending on the user input and the scaling factor, which includes one of a secondary horizontal and a secondary vertical component such that the secondary horizontal component corresponds to the primary horizontal component scaled by the scaling factor and the secondary vertical component corresponds to the primary vertical component, wherein the scaling factor is based on a ratio of the distances of marked pixels from a center line of the human face.

12. The method according to claim 11, wherein the image database comprises a plurality of image area data sets, each of which comprises a plurality of image data elements such that image data elements included in different image area data sets represent different parts of human faces, while different image data elements within one image area data set represent substantially the same part of the face of different human faces.

13. The method according to claim 11, further comprising:

acquiring a user input for at least one of positioning, orienting, and dimensioning the representation of the first image data element of the second part of the face in the first display area relative to the representation of the first image data element of the first part of the face in the first display area; and

adapting at least one of a position, an orientation, and a dimension of the representation of the corresponding second image data element of the second part of the face in the second display area relative to the representation of the second image data element of the first part of the face in the second display area depending on the acquired user input for the at least one of positioning, orienting, and dimensioning in the first display area.

14. The method according to claim 13, further comprising activating a two-dimensional (2D) editing mode in which during the acquisition of user inputs for the at least one of selecting, positioning, orienting, and dimensioning first image data elements, the first display area, but not the second display area, is represented graphically.

15. The method according to claim 13 further comprising activating a three-dimensional (3D) editing mode in which during the acquisition of user inputs for selecting, positioning, orienting, and dimensioning of the first image data elements, the first display area and the second display area are represented graphically at the same time.

16. The method according to claim 15, further comprising:

acquiring a user input for at least one of positioning, orienting, and dimensioning the representation of the second image data element of the second part of the face in the second display area relative to the representation of the second image data element of the first part of the face in the second display area; and

adapting at least one of a position, orientation, and dimension of the representation of the corresponding first image data element of the second part of the face in the first display area relative to the representation of the first image data element of the first part of the face in the first display area depending on the acquired user input for the at least one of positioning, orienting, and dimensioning in the second display area.

17. The method according to claim 11, wherein the database is provided such that for each first image data element the database comprises a corresponding third image data element, such that the third image data element represents at least partially the same part of the face of the same human face in a third perspective, wherein the method comprises determining the corresponding third image data elements corresponding to the selected first image data elements.

18. The method according to claim 17, wherein the image database is provided such that at least one of the first, second, and third perspective(s) comprises at least one of a front view, a profile view, and a semi-profile view of the respective human face.

19. The method according to claim 18, further comprising:

constructing a 3D model of a composite human face from the selected first image data elements and at least one of the determined corresponding second and third image data elements; and

graphically representing the 3D-modeled human face in a further perspective different from the first and second perspectives.

20. The method of claim 11, further comprising:

constructing at least one of a two-dimensional (2D) model or three dimensional (3D) model of a composite human face from the selected first image data elements and corresponding second image data elements.

21. A facial composite construction computer system, comprising:

a processor;

an image database comprising a plurality of first image data elements, wherein each first image data element represents a part of a human face in a first perspective, and wherein for every first image data element in the image database comprises a corresponding second image data element, such that the second image data element represents at least partially the same part of the same human face in a second perspective;

an input interface configured to;

acquire a user input for selection of a first image data element of a first part of a human face; and

acquire a user input for selection of a first image data element of a second part of a human face;

a data processing device for determining the corresponding second image data elements corresponding to the selected first image data elements; and

a graphical output interface configured to;

graphically illustrate the selected first image data elements in a first display area; and

graphically illustrate the determined second image data elements in a second display area,

wherein at least one of the first and second perspectives represents a partial profile view, and the other of the first and second perspectives represents a front view, and wherein the system is further configured to;

determine a first editing area of the partial profile view, wherein the first editing area represents a first side of the face in the partial profile view, and a corresponding first editing area of the front view, wherein the first editing area represents the first side of the face in the front view;

determine a first scaling factor;

acquire a user input that graphically edits the representation in the first editing area of the partial profile view or the front view, which comprises shifting at least one pixel of at least one of the representation in the first editing area of the partial profile view or the front view and a primary editing tool represented in the first editing area of the partial profile view or the front view by a primary position change stipulated by the user input, which includes a primary horizontal and/or a primary vertical component of the position change;

shift at least one pixel of at least one of the representation in the first editing area of the front view or partial profile view and a secondary editing tool represented in the first editing area of the front view or partial profile view by a secondary position change determined depending on the user input and the first scaling factor, which includes at least one of a secondary horizontal and a secondary vertical component, such that the secondary horizontal component corresponds to the primary horizontal component scaled by the first scaling factor and the secondary vertical component corresponds to the primary vertical component, wherein the scaling factor is based on a ratio of the distances of marked pixels from a center line of the human face.

22. The system of claim 21, wherein the wherein the data processing device and graphical output interface are configured to cooperatively graphically construct at least one of a two-dimensional (2D) model or three dimensional (3D) model of a composite human face from the selected first image data elements and corresponding second image data elements.

23. A computer program product, which comprises machine-readable program code stored in a non-transitory storage media, which, when loaded on a computer, is suitable for executing a method of constructing facial composites, the method comprising:

providing an image database comprising a plurality of first image data elements, wherein each first image data element represents a part of a human face in a first perspective, and wherein for every first image data element the image database comprises a corresponding second image data element such that the second image data element represents at least partially the same part of the same human face in a second perspective;

acquiring a user input for selection of a first image data element of a first part of a human face;

acquiring a user input for selection of a first image data element of a second part of a human face;

graphically representing the selected first image data elements in a first display area of a graphical user interface;

determining the corresponding second image data elements corresponding to the selected first image data elements; and

graphically representing the determined second image data elements in a second display area of the graphical user interface, wherein at least one of the first and second perspectives represents a partial profile view, and the other of the first and second perspectives represents a front view, and wherein the method comprises;

determining a first editing area of the partial profile view, wherein the first editing area represents a first side of the face in the partial profile view, and a corresponding first editing area of the front view, wherein the first editing area represents the first side of the face in the front view;

determining a first scaling factor;

acquiring a user input for graphically editing the representation in the first editing area of the partial profile view or the front view, which comprises shifting at least one pixel of at least one of the representation in the first editing area of the partial profile view or the front view and a primary editing tool represented in the first editing area of the partial profile view or the front view by a primary position change stipulated by the user input, which includes a primary horizontal and/or a primary vertical component of the position change; and

shifting at least one pixel of at least one of the representation in the first editing area of the front view or partial profile view and a secondary editing tool represented in the first editing area of the front view or partial profile view by a secondary position change determined depending on the user input and the first scaling factor, which includes at least one of a secondary horizontal and a secondary vertical component, such that the secondary horizontal component corresponds to the primary horizontal component scaled by the first scaling factor and the secondary vertical component corresponds to the primary vertical component,

wherein the scaling factor is based on a ratio of the distances of marked pixels from a center line of the human face.

24. The computer program product of claim 23, wherein the method further comprises:

constructing at least one of a two-dimensional (2D) model or three dimensional (3D) model of a composite human face from the selected first image data elements and corresponding second image data elements.

25. A facial composite construction computer system, comprising:

a processor;

an image database comprising a plurality of first image data elements, wherein each first image data element represents a part of a human face in a first perspective, and wherein for every first image data element in the image database comprises a corresponding second image data element, such that the second image data element represents at least partially the same part of the same human face in a second perspective;

an input interface configured to;

acquire a user input for selection of a first image data element of a first part of a human face; and

acquire a user input for selection of a first image data element of a second part of a human face;

a data processing device for determining the corresponding second image data elements corresponding to the selected first image data elements; and

a graphical output interface configured to;

graphically illustrate the selected first image data elements in a first display area; and

graphically illustrate the determined second image data elements in a second display area,

wherein at least one of the first and second perspectives represents a partial profile view, and wherein the system is further configured to;

determine a first editing area of the first or second display area, wherein the first editing area represents a side of the face partially averted in a viewing direction in the first or second perspective, and a second editing area of the first or second display area, wherein the second editing area represents a side of the face facing toward the viewing direction;

determine a scaling factor;

acquire a user input for graphically editing the representation in the first editing area, which comprises shifting at least one pixel of at least one of the representation in the first or second editing area and a primary editing tool represented in the first or second editing area by a primary position change stipulated by the user input, which includes one of a primary horizontal and a primary vertical component of the position change;

shift at least one pixel of at least one of the representation in the second or first editing area and a secondary editing tool represented in the second or first editing area by a secondary position change determined depending on the user input and the scaling factor, which includes one of a secondary horizontal and a secondary vertical component such that the secondary horizontal component corresponds to the primary horizontal component scaled by the scaling factor and the secondary vertical component corresponds to the primary vertical component, wherein the scaling factor is based on a ratio of the distances of marked pixels from a center line of the human face.

26. The system of claim 25, wherein the wherein the data processing device and graphical output interface are configured to cooperatively graphically construct at least one of a two-dimensional (2D) model or three dimensional (3D) model of a composite human face from the selected first image data elements and corresponding second image data elements.

27. A computer program product, which comprises machine-readable program code stored in a non-transitory storage media, which, when loaded on a computer, is suitable for executing a method of constructing facial composites, the method comprising:

providing an image database comprising a plurality of first image data elements, wherein each first image data element represents a part of a human face in a first perspective, and wherein for every first image data element the image database comprises a corresponding second image data element such that the second image data element represents at least partially the same part of the same human face in a second perspective;

electronically acquiring a user input for selection of a first image data element of a first part of a human face;

electronically acquiring a user input for selection of a first image data element of a second part of a human face;

graphically representing the selected first image data elements in a first display area of a graphical user interface;

electronically determining the corresponding second image data elements corresponding to the selected first image data elements; and

graphically representing the determined second image data elements in a second display area of the graphical user interface,

wherein at least one of the first and second perspectives represents a partial profile view, and wherein the method comprises;

determining a first editing area of the first or second display area, wherein the first editing area represents a side of the face partially averted in a viewing direction in the first or second perspective, and a second editing area of the first or second display area, wherein the second editing area represents a side of the face facing toward the viewing direction;

determining a scaling factor;

acquiring a user input for graphically editing the representation in the first editing area, which comprises shifting at least one pixel of at least one of the representation in the first or second editing area and a primary editing tool represented in the first or second editing area by a primary position change stipulated by the user input, which includes one of a primary horizontal and a primary vertical component of the position change; and

shifting at least one pixel of at least one of the representation in the second or first editing area and a secondary editing tool represented in the second or first editing area by a secondary position change determined depending on the user input and the scaling factor, which includes one of a secondary horizontal and/or a secondary vertical component such that the secondary horizontal component corresponds to the primary horizontal component scaled by the scaling factor and the secondary vertical component corresponds to the primary vertical component, wherein the scaling factor is based on a ratio of the distances of marked pixels from a center line of the human face.

28. The computer program product of claim 27, wherein the method further comprises:

constructing at least one of a two-dimensional (2D) model or three dimensional (3D) model of a composite human face from the selected first image data elements and corresponding second image data elements.