Method and apparatus for generating models of individuals

• US 20030099409A1
• Filed: 11/22/2002
• Published: 05/29/2003
• Est. Priority Date: 11/23/2001
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Rowe, Simon Michael
• Granted Patent: US 7,266,251 B2
• Time in Patent Office: Days
• US Class Current: 382/285
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes

Abstract

A communications system is described in which a plurality of mobile phones are provided, each including means for generating a user interface for enabling users to identify points in images as feature points. All of the phones are then operable to utilise points identified by users to generate 3D models of individuals in the images. The 3D models are generated by associating stored average depth data for points corresponding to the co-ordinates of the feature points and further points whose position is identified relative to the co-ordinates of the feature points. The 3D models are then used to generate images which are displayed on the mobile phones.

Claims

1. Apparatus for generating 3D models of individuals from 2D images, said apparatus comprising:

a data store operable to store a 3D model defining the relative 3D locations of a plurality of points on the surface of a generic individual;

a feature identifier operable to process 2D images of individuals to identify a number of points in said images as 2D co-ordinate locations of a number of features of said individuals in said images; and

a modelling unit operable to process said 2D images of individuals and said identified 2D co-ordinate locations of features to generate 3D models of said individuals, said modelling unit being operable to generate said 3D models by associating said identified 2D co-ordinate locations of features with data indicative of relative 3D locations for said features determined utilising the relative 3D locations of corresponding features defined by a 3D model stored in said data store.

2. Apparatus in accordance with claim 1, wherein said modelling unit is operable to process said 2D images of individuals and said identified 2D co-ordinate locations of features by identifying a plurality of points in said 2D images relative to the positions of groups of said identified 2D co-ordinate locations and to associate said plurality of points with data indicative of relative 3D locations utilising the relative 3D locations of points on the surface of a generic individual in defined positions relative to groups of said corresponding features defined by a 3D model stored in said data store.

3. Apparatus in accordance with claim 1, wherein said modelling unit is operable to generate 3D models of individuals wherein said feature identifier is operable to identify points in said images corresponding to points outlining any of the face, eyes, nose, eyebrows or mouth of an individual in said images.

4. Apparatus in accordance with claim 1, wherein said feature identifier comprises a user interface enabling a user to input co-ordinate data identifying 2D co-ordinate locations of said features of individuals in said images.

5. Apparatus in accordance with claim 1, wherein said data store stores a 3D model defining the relative 3D locations of an average position of a plurality of points on the surface of a group of individuals.

6. Apparatus in accordance with claim 1, wherein said data store is operable to store a plurality of 3D models defining the relative 3D locations of positions of a plurality of points on the surface of a plurality of individuals, and said modelling unit further comprises a selection unit for selecting from a plurality of 3D models stored in said data store, a 3D model to be utilised to generate a 3D model for an individual.

7. Apparatus in accordance with claim 1, wherein said modelling unit is operable to determine the orientation of an individual in an image and to process identified co-ordinate locations to determine corresponding 2D co-ordinates in a predefined plane, wherein said generation of 3D models comprises associating depth data with said 2D co-ordinates in said predefined plane.

8. Apparatus in accordance with claim 1, further comprising:

a receiver operable to receive 2D co-ordinates and images of individuals, wherein said modelling unit is operable to generate models utilising data received by said receiver.

9. Apparatus in accordance with claim 8, wherein said apparatus further comprises a display, said apparatus being arranged to display images generated utilising said models generated by said modelling unit upon receipt of data by said receiver.

10. Apparatus in accordance with claim 1, further comprising:

a transmitter unit operable to transmit 2D co-ordinates and images of individuals, said 2D co-ordinates being identified in said images by said feature identifier.

11. A communications system comprising:

a plurality of remote terminals each comprising apparatus in accordance with claim 10;

and

a communications network for transmitting data between said terminals wherein said remote terminals are operable to transmit said images and send 2D co-ordinates whenever communication between said terminal and another of said terminals is initiated.

12. A cellular communications device comprising apparatus in accordance with claim 10, wherein said device is operable to transmit said images and said 2D co-ordinates whenever a call is made or received.

25. A storage medium storing computer interpretable instructions for causing a programmable computer to be configured to be apparatus in accordance with claim 1.

13. A communications system comprising:

a plurality of remote terminals; and

a communications network for transmitting data between said remote terminals, said remote terminals each comprising;

an image input unit operable to obtain 2D images of individuals;

a screen operable to display images; and

an input device for entering data and initiating communication via said network;

wherein each of said remote terminals further comprises;

a feature identifier operable to enable a user utilising said input device to identify co-ordinate locations of features of individuals in images obtained by said image input unit, displayed on said screen;

a transmission unit operable to transmit images obtained by said input unit together with data identifying co-ordinate locations of features of individuals in said images entered utilising said feature identifier of said remote terminal; and

a modelling unit operable to generate 3D models of an individual utilising data derived from a 2D image of said individual and co-ordinates of features of said individual in said images received from another of said remote terminals via said cellular communications network.

14. A communications system in accordance with claim 13, wherein said feature identifier is operable to enable a user to utilise said input device to move a cursor relative to an image displayed on said screen thereby enabling identification of co-ordinate locations in said image.

15. A communications system in accordance with claim 13, wherein said feature identifier is operable to enable a user to identify co-ordinate locations of points outlining features of individuals in images obtained by said image input unit.

16. A communications system in accordance with claim 15, wherein said feature identifier is operable to enable a user to identify co-ordinate locations of points outlining any of the face, eyes, nose, eyebrows or mouth of an individual in said images.

17. A communications system in accordance with claim 15, wherein said feature identifier is operable to cause an outline to be displayed superimposed on an image on said screen, said outline comprising a polygon connecting points, at least some of said points corresponding to co-ordinate locations identified utilising said feature identifier.

18. A communications system in accordance with claim 13, wherein said feature identifier is operable to cause enlarged portions of an image obtained by said image input unit to be displayed on said screen, said feature identifier being operable to enable a user to identify co-ordinate locations of features of individuals in said enclosed portions of images displayed on said screen.

19. A communications system in accordance with claim 13, wherein said input device comprises a keyboard.

20. A communications system in accordance with claim 13, wherein said remote terminals further comprise a data store operable to store images obtained by said input unit and data identifying co-ordinates of locations of features of individuals in said images, and said transmission unit is operable to transmit images and data stored in said data store to another of said remote terminals when communication is initiated with or received from said another remote terminal.

21. A communications system in accordance with claim 13, further comprising:

a principle component analysis modelling unit, said unit comprising;

a receiver operable to receive images and co-ordinate locations from said remote terminals;

a processing unit for processing received images and co-ordinate locations to determine principle component analysis Eigen value data for representing 3D-models of individuals in said received images utilising said co-ordinate locations; and

a transmission unit for transmitting said Eigen value data to a remote terminal from which an image of an individual and co-ordinate locations have been received by said receiver;

said remote terminals each further comprising a data store for storing Eigen value data output by said principle component analysis modelling unit, and each said transmission unit being operable to transmit Eigen value data stored in said data store to another of said remote terminals when a call is initiated with or received from said another remote terminal, said modelling units being operable to generate 3D models of individuals utilising said Eigen value data.

22. A communications system in accordance with claim 13, wherein said network comprises a cellular communications network and said remote terminals comprise mobile phones.

23. A remote terminal for use with a communications system in accordance with claim 13.

24. A storage medium storing computer interpretable instructions for causing a programmable computer to be configured to be a remote terminal in accordance with claim 23.

26. A storage medium in accordance with claim 24 comprising a SIM card.

27. A storage medium in accordance with claim 24 comprising an electrical signal in a communications network.