Do-It-Yourself Photo Realistic Talking Head Creation System and Method

• US 20100007665A1
• Filed: 04/10/2006
• Published: 01/14/2010
• Est. Priority Date: 08/14/2002
• Status: Active Grant

• Current Assignee: Avaworks Incorporated
• Original Assignee: Avaworks Incorporated
• Inventors: Cheiky, Michael, Smith, Shawn, Gately, Peter
• Granted Patent: US 8,553,037 B2
• Time in Patent Office: Days
• US Class Current: 345/473
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

A do-it-yourself photo realistic talking head creation system comprising: a template; handheld device comprising display and video camera having an image output signal of a subject; a computer having a mixer program for mixing the template and image output signal of the subject into a composite image, and an output signal representational of the composite image; a computer adapted to communicate the composite image signal to the display for display to the subject as a composite image; the display and the video camera adapted to allow the video camera to collect the image of the subject, the subject to view the composite image, and the subject to align the image of the subject with the template; storage means having an input for receiving the output signal of the video camera representational of the collected image of the subject and storing the image of the subject substantially aligned with the template.

Claims

1. An apparatus for constructing a photo realistic human talking head, comprising:

a handheld device;

a network;

a server;

said network and said handheld device communicating one with the other;

said network and said server communicating one with the other;

a library of alignment templates,

said server comprising said library of alignment templates,

each said alignment template being different one from the other and representational of an alignment template frame of a photo realistic human talking head having an alignment template head angular position,

comprising a template head tilt, a template head nod, and a template head swivel component,

each said alignment template frame different one form the other,

each said alignment template head angular position different one from the other;

a controller,

said server comprising said controller,

said controller selecting a said alignment template from said library of alignment templates corresponding to a specified alignment template head angular position and having an image output signal representational of said alignment template;

a video camera,

said handheld device comprising said video camera,

said video camera collecting an image of a human subject having a head having a human subject head angular position, comprising a human subject head tilt, a human subject head nod, and a human subject head swivel component,

said video camera having an output signal representational of said collected image of said human subject,

said handheld device communicating said output signal of said video camera representational of said collected image of said human subject to said server via said network;

said server,

said server having an input receiving said output signal of said video camera representational of said collected image of said human subject,

said server having a mixer,

said server receiving said selected alignment template image output signal from said controller, and communicating said selected alignment template image output signal and said received collected image signal of said human subject to said mixer,

said mixer receiving said selected alignment template image output signal and said communicated collected image signal of said human subject, and mixing one with the other into an output signal representational of a composite image of said collected image of said human subject and said selected alignment template, and

communicating said composite image signal of said collected image of said human subject and said selected alignment template to said server,

said server having an output signal representational of said composite image signal of said collected image of said human subject and said selected alignment template received from said mixer,

said server communicating said output signal representational of said composite image signal of said collected image of said human subject and said selected alignment template to said handheld device via said network;

a display,

said handheld device comprising said display,

said display having an input receiving said output signal representational of said composite image signal of said collected image of said human subject and said selected alignment template,

said display and said video camera adapted to allow said video camera to collect said image of said human subject and said human subject to view said composite image and said human subject to align said image of said head of said human subject with said alignment template head at substantially the same angular position as said specified alignment template head angular position;

storage means storing a library of collected images of said human subject with said head of said subject at different human subject head angular positions,

said server comprising said storage means,

said server communicating said received collected image signal of said human subject to said storage means,

said storage means receiving and storing said received collected image signal of said human subject as a stored image of said human subject,

when said human subject has said head of said human subject substantially aligned with said alignment template head,

said stored image of said human subject having said human subject head angular position substantially the same as said specified alignment template head angular position,

each said stored image in said library of collected images being different one from the other,

each said stored image human subject head angular position different one from the other;

each said stored image human subject head angular position of said library of collected images corresponding to and substantially the same as and aligned with a selected said alignment template head angular position of said library of alignment templates;

each said stored image representing a different frame of a photo realistic human talking head.

2. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said server comprises a computer comprising said library of alignment templates.

3. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said server comprises a computer comprising said controller.

4. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said server comprises a computer comprising at least one program, said at least one program comprising said mixer.

5. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said server comprises a computer comprising said storage means.

6. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said collected image of said human subject further comprises additional collected images of at least one facial characteristic of said head of said human subject corresponding to said specified alignment template head angular position, said library of collected images further comprises different stored images of said at least one facial characteristic at each said different human subject head angular position;

each said stored facial characteristic image representing an additional frame of said photo realistic human talking head.

7. The apparatus for constructing a photo realistic human talking head according to claim 6, wherein said at least one facial characteristic comprises different mouth forms.

8. The apparatus for constructing a photo realistic human talking head according to claim 6, wherein said at least one facial characteristic comprises different eye forms.

9. The apparatus for constructing a photo realistic human talking head according to claim 7, wherein said at least one facial characteristic comprises different eye forms.

10. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said server comprises a computer comprising at least one program comprising said controller, which comprises a control program.

11. The apparatus for constructing a photo realistic human talking head according to claim 10, wherein said alignment template has at least one characteristic, said control program is adapted to control said at least one characteristic of said selected alignment template.

12. The apparatus for constructing a photo realistic human talking head according to claim 10, wherein said collected image of said human subject has at least one characteristic, said control program is adapted to control said at least one characteristic of said collected image of said human subject.

13. The apparatus for constructing a photo realistic human talking head according to claim 11, wherein said collected image of said human subject has at least one characteristic, said control program is adapted to control said at least one characteristic of said collected image of said human subject.

14. The apparatus for constructing a photo realistic human talking head according to claim 2, wherein said library of alignment templates comprises at least two libraries of alignment templates.

15. The apparatus for constructing a photo realistic human talking head according to claim 14, wherein each said library of alignment templates is representational of a human talking head having different facial features.

16. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said mixer comprises a software mixer.

17. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said network comprises a distributed network.

18. The apparatus for constructing a photo realistic human talking head according to claim 17, wherein said distributed network comprises the internet.

19. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said network is from the group consisting of a cellular network, a wireless network, a wireless digital network, a distributed network, the internet, a global network, a wide area network, a metropolitan area network, a local area network, and combination thereof.

20. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said handheld device is from the group consisting of a cell phone, a personal digital assistant, a PDA, a web-enabled phone, a portable phone, a personal computer, a laptop computer, a tablet computer, a video phone, a television, a handheld television, a wireless digital camera, a wireless camcorder, a wireless camera having a display, an e-mail device, an instant messaging device, a pc phone, a video conferencing device, a mobile phone, a handheld device, a wireless device, and a wireless handheld device.

21. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said server comprises a computer that is adapted to translate said received collected image signal to computer readable format.

22. The apparatus for constructing a photo realistic human talking head according to claim 1, wherein said apparatus for constructing a photo realistic human talking head is a do-it-yourself apparatus for constructing a photo realistic human talking head.

23. An apparatus for constructing a photo realistic human talking head, comprising:

a wireless handheld device;

a wireless network;

a server;

said wireless network and said wireless handheld device wirelessly communicating one with the other;

said wireless network and said server communicating one with the other;

a library of alignment templates,

said server comprising said library of alignment templates,

each said alignment template being different one from the other and representational of an alignment template frame of a photo realistic human talking head having an alignment template head angular position,

comprising a template head tilt, a template head nod, and a template head swivel component,

each said alignment template frame different one form the other,

each said alignment template head angular position different one from the other;

a controller,

said server comprising said controller,

said controller selecting a said alignment template from said library of alignment templates corresponding to a specified alignment template head angular position and having an image output signal representational of said alignment template;

a video camera,

said wireless handheld device comprising said video camera,

said video camera collecting an image of a human subject having a head having a human subject head angular position, comprising a human subject head tilt, a human subject head nod, and a human subject head swivel component,

said video camera having an output signal representational of said collected image of said human subject,

said wireless handheld device communicating said output signal of said video camera representational of said collected image of said human subject to said server via said wireless network;

said server,

said server having an input receiving said output signal of said video camera representational of said collected image of said human subject,

said server having a mixer,

said server receiving said selected alignment template image output signal from said controller, and communicating said selected alignment template image output signal and said received collected image signal of said human subject to said mixer,

said mixer receiving said selected alignment template image output signal and said communicated collected image signal of said human subject, and mixing one with the other into an output signal representational of a composite image of said collected image of said human subject and said selected alignment template, and

communicating said composite image signal of said collected image of said human subject and said selected alignment template to said server,

said server having an output signal representational of said composite image signal of said collected image of said human subject and said selected alignment template received from said mixer,

said server communicating said output signal representational of said composite image signal of said collected image of said human subject and said selected alignment template to said wireless handheld device via said wireless network;

a display,

said wireless handheld device comprising said display,

said display having an input receiving said output signal representational of said composite image signal of said collected image of said human subject and said selected alignment template,

said display and said video camera adapted to allow said video camera to collect said image of said human subject and said human subject to view said composite image and said human subject to align said image of said head of said human subject with said alignment template head at substantially the same angular position as said specified alignment template head angular position;

storage means storing a library of collected images of said human subject with said head of said subject at different human subject head angular positions,

said server comprising said storage means,

said server communicating said received collected image signal of said human subject to said storage means,

said storage means receiving and storing said received collected image signal of said human subject as a stored image of said human subject,

when said human subject has said head of said human subject substantially aligned with said alignment template head,

said stored image of said human subject having said human subject head angular position substantially the same as said specified alignment template head angular position,

each said stored image in said library of collected images being different one from the other,

each said stored image human subject head angular position different one from the other;

each said stored image human subject head angular position of said library of collected images corresponding to and substantially the same as and aligned with a selected said alignment template head angular position of said library of alignment templates;

each said stored image representing a different frame of a photo realistic human talking head.

24. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said server comprises a computer comprising said library of alignment templates.

25. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said server comprises a computer comprising said controller.

26. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said server comprises a computer comprising at least one program, said at least one program comprising said mixer.

27. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said server comprises a computer comprising said storage means.

28. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said collected image of said human subject further comprises additional collected images of at least one facial characteristic of said head of said human subject corresponding to said specified alignment template head angular position, said library of collected images further comprises different stored images of said at least one facial characteristic at each said different human subject head angular position;

each said stored facial characteristic image representing an additional frame of said photo realistic human talking head.

29. The apparatus for constructing a photo realistic human talking head according to claim 28, wherein said at least one facial characteristic comprises different mouth forms.

30. The apparatus for constructing a photo realistic human talking head according to claim 28, wherein said at least one facial characteristic comprises different eye forms.

31. The apparatus for constructing a photo realistic human talking head according to claim 29, wherein said at least one facial characteristic comprises different eye forms.

32. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said server comprises a computer comprising at least one program comprising said controller, which comprises a control program.

33. The apparatus for constructing a photo realistic human talking head according to claim 32, wherein said alignment template has at least one characteristic, said control program is adapted to control said at least one characteristic of said selected alignment template.

34. The apparatus for constructing a photo realistic human talking head according to claim 32, wherein said collected image of said human subject has at least one characteristic, said control program is adapted to control said at least one characteristic of said collected image of said human subject.

35. The apparatus for constructing a photo realistic human talking head according to claim 33, wherein said collected image of said human subject has at least one characteristic, said control program is adapted to control said at least one characteristic of said collected image of said human subject.

36. The apparatus for constructing a photo realistic human talking head according to claim 24, wherein said library of alignment templates comprises at least two libraries of alignment templates.

37. The apparatus for constructing a photo realistic human talking head according to claim 36, wherein each said library of alignment templates is representational of a human talking head having different facial features.

38. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said mixer comprises a software mixer.

39. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said wireless network further comprises a distributed network.

40. The apparatus for constructing a photo realistic human talking head according to claim 39, wherein said distributed network comprises the internet.

41. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said wireless network is from the group consisting of a cellular network, a wireless network, a wireless digital network, a distributed network, the internet, a global network, a wide area network, a metropolitan area network, a local area network, and combination thereof.

42. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said wireless handheld device is from the group consisting of a cell phone, a personal digital assistant, a PDA, a web-enabled phone, a portable phone, a personal computer, a laptop computer, a tablet computer, a video phone, a television, a handheld television, a wireless digital camera, a wireless camcorder, a wireless camera having a display, an e-mail device, an instant messaging device, a pc phone, a video conferencing device, a mobile phone, a handheld device, a wireless device, and a wireless handheld device.

43. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said server comprises a computer that is adapted to translate said received collected image signal to computer readable format.

44. The apparatus for constructing a photo realistic human talking head according to claim 23, wherein said apparatus for constructing a photo realistic human talking head is a do-it-yourself apparatus for constructing a photo realistic human talking head.

45. A device for constructing a photo realistic human talking head, comprising:

a library of alignment templates,

each said alignment template being different one from the other and representational of an alignment template frame of a photo realistic human talking head having an alignment template head angular position,

comprising a template head tilt, a template head nod, and a template head swivel component,

each said alignment template frame different one form the other,

each said alignment template head angular position different one from the other;

a controller,

said controller selecting a said alignment template from said library of alignment templates corresponding to a specified alignment template head angular position and having an image output signal representational of said alignment template;

a video camera,

said video camera collecting an image of a human subject having a head having a human subject head angular position, comprising a human subject head tilt, a human subject head nod, and a human subject head swivel component,

said video camera having an output signal representational of said collected image of said human subject;

a computer,

said computer having an input receiving said output signal of said video camera representational of said collected image of said human subject,

said computer having a mixer,

said computer receiving said selected alignment template image output signal from said controller, and communicating said selected alignment template image output signal and said received collected image signal of said human subject to said mixer,

said mixer receiving said selected alignment template image output signal and said communicated collected image signal of said human subject, and mixing one with the other into an output signal representational of a composite image of said collected image of said human subject and said selected alignment template, and

communicating said composite image signal of said collected image of said human subject and said selected alignment template to said computer,

said computer having an output signal representational of said composite image signal of said collected image of said human subject and said selected alignment template received from said mixer;

a display,

said display having an input receiving said computer output signal representational of said composite image signal of said collected image of said human subject and said selected alignment template,

said display and said video camera adapted to allow said video camera to collect said image of said human subject and said human subject to view said composite image and said human subject to align said image of said head of said human subject with said alignment template head at substantially the same angular position as said specified alignment template head angular position;

storage means storing a library of collected images of said human subject with said head of said subject at different human subject head angular positions,

said computer communicating said received collected image signal of said human subject to said storage means,

said storage means receiving and storing said received collected image signal of said human subject as a stored image of said human subject,

when said human subject has said head of said human subject substantially aligned with said alignment template head,

said stored image of said human subject having said human subject head angular position substantially the same as said specified alignment template head angular position,

each said stored image in said library of collected images being different one from the other,

each said stored image human subject head angular position different one from the other;

each said stored image human subject head angular position of said library of collected images corresponding to and substantially the same as and aligned with a selected said alignment template head angular position of said library of alignment templates;

each said stored image representing a different frame of a photo realistic human talking head.

46. The device for constructing a photo realistic human talking head according to claim 45, wherein said computer comprises said library of alignment templates.

47. The device for constructing a photo realistic human talking head according to claim 45, wherein said computer comprises said controller.

48. The device for constructing a photo realistic human talking head according to claim 45, wherein said computer comprises at least one program, said at least one program comprising said mixer.

49. The device for constructing a photo realistic human talking head according to claim 45, wherein said computer comprises said storage means.

50. The device for constructing a photo realistic human talking head according to claim 45, wherein said collected image of said human subject further comprises additional collected images of at least one facial characteristic of said head of said human subject corresponding to said specified alignment template head angular position, said library of collected images further comprises different stored images of said at least one facial characteristic at

each said different human subject head angular position;

each said stored facial characteristic image representing an additional frame of said photo realistic human talking head.

51. The device for constructing a photo realistic human talking head according to claim 50, wherein said at least one facial characteristic comprises different mouth forms.

52. The device for constructing a photo realistic human talking head according to claim 50, wherein said at least one facial characteristic comprises different eye forms.

53. The device for constructing a photo realistic human talking head according to claim 51, wherein said at least one facial characteristic comprises different eye forms.

54. The device for constructing a photo realistic human talking head according to claim 45, wherein said computer comprises at least one program comprising said controller, which comprises a control program.

55. The device for constructing a photo realistic human talking head according to claim 54, wherein said alignment template has at least one characteristic, said control program is adapted to control said at least one characteristic of said selected alignment template.

56. The device for constructing a photo realistic human talking head according to claim 54, wherein said collected image of said human subject has at least one characteristic, said control program is adapted to control said at least one characteristic of said collected image of said human subject.

57. The device for constructing a photo realistic human talking head according to claim 55, wherein said collected image of said human subject has at least one characteristic, said control program is adapted to control said at least one characteristic of said collected image of said human subject.

58. The device for constructing a photo realistic human talking head according to claim 46, wherein said library of alignment templates comprises at least two libraries of alignment templates.

59. The device for constructing a photo realistic human talking head according to claim 58, wherein each said library of alignment templates is representational of a human talking head having different facial features.

60. The device for constructing a photo realistic human talking head according to claim 45, wherein said mixer comprises a software mixer.

61. The device for constructing a photo realistic human talking head according to claim 45, wherein said display is from the group consisting of a monitor, a display, and a television.

62. The device for constructing a photo realistic human talking head according to claim 45, wherein said video camera is mounted to said display, said display comprising a monitor.

63. The device for constructing a photo realistic human talking head according to claim 45, wherein said video camera is mounted atop said display, said display comprising a monitor.

64. The device for constructing a photo realistic human talking head according to claim 45, wherein said device for constructing a photo realistic human talking head has an adjustable cantilevered support arm adapted to support said video camera.

65. The device for constructing a photo realistic human talking head according to claim 64, wherein said adjustable cantilevered support arm is mounted to said display, said display comprising a monitor.

66. The apparatus for constructing a photo realistic human talking head according to claim 45, wherein said server comprises a computer that is adapted to translate said received collected image signal to computer readable format.

67. The apparatus for constructing a photo realistic human talking head according to claim 45, wherein said apparatus for constructing a photo realistic human talking head is a do-it-yourself apparatus for constructing a photo realistic human talking head.

68. A method of constructing a photo realistic human talking head, comprising at least the following steps:

selecting an alignment template from a library of alignment templates,

each said alignment template being different one from the other and representational of an alignment template frame of a photo realistic human talking head having an alignment template head angular position,

comprising a template head tilt, a template head nod, and a template head swivel component,

each said alignment template frame different one form the other,

each said alignment template head angular position different one from the other;

collecting an image of a human subject with a video camera;

communicating said collected image of said human subject to a computer, comprising a mixer;

mixing said collected image of said human subject with an image of said selected alignment template in said mixer, thus, creating a composite image of said human subject and said selected alignment template;

communicating said composite image to a display for display to said human subject, said display adapted to facilitate said human subject aligning an image of a head of said human subject with said image of said selected alignment template;

substantially aligning said head of said human subject, having a human subject head angular position, comprising a human subject head tilt, a human subject head nod, and a human subject head swivel component, with said image of said selected alignment template head at substantially the same angular position as said selected alignment template head angular position;

collecting and storing an image of said substantially aligned human subject in a library of collected images,

each said collected image having a different human subject angular position, which is substantially the same as a said selected alignment template head angular position,

each said stored image representing a different frame of a photo realistic human talking head.

69. The method of constructing a photo realistic human talking head according to claim 68, wherein said computer comprises at least one program, said at least one program comprising said mixer having a mixer program, said mixer program adapted to create said composite image from said collected image of said human subject and said image of said selected alignment template.

70. The method of constructing a photo realistic human talking head according to claim 68, further comprising:

capturing different facial characteristics of said substantially aligned human subject head at substantially the same angular position as said selected alignment template head angular position

storing images of said different facial characteristics in said library of collected images,

each said stored facial characteristic image representing an additional frame of said photo realistic human talking head.

71. The method of constructing a photo realistic human talking head according to claim 68, further comprising:

capturing mouth forms.

72. The method of constructing a photo realistic human talking head according to claim 68, further comprising:

capturing eye forms.

73. The method of constructing a photo realistic human talking head according to claim 68, further comprising:

capturing mouth forms and eye forms.

74. The method of constructing a photo realistic human talking head according to claim 73, further comprising:

capturing additional facial characteristics.

75. The method of constructing a photo realistic human talking head according to claim 68, wherein said method of constructing a photo realistic human talking head is a do-it-yourself method of constructing a photo realistic human talking head.