Image processing device executing image processing on input image and processing method for the same

• US 7,239,741 B2
• Filed: 03/19/2003
• Issued: 07/03/2007
• Est. Priority Date: 01/07/2003
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Hiwada, Kazuhiro
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US10/390,778
• Publication Number:
  US 20040131247A1
• Time in Patent Office: 1,567 Days
• Field of Search: 382/154, 382/233, 382/285, 382/302, 382/243, 382/103, 382/190, 345/420, 345/419, 399/119, 399/262, 250/363.04, 430/120
• US Class Current: 382/154
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An image processing device is composed of an image drawing section, an image input section, a corresponding-point searching section, and a processing parameter improving section. The image drawing section has a preset three-dimensional model image to draw a three-dimensional image on the basis of the three-dimensional model image and a first processing parameter. An input image of a still image or a motion picture is input to the image input section. The corresponding-point searching section searches for that point on the input image input to the image input section which corresponds to a certain point on the three-dimensional image drawn by the image drawing section, to obtain corresponding-point position information. The processing parameter improving section uses the corresponding-point position information obtained by the corresponding-point searching section to improve a value for the first processing parameter to determine a second processing parameter.

Claims

1. An image processing device comprising:

an image drawing section which has a preset three-dimensional model image to draw a three-dimensional image on the basis of the three-dimensional model image and a first movement and model parameter;

an image input section to which an input image of a still image or a motion picture is input;

a corresponding-point searching section which searches for that point on the input image input to the image input section which corresponds to a certain point on the three-dimensional image drawn by the image drawing section, to obtain corresponding-point position information; and

a movement and model parameter improving section which determines a second movement and model parameter by improving a value of the first movement and model parameter, to minimize a sum of distance functions determined by each of the corresponding-point position information obtained by the corresponding-point searching section.

2. An image processing device according to claim 1, wherein the corresponding-point searching section obtains the corresponding-point position information by arranging a plurality of points on the three-dimensional image and searching for a point on the input image which corresponds to each of the plurality of points.

3. An image processing device according to claim 2, wherein when the image drawing section has drawn a three-dimensional image on the basis of the three-dimensional model image and the second movement and model parameter, the movement and model parameter improving section determines the second movement and model parameter so as to minimize the distance between two corresponding points for the plurality of points arranged on the three-dimensional image and the corresponding plurality of points on the input image.

4. An image processing device according to claim 1, wherein the second movement and model parameter determined by the movement and model parameter improving section is input to the image drawing section,

the image drawing section draws a three-dimensional image on the basis of the three-dimensional model image and the second movement and model parameter,

the corresponding-point searching section obtains corresponding-point position information on the three-dimensional image and the input image, and

the movement and model parameter improving section uses the corresponding-point position information to improve a value for the second movement and model parameter to determine a third movement and model parameter.

5. An image processing device according to claim 4, further comprising an image output section which externally outputs the three-dimensional image drawn by the image drawing section on the basis of the three-dimensional model image and the third movement and model parameter.

6. An image processing device according to claim 5, wherein if the input image input to the image input section is an image of a person, the image output section outputs an avatar image for movement, posture, or expression approximating to that in the image of the person.

7. An image processing device according to claim 1, wherein the image drawing section and the corresponding-point searching section are formed on a single semiconductor substrate.

8. An image processing device according to claim 7, wherein the movement and model parameter improving section is further formed on the semiconductor substrate.

9. An image processing device according to claim 1, further comprising a network connecting section which outputs the second movement and model parameter determined by the movement and model parameter improving section, to a network.

10. An image processing device according to claim 9, wherein the network connecting section receives the first movement and model parameter from the network and outputs the first movement and model parameter to the image drawing section.

11. An image processing device comprising:

a model image creating section which uses an input image of a still image or a motion picture to create a three-dimensional model image and a first movement and model parameter;

an image drawing section which draws a three-dimensional image on the basis of the three-dimensional model image and first movement and model parameter created by the model image creating section;

an image input section to which the input image of the still image or the motion picture is input;

a corresponding-point searching section which searches for that point on the input image input to the image input section which corresponds to a certain point on the three-dimensional image drawn by the image drawing section, to obtain corresponding-point position information; and

a movement and model parameter improving section which determines a second movement and model parameter by improving a value of the first movement and model parameter, to minimize a sum of distance functions determined by each of the corresponding-point position information obtained by the corresponding-point searching section.

12. An image processing device according to claim 11, wherein the corresponding-point searching section obtains the corresponding-point position information by arranging a plurality of points on the three-dimensional image and searching for a point on the input image which corresponds to each of the plurality of points.

13. An image processing device according to claim 12, wherein when the image drawing section has drawn the three-dimensional image on the basis of the three-dimensional model image and the second movement and model parameter, the movement and model parameter improving section determines the second movement and model parameter so as to minimize the distance between two corresponding points for the plurality of points arranged on the three-dimensional image and the corresponding plurality of points on the input image.

14. An image processing device according to claim 11, wherein the second movement and model parameter determined by the movement and model parameter improving section is input to the image drawing section,

the image drawing section draws a three-dimensional image on the basis of the three-dimensional model image and the second movement and model parameter,

the corresponding-point searching section obtains corresponding-point position information on the three-dimensional image and the input image, and

the movement and model parameter improving section uses the corresponding-point position information to improve a value for the second movement and model parameter to determine a third movement and model parameter.

15. An image processing device according to claim 14, further comprising an image output section which externally outputs the three-dimensional image drawn by the image drawing section on the basis of the three-dimensional model image and the third movement and model parameter.

16. An image processing device according to claim 15, wherein if the input image input to the image input section is an image of a person, the image output section outputs an avatar image for movement, posture, or expression approximating to that in the image of the person.

17. An image processing device according to claim 11, wherein the image drawing section and the corresponding-point searching section are formed on a single semiconductor substrate.

18. An image processing device according to claim 17, wherein the movement and model parameter improving section is further formed on the semiconductor substrate.

19. An image processing device according to claim 11, further comprising a network connecting section which outputs the second movement and model parameter determined by the movement and model parameter improving section, to a network.

20. An image processing device according to claim 19, wherein the network connecting section receives the first movement and model parameter from the network and outputs the first movement and model parameter to the image drawing section.

21. An image processing method comprising:

drawing a three-dimensional image on the basis of a three-dimensional model image and a first movement and model parameter;

receiving an input image of a still image or a motion picture;

searching for a point on the input image which corresponds to a certain point on the three-dimensional image to obtain corresponding-point position information; and

determining a second movement and model parameter by improving a value of the first movement and model parameter, to minimize a sum of distance functions determined by each of the corresponding-point position information obtained by the corresponding-point searching section.

22. An image processing method according to claim 21, wherein the corresponding-point position information is obtained by arranging a plurality of points on the three-dimensional image and searching for a point on the input image which corresponds to each of the plurality of points.

23. An image processing method according to claim 22, wherein the second movement and model parameter is calculated so as to minimize the distance between two corresponding points for the plurality of points arranged on the three-dimensional image and the corresponding plurality of points on the input image, when the three-dimensional image is drawn on the basis of the three-dimensional model image and the second movement and model parameter.

24. An image processing method according to claim 21, wherein a three-dimensional image is drawn on the basis of the three-dimensional model image and the second movement and model parameter,

corresponding-point position information is obtained on the three-dimensional image and the input image, and

the corresponding-point position information is used to improve a value for the second movement and model parameter to determine a third movement and model parameter.

25. An image processing method according to claim 24, wherein the three-dimensional image drawn on the basis of the three-dimensional model image and the third movement and model parameter is externally output.

26. An image processing method according to claim 21, wherein the second movement and model parameter is output to a network.

27. An image processing method according to claim 21, wherein the first movement and model parameter is input from a network.

28. An image processing method according to claim 21, wherein if the input image is an image of a person, an avatar image for movement, posture, or expression approximating to that in the image of the person is output.

29. An image processing method comprising:

using an input image of a still image or a motion picture to create a three-dimensional model image and a first movement and model parameter;

drawing a three-dimensional image on the basis of the three-dimensional model image and the first movement and model parameter;

receiving the input image of the still image or the motion picture;

searching for a point on the input image which corresponds to a certain point on the three-dimensional image to obtain corresponding-point position information; and

determining a second movement and model parameter by improving a value of the first movement and model parameter, to minimize a sum of distance functions determined by each of the corresponding-point position information obtained by the corresponding-point searching section.

30. An image processing method according to claim 29, wherein the corresponding-point position information is obtained by arranging a plurality of points on the three-dimensional image and searching for a point on the input image which corresponds to each of the plurality of points.

31. An image processing method according to claim 30, wherein the second movement and model parameter is calculated so as to minimize the distance between two corresponding points for the plurality of points arranged on the three-dimensional image and the corresponding plurality of points on the input image, when the three-dimensional image is drawn on the basis of the three-dimensional model image and the second movement and model parameter.

32. An image processing method according to claim 29, wherein a three-dimensional image is drawn on the basis of the three-dimensional model image and the second movement and model parameter,

corresponding-point position information is obtained on the three-dimensional image and the input image, and

the corresponding-point position information is used to improve a value for the second movement and model parameter to determine a third movement and model parameter.

33. An image processing method according to claim 32, wherein the three-dimensional image drawn on the basis of the three-dimensional model image and the third movement and model parameter is externally output.

34. An image processing method according to claim 29, wherein the second movement and model parameter is output to a network.

35. An image processing method according to claim 29, wherein the first movement and model parameter is input from a network.

36. An image processing method according to claim 29, wherein if the input image is an image of a person, an avatar image for movement, posture, or expression approximating to that in the image of the person is output.