Form changing device, object action encoding device, and object action decoding device

• US 7,668,401 B2
• Filed: 08/25/2004
• Issued: 02/23/2010
• Est. Priority Date: 09/03/2003
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Marugame, Atsushi
• Primary Examiner(s): Kassa; Yosef
• Application Number:
  US10/570,560
• Publication Number:
  US 20060285758A1
• Time in Patent Office: 2,008 Days
• Field of Search: 382/128, 382/131, 382/132, 382/209, 382/219
• US Class Current: 382/278
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame

Abstract

A shape deformation encoding device includes: a calculation unit for calculating a difference data between a pre-deformation shape data which represents a shape of an object before shape change and a post-deformation shape data which represents a shape of the object after shape change; and a determination unit for determining an action area on which the shape change of the object is arisen and an external force which is acted to the action area for the shape change, based on the pre-deformation shape data and the difference data. The action area includes a plurality of small areas, each of the plurality of small areas includes a plurality of control points, and the external force is calculated as an external force required to deform the plurality of small areas from before to after the shape change independently based on the physical model structure of each of the plurality of small areas. The physical model structure in which the plurality of control points are connected together with springs and dampers are used as a physical model structure.

Claims

1. A shape deformation encoding device comprising:

a calculation unit configured to calculate a difference data between a pre-deformation shape data which represents a shape of an object before shape change and a post-deformation shape data which represents a shape of an object after shape change; and

a determination unit configured to determine an action area on which said shape change of said object is arisen and an external force which is acted to said action area for said shape change, based on said pre-deformation shape data and said difference data.

2. The shape deformation encoding device according to claim 1,

wherein said action area includes a plurality of small areas, each of said plurality of small areas includes a plurality of control points, and

a physical model structure in which said plurality of control points are connected together with springs and dampers as a physical model structure for obtaining said external force.

3. The shape deformation encoding device according to claim 2, wherein said calculation unit includes:

a transformation unit transforming said pre-deformation shape data and said post-deformation shape data into a pre-deformation shape model data and post deformation shape model data; and

a difference calculation unit calculating said difference data from a difference between said pre-deformation shape model data and said post-deformation shape model data.

4. The shape deformation encoding device according to claim 3,

wherein said model is a three-dimensional polygon mesh model, and said pre-deformation shape data and said post-deformation shape data are three-dimensional data.

5. The shape deformation encoding device according to claim 2,

said external force is calculated as an external force required to deform said plurality of small areas from before to after said shape change independently based on said physical model structure of each of said plurality of small areas.

6. The shape deformation encoding device according to claim 1,

wherein a number of said plurality of control points and said physical model structure are same for all said plurality of small areas,

said external force is calculated as an external force required to deform said plurality of small areas from before to after said shape change independently based on said physical model structure of each of said plurality of small areas, and

a number of said plurality of control points and said physical model structure are same for all said plurality of small areas.

7. The shape deformation encoding device according to claim 1,

wherein said object includes a human face, said shape of an object before shape change represents an expressionless status of a face, and said shape of an object after shape change represents an expressive status of a face.

8. A shape deformation decoding device comprising:

a decoding unit configured to determine a movement position of a control point within an action area based on a pre-deformation shape data representing a shape of an object before shape change and information of an external force acted to said action area correlated with a shape change, wherein said action area includes a plurality of small areas, and a movement position of said action point of each of said plurality of small areas is determined by using a physical model by a unit of said plurality of small areas; and

a shape generation unit configured to generate a post-deformation shape data which represents a shape of said object after said shape change from said pre-deformation shape data and a movement position of each of said plurality of control points.

9. The shape deformation decoding device according to claim 8,

wherein said shape generation unit generates a shape of said object after said shape change by a unit of said plurality of small areas from said pre-deformation shape data and a movement position of each of said plurality of control points,

and further comprising:

an adjacent small areas adjustment unit configured to smooth a discontinuous portion occurring at a boundary part of adjacent ones of said plurality of small areas after said shape change.

10. The shape deformation decoding device according to claim 9,

wherein said pre-deformation shape data is three-dimensional data, and

said decoding unit includes:

a model data generation unit generating pre-deformation shape model data based on a three-dimensional model of a shape of said object before said shape change and said pre-deformation shape data; and

a movement amount calculation unit determining a movement position of each of control points within said action area from said pre-deformation shape model data based on said information of an external force.

11. The shape deformation decoding device according to claim 9,

wherein said pre-deformation shape data is three-dimensional data, and

said decoding unit includes:

a mesh model data generation unit generating a pre-deformation shape mesh model data based on a three dimensional polygon mesh model and said pre-deformation shape data; and

a movement amount calculation unit deciding a movement position of each of control points within said action area from said pre-deformation shape mesh model data based on said information of an external force.

12. The shape deformation decoding device according to claim 9, further comprising:

an action interpolation unit calculating a movement position of non-control point which is not the control point from a movement point of said control point adjacent to said non-control point for each of said plurality of small areas.

13. The shape deformation decoding device according to claim 8, further comprising:

a non-action area adjustment unit configured to smooth a discontinuous portion occurring at a boundary part between said action area and a surrounding non-action area after said shape change.

14. The shape deformation decoding device according to claim 8,

wherein said plurality of control points are connected by a spring and a damper in a physical model of each of said plurality of small areas.

15. The shape deformation decoding device according to claim 8,

wherein said object includes a human face, said shape before shape change represents an expressionless face, and said shape after shape change represents an expressive face.

16. The shape deformation decoding device according to claim 8, further comprising:

a three-dimensional actuator group driving said action area of said object having flexibility based on said post-deformation shape data.

17. A shape deformation encoding-decoding device comprising:

a shape deformation encoding device comprising a calculation unit configured to calculate a difference data between a pre-deformation shape data which represents a shape of an object before shape change and a post-deformation shape data which represents a shape of an object after shape change;

and a determination unit configured to determine an action area on which said shape change of said object is arisen and an external force which is acted to said action area for said shape change, based on said pre-deformation shape data and said difference data;

a shape deformation encoding device comprising a decoding unit configured to determine a movement position of a control point within an action area based on a pre-deformation shape data representing a shape of an object before shape change and information of an external force acted to said action area correlated with a shape change, wherein said action area includes a plurality of small areas, and a movement position of said action point of each of said plurality of small areas is determined by using a physical model by a unit of said plurality of small areas;

and a shape generation unit configured to generate a post-deformation shape data which represents a shape of said object after said shape change from said pre-deformation shape data and a movement position of each of said plurality of control points;

and a transmission device configured to transmit said action area generated by said shape deformation encoding device and said information of an external force to said shape deformation decoding device through a communication path.

18. A shape deformation encoding method comprising the steps of:

calculating a difference data between a pre-deformation shape data which represents a shape of an object before shape change and a post-deformation shape data which represents a shape of an object after shape change; and

determining an action area on which said shape change of said object is arisen and an external force which is acted to said action area for said shape change, based on said pre-deformation shape data and said difference data.

19. The shape deformation encoding method according to claim 18,

wherein said action area includes a plurality of small areas, each of said plurality of small areas includes a plurality of control points, and

a physical model structure in which said plurality of control points are connected together with springs and dampers are used as a physical model structure for obtaining said external force.

20. The shape deformation encoding method according to claim 19, wherein said calculating step includes the steps of:

transforming said pre-deformation shape data and said post-deformation shape data into a pre-deformation shape model data and post-deformation shape model data; and

calculating said difference data from a difference between said pre-deformation shape model data and said post-deformation shape model data.

21. The shape deformation encoding method according to claim 20,

wherein said model is a three-dimensional polygon mesh model, and said pre-deformation shape data and said post-deformation shape data are three-dimensional data.

22. The shape deformation encoding method according to claim 18,

wherein said action area includes a plurality of small areas, each of said plurality of small areas includes a plurality of control points, and

said external force is calculated as an external force required to deform said plurality of small areas from before to after said shape change independently based on a physical model structure of each of said plurality of small areas.

23. The shape deformation encoding method according to claim 18,

wherein said action area includes a plurality of small areas, each of said plurality of small areas includes a plurality of control points, and

said external force is calculated as an external force required to deform said plurality of small areas from before to after said shape change independently based on a physical model structure of each of said plurality of small areas, and

a number of said plurality of control points and said physical model structure are same for all said plurality of small areas.

24. The shape deformation encoding method according to claim 18,

wherein said object includes a human face, said shape of an object before shape change represents an expressionless status of a face, and said shape of an object after shape change represents an expressive status of a face.

25. A shape deformation decoding method comprising the steps of:

determining a movement position of a control point within an action area based on a pre-deformation shape data representing a shape of an object before shape change and information of an external force acted to said action area correlated with a shape change, wherein said action area includes a plurality of small areas, and a movement position of said action point of each of said plurality of small areas is determined by using a physical model by a unit of said plurality of small areas; and

generating a post-deformation shape data which represents a shape of said object after said shape change from said pre-deformation shape data and a movement position of each of said plurality of control points.

26. The shape deformation decoding method according to claim 25,

wherein said step of generating a post-deformation shape data generates a shape of said object after said shape change by a unit of said plurality of small areas from said pre-deformation shape data and a movement position of each of said plurality of control points,

and further comprising the step of:

smoothing a discontinuous portion occurring at a boundary part of adjacent ones of said plurality of small areas after said shape change.

27. The shape deformation decoding method according to claim 26,

wherein said pre-deformation shape data is three-dimensional data, and

said step of determining a movement position includes the steps of:

generating pre-deformation shape model data based on a three-dimensional model of a shape of said object before said shape change and said pre-deformation shape data; and

determining a movement position of each of control points within said action area from said pre-deformation shape model data based on said information of an external force.

28. The shape deformation decoding method according to claim 26,

wherein said pre-deformation shape data is three-dimensional data, and

said step of determining a movement position includes the steps of:

generating a pre-deformation shape mesh model data based on a three dimensional polygon mesh model and said pre-deformation shape data; and

deciding a movement position of each of control points within said action area from said pre-deformation shape mesh model data based on said information of an external force.

29. The shape deformation decoding method according to claim 26, further comprising the step of:

calculating a movement position of non-control point which is not the control point from a movement point of said control point adjacent to said non-control point for each of said plurality of small areas.

30. The shape deformation decoding method according to claim 25,

wherein said step of generating a post-deformation shape data generates a shape of said object after said shape change by a unit of said plurality of small areas from said pre-deformation shape data and a movement position of each of said plurality of control points,

and further comprising the step of:

smoothing a discontinuous portion occurring at a boundary part between said action area and a surrounding non-action area after said shape change.

31. The shape deformation decoding method according to claim 25,

wherein said plurality of control points are connected by a spring and a damper in a physical model of each of said plurality of small areas.

32. The shape deformation decoding method according to claim 25,

wherein said object includes a human face, said shape before shape change represents an expressionless face, and said shape after shape change represents an expressive face.

33. The shape deformation decoding method according to claim 25,

wherein said object is a robot, and further comprising the step of:

driving said action area of said object having flexibility based on said post-deformation shape data.

34. A shape deformation encoding-decoding method comprising the steps of:

generating information of an action area and information of an external force by calculating a difference data between a pre-deformation shape data which represents a shape of an object before shape change and a post-deformation shape data which represents a shape of an object after shave change, and determining an action area on which said shape change of said object is arisen and an external force which is acted to said action area for said shape change, based on said pre-deformation shape data and said difference data;

transmitting said information of an action area and said information of an external force through a communication path; and

generating post-deformation shape data representing a shape of said object from said transmitted information of an action area and said transmitted information of an external force by the method according to claim 25.

35. A software product embodied on a computer-readable medium and comprising code that, when executed, causes a computer to perform a shape deformation encoding method including the following steps of:

calculating a difference data between a pre-deformation shape data which represents a shape of an object before shape change and a post-deformation shape data which represents a shape of an object after shape change; and

determining an action area on which said shape change of said object is arisen and an external force which is acted to said action area for said shape change, based on said pre-deformation shape data and said difference data.

36. The software product according to claim 35,

wherein said action area includes a plurality of small areas, each of said plurality of small areas includes a plurality of control points, and

a physical model structure in which said plurality of control points are connected together with springs and dampers is used as a physical model structure for determining said external force.

37. The software product according to claim 36,

wherein said step of calculating includes the steps of:

transforming said pre-deformation shape data and said post-deformation shape data into pre-deformation shape model data and post-deformation shape model data based on a three-dimensional model; and

calculating said difference data from a difference between said pre-deformation shape model data and said post-deformation shape model data.

38. The software product according to claim 37,

wherein said model is a three-dimensional polygon mesh model, and said pre-deformation shape data and said post-deformation shape data are three-dimensional data.

39. The software product according to claim 36,

wherein said action area includes a plurality of small areas, each of said plurality of small areas includes a plurality of control points, and

said external force is calculated as an external force required to deform said plurality of small areas from before to after said shape change independently based on a physical model structure of each of said plurality of small areas.

40. The software product according to claim 35,

wherein said action area includes a plurality of small areas, each of said plurality of small areas includes a plurality of control points,

said physical model structure in which said plurality of control points are connected together with springs and dampers are used as a physical model structure, and

a number of said plurality of control points and said physical model structure are same for all said plurality of small areas.

41. The software product according to claim 35,

wherein said object includes a human face, said shape of an object before shape change represents an expressionless status of a face, and said shape of an object after shape change represents an expressive status of a face.

42. A software product embodied on a computer readable medium and comprising code that, when executed, causes a computer to perform a shape deformation decoding method including the following steps of:

determining a movement position of a control point within an action area based on a pre-deformation shape data representing a shape of an object before shape change and information of an external force acted to said action area correlated with a shape change, wherein said action area includes a plurality of small areas, and a movement position of said action point of each of said plurality of small areas is determined by using a physical model by a unit of said plurality of small areas; and

generating a post-deformation shape data which represents a shape of said object after said shape change from said pre-deformation shape data and a movement position of each of said plurality of control points,

wherein said plurality of control points are connected to each other through a spring and a damper at a physical model of each of said plurality of small areas.

43. The software product according to claim 42,

wherein said pre-deformation shape data is three-dimensional data, and

said step of determining a movement position includes the steps of:

generating pre-deformation shape model data based on a three-dimensional model of a shape of said object before said shape change and said pre-deformation shape data; and

determining a movement position of each of control points within said action area from said pre-deformation shape model data based on said information of an external force.

44. The software product according to claim 42,

wherein said pre-deformation shape data is three-dimensional data, and

said step of determining a movement position includes the steps of:

generating a pre-deformation shape mesh model data based on a three dimensional polygon mesh model and said pre-deformation shape data; and

deciding a movement position of each of control points within said action area from said pre-deformation shape mesh model data based on said information of an external force.

45. The software product according to claim 42, the method further comprises the step of:

calculating a movement position of non-control point which is not the control point from a movement point of said control point adjacent to said non-control point for each of said plurality of small areas.

46. The software product according to claim 42,

wherein said step of generating a post-deformation shape data generates a shape of said object after said shape change by a unit of said plurality of small areas from said pre-deformation shape data and a movement position of each of said plurality of control points.

47. The software product according to claim 42, the method further comprises the step of:

smoothing a discontinuous portion occurring at a boundary part of adjacent ones of said plurality of small areas after said shape change.

48. The software product according to claim 42, the method further comprises the step of:

smoothing a discontinuous portion occurring at a boundary part between said action area and a surrounding non-action area after said shape change.

49. The software product according to claim 42,

wherein said object includes a human face, said shape before shape change represents an expressionless face, and said shape after shape change represents an expressive face.

50. The software product according to claim 42,

wherein said object is a robot, and the method further comprises the step of:

driving said action area of said object having flexibility based on said post-deformation shape data.