Motion Capture Apparatus and Method

• US 20090322763A1
• Filed: 06/30/2009
• Published: 12/31/2009
• Est. Priority Date: 06/30/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Hyong-euk, Bang, Won-Chul
• Granted Patent: US 8,988,438 B2
• Time in Patent Office: Days
• US Class Current: 345/474
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1121 Determining geometric values, e.g. centre of rotation or angular range of movement
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 2218/00 Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

Provided are an apparatus and a method of effectively creating real-time movements of a three dimensional virtual character by use of a small number of sensors. More specifically, the motion capture method, which maps movements of a human body into a skeleton model to generate movements of a three-dimensional (3D) virtual character, includes measuring a distance between a portion of a human body to which a measurement sensor is positioned and a reference position and rotation angles of the portion, and estimating relative rotation angles and position coordinates of each portion of the human body by use of the measured distance and rotation angles.

Claims

1. A motion capture apparatus, comprising:

an estimation unit to estimate posture information, including one or more rotation angles of plural portions of a body different from a portion of the body where a measurement sensor is positioned at, based on a detected distance representation and one or more of rotation angle measurements by the measurement sensor, according to the measurement sensor being positioned at the portion of the body and detecting the distance representation between the measurement sensor and a reference position.

2. The motion capture apparatus of claim 1, wherein the plural portions of the body include another portion of the body having plural freedoms of motion, and wherein estimation of posture information for the other portion of the body is not based on measured posture information at the other portion of the body.

3. The motion capture apparatus of claim 2, wherein the portion of the body where the measurement sensor is positioned at has plural freedoms of motion.

4. The motion capture apparatus of claim 1, wherein the distance representation between the measurement sensor and the reference position defines a side of a triangle, with two other sides of the triangle being defined by two respective distances between portions of the body, and wherein the estimation unit estimates the posture information by defining at least one interior angle of the triangle.

5. The motion capture apparatus of claim 1, wherein, in the estimation of the posture information, the estimation unit defines a parallelogram, based on the distance representation between the measurement sensor and the reference position, to represent alternative positions of one of the plural portions, wherein sides of the parallelogram are defined by two respective distances between portions of the body.

6. The motion capture apparatus of claim 5, wherein the estimation unit selects one of the alternative positions of the one portion as a position of the one portion based on determined overlapping points of a first sphere having a radius of the distance representation and a second sphere having a radius of one of the two respective distances.

7. The motion capture apparatus of claim 1, wherein, in the estimation of the posture information, the estimation unit estimates the posture information by projecting into a plane one axis for one or more portions of the body and one axis of the portion of the body where the measurement sensor is positioned at to estimate posture information of the one or more portions of the body.

8. The motion capture apparatus of claim 1, further comprising the measurement sensor positioned at the portion of a body and to detect the distance representation between the measurement sensor and the reference position.

9. The motion capture apparatus of claim 8, further comprising a reference sensor to output reference position information, wherein the measurement sensor receives the output reference position information to detect the distance representation.

10. The motion capture apparatus of claim 9, wherein the reference sensor is positioned at the reference position.

11. The motion capture apparatus of claim 10, wherein the reference sensor is positioned on the body.

12. The motion capture apparatus of claim 11, wherein the reference sensor is positioned at a reference position portion of the body different from the portion of the body where the measurement sensor is positioned at and different from the plural portions of the body different from the portion of the body where the measurement sensor is positioned at.

13. The motion capture apparatus of claim 12, wherein the motion capture apparatus comprises at least four measurement sensors, including the measurement sensor positioned at the portion of the body, positioned at different portions of the body detecting distance representations between respective measurement sensors and the reference position, and the estimation unit estimates posture information, including one or more rotation angles of plural portions of the body different from the different portions of the body where the at least four measurement sensors are positioned at, based on the detected distance representations and one or more of rotation angle measurements of the respective measurement sensors.

14. The motion capture apparatus of claim 1, wherein the motion capture apparatus comprises at least four measurement sensors, including the measurement sensor positioned at the portion of the body, positioned at different portions of the body detecting distance representations between respective measurement sensors and the reference position, and

the estimation unit estimates posture information, including one or more rotation angles of plural portions of the body different from the different portions of the body where the at least four measurement sensors are positioned at, based on the detected distance representations and one or more of rotation angle measurements of the respective measurement sensors.

15. The motion capture apparatus of claim 1, further comprising the measurement sensor positioned at the portion of a body and to detect the distance representation between the measurement sensor and the reference position,

wherein the measurement sensor includes an inertial sensor that obtains position and rotations angles of the portion of the body where the measurement sensor is positioned at based on detected inertial information from the inertial sensor.

16. The motion capture apparatus of claim 15, wherein the inertial sensor measures an acceleration along at least one axis and an angular velocity along at least one axis, with the inertial information including the measured acceleration and the measured angular velocity.

17. The motion capture apparatus of claim 1, wherein the posture information includes rotation angles and position coordinates of the plural portions of the body and the measurement sensor measures posture information of the portion of the body where the measurement sensor is positioned at.

18. The motion capture apparatus of claim 1, wherein the posture information of the body is estimated based on an interpretation of the body, including the plural portions of the body, as an articulation model set having a link structure.

19. The motion capture apparatus of claim 18, wherein a portion among from ends of the link, which is a center of joint movements, is defined as a rotation center and the estimation unit estimates the posture information in consideration of a distance between the rotation center and the reference position.

20. The motion capture apparatus of claim 1, wherein the estimation unit estimates the posture information by restricting a range of a rotation angle of a joint in consideration of a human behavioral pattern or by specifying a movement plane.

21. The motion capture apparatus of claim 1, wherein the body is a human body, and the motion capture apparatus further comprises measurement sensors, including the measurement sensor positioned at the portion of the body, positioned at each of a head, wrists, and ankles of the human body, the measurement sensors respectively detecting distance representations between each measurement sensor and the reference position, and

wherein the estimation unit estimates posture information, including one or more rotation angles of the plural portions of the body different from rotation angle measurements of the measurement sensors, based on the detected distance representations by the measurement sensors and one or more rotation angles for each of the portions of the body measured by the measurement sensors.

22. The motion capture apparatus of claim 21, wherein at least one of the measurement sensors positioned at each of the wrists and ankles is attached to respective portions of the body physically adjacent to the respective wrists and ankles.

23. The motion capture apparatus of claim 1, wherein the reference position is a center portion of the body.

24. The motion capture apparatus of claim 1, wherein the body is a human body.

25. A motion capture method which maps movements of a body into a skeleton model for generating movements of a three-dimensional (3D) virtual character, the method comprising:

estimating posture information, including one or more rotation angles of plural portions of a body different from a portion of the body where a measurement sensor is positioned at, based on a detected distance representation and one or more of rotation angle measurements by the measurement sensor, according to the measurement sensor being positioned at the portion of the body and detecting the distance representation between the measurement sensor and a reference position.

26. The motion capture method of claim 25, wherein the plural portions of the body include another portion of the body having plural freedoms of motion, and wherein the estimating of posture information for the other portion of the body is not based on measured posture information by the other portion of the body.

27. The motion capture method of claim 26, wherein the portion of the body where the measurement sensor is positioned at has plural freedoms of motion.

28. The motion capture method of claim 25, wherein the distance representation between the measurement sensor and the reference position defines a side of a triangle, with two other sides of the triangle being defined by two respective distances between portions of the body, and wherein the estimating further comprises estimating the posture information by defining at least one interior angle of the triangle.

29. The motion capture method of claim 25, wherein, in the estimation of the posture information, the estimating further comprises defining a parallelogram, based on the distance representation between the measurement sensor and the reference position, to represent alternative positions of one of the plural portions, wherein sides of the parallelogram are defined by two respective distances between portions of the body.

30. The motion capture method of claim 29, wherein the estimating further comprises selecting one of the alternative positions of the one portion as a position of the one portion based on determined overlapping points of a first sphere having a radius of the distance representation and a second sphere having a radius of one of the two respective distances.

31. The motion capture method of claim 25, wherein, in the estimating of the posture information, the estimating further comprises estimating the posture information by projecting into a plane one axis for one or more portions of the body and one axis of the portion of the body where the measurement sensor is positioned at to estimate posture information of the one or more portions of the body.

32. The motion capture method of claim 25, further comprising outputting reference position information, from a reference sensor, and the measurement sensor detecting the distance representation based on receipt of the reference position information.

33. The motion capture method of claim 32, wherein the reference sensor is positioned at the reference position.

34. The motion capture method of claim 33, wherein the reference sensor is positioned on the body.

35. The motion capture method of claim 34, wherein the reference sensor is positioned at a reference position portion of the body different from the portion of the body where the measurement sensor is positioned at and different from the plural portions of the body different from the portion of the body where the measurement sensor is positioned at.

36. The motion capture method of claim 35, wherein the estimating further comprises estimating posture information, including one or more rotation angles of plural portions of the body different from different portions of the body where at least four measurement sensors are positioned at, based on detected distance representations respectively from the at least four measurement sensors and one or more of rotation angle measurements of the respective measurement sensors,

wherein the at least four measurement sensors include the measurement sensor positioned at the portion of the body.

37. The motion capture method of claim 25, wherein the estimating further comprises estimating posture information, including one or more rotation angles of plural portions of the body different from different portions of the body where at least four measurement sensors are positioned at, based on detected distance representations respectively from the at least four measurement sensors and one or more of rotation angle measurements of the respective measurement sensors,

wherein the at least four measurement sensors include the measurement sensor positioned at the portion of the body.

38. The motion capture method of claim 25, further comprising:

generating a 3D virtual character corresponding to the body by use of the one or more rotation angles and position coordinates of the plural portions of the body and the portion of the body where the measurement sensor is positioned at.

39. The motion capture method of claim 25, wherein the posture information of the body is estimated based on an interpretation of the body, including the plural portions of the body, as an articulation model set having a link structure.

40. The motion capture method of claim 39, wherein a portion among from ends of the link, which is a center of joint movements, is defined as a rotation center and the estimating further comprises estimating the posture information in consideration of a distance between the rotation center and the reference position.

41. The motion capture method of claim 39, wherein, in the estimating of the one or more rotation angles and an estimating of position coordinates, when the rotation center of the link structure is co-located with the reference position, a rotation angle of a joint having one degree of freedom is calculated before rotation angles of a joint having three degrees of freedom are calculated.

42. The motion capture method of claim 39, wherein, in the estimating of the one or more rotation angles and an estimating of position coordinates, when the rotation center of the link structure is not co-located with the reference position, candidates for a rotation angle and a rotation axis of a joint having one degree of freedom are estimated, a movement plane is set in consideration of a behavioral pattern of the body, and then rotation angles of previously unestimated joints are calculated.

43. The motion capture method of claim 25, wherein the one or more rotation angles and an estimating of position coordinates of the plural portions are restricted or a movement plane is determined according to situations in consideration of a human behavioral pattern in order to estimate one or more rotation angles and the position coordinates.

44. The motion capture method of claim 25, wherein the body is a human body, wherein the estimating further comprises estimating posture information, including one or more rotation angles of the plural portions of the body different from rotation angle measurements of measurement sensors positioned at each of a head, wrists, and ankles portions of the body, based on detected distance representations by the measurement sensors, and one or more rotation angles for each portion of the body measured by the measurement sensors,

wherein the measurement sensors respectively detect the distance representations between each measurement sensor and the reference position.

45. The motion capture method of claim 44, wherein at least one of the measurement sensors positioned at each of the wrists and ankles is attached to respective portions of the body physically adjacent to the respective wrists and ankles.

46. The method of claim 25, wherein the reference position is at a center portion of the body or not on the body.

47. A computer readable recording medium comprising computer readable code to control at least one processing device to implement the method of claim 25.

48. A motion capture apparatus, comprising:

an estimation unit which estimates posture information, including one or more rotation angles of non-extremity portions of a body based on one or more detected distance representations and rotation angle measurements measured by one or more measurement sensors, with the non-extremity portions including at least one of both an elbow and a shoulder and both a knee and a pelvis based on the one or more measurement sensors being positioned at a corresponding wrist area and/or ankle area of the body based on which non-extremity portions of the body posture information is being estimated by the estimation unit,

wherein the one or more measurement sensors are positioned only at respective different extremity portions of the body and respectively detect the one or more distance representations as between each measurement sensor and a reference position.

49. The motion capture apparatus of claim 48, further comprising the one or more measurement sensors to be positioned only at the different extremity portions of the body and to respectively detect the one or more distance representations.

50. A motion capture method, comprising:

estimating posture information, including one or more rotation angles of non-extremity portions of a body based on one or more detected distance representations and rotation angle measurements measured by one or more measurement sensors, with the non-extremity portions including at least one of both an elbow and a shoulder and both a knee and a pelvis based on the one or more measurement sensors being positioned at a corresponding wrist area and/or ankle area of the body based on which non-extremity portions of the body posture information is being estimated by the estimating,

wherein the one or more measurement sensors are positioned only at respective different extremity portions of the body and respectively detect the one or more distance representations as between each measurement sensor and a reference position.

51. The motion capture method of claim 50, further comprising detecting the one or more distance representations by the measurement sensors.

52. A computer readable recording medium comprising computer readable code to control at least one processing device to implement the method of claim 50.