Method and system for tracking positions of human extremities

• US 8,204,274 B2
• Filed: 02/13/2009
• Issued: 06/19/2012
• Est. Priority Date: 07/21/2008
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chen, Po-Lung, Kuo, Chien-Chun, Wang, Ko-Shyang
• Primary Examiner(s): Bhatnagar, Anand
• Application Number:
  US12/371,487
• Publication Number:
  US 20100014710A1
• Time in Patent Office: 1,222 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 382115-118, 382/154, 382/181, 382/190, 382/195, 382/206, 348/143, 348169-172, 345419-427, 356/12
• US Class Current: 382/103
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method for tracking positions of human extremities is disclosed. A left image of a first extremity portion is retrieved using a first picturing device and an outline candidate position of the first extremity portion is obtained according to feature information of the left image. A right image of the first extremity portion is retrieved using a second picturing device and a depth candidate position of the first extremity portion is obtained according to depth information of the right image. Geometry relations between the outline candidate position and the depth candidate position and a second extremity portion of a second extremity position are calculated to determine whether a current extremity position of the first extremity portion is required to be updated.

Claims

1. A method for tracking positions of human extremities, comprising:

retrieving a left image of a first extremity portion using a first picturing device;

obtaining an outline candidate position of the first extremity portion according to feature information of the left image;

retrieving a right image of the first extremity portion using a second picturing device;

obtaining a depth candidate position of the first extremity portion according to depth information of the right image;

calculating geometry relations between the outline candidate position and the depth candidate position and a second extremity position of a second extremity portion; and

determining whether a current extremity position of the first extremity portion is required to be updated.

2. The method for tracking positions of human extremities as claimed in claim 1, further comprising:

implementing a background and edge/outline removing to the left image to obtain feature information of the first extremity portion;

obtaining the outline candidate position of edge/outline of the first extremity portion according to the feature information of the first extremity position;

based on a extremity position previously tracked of the first extremity portion, calculating a first distance between the outline candidate position and the extremity position and then calculating outline tracking reliability of the first extremity portion; and

calculating outline geometry reliability of the outline candidate position according to the second extremity position of the second extremity portion.

3. The method for tracking positions of human extremities as claimed in claim 2, further comprising:

calculating depth information of the first extremity portion according to the left and right images;

obtaining the depth candidate position of the first extremity portion according to the depth information of the right image;

based on the extremity position previously tracked of the first extremity portion, calculating a second distance between the depth candidate position and the extremity position and then calculating depth tracking reliability of the first extremity portion; and

calculating depth geometry reliability of the depth candidate position according to the second extremity position of the second extremity portion.

4. The method for tracking positions of human extremities as claimed in claim 3, further comprising:

comparing the outline tracking reliability with the depth tracking reliability and comparing the outline geometry reliability with the depth geometry reliability to select one of the reliability with greater reliability;

comparing the selected reliability with predefined reliability threshold to determine whether the selected reliability is greater than the reliability threshold;

if greater than the reliability threshold, updating a candidate position corresponding to the selected reliability by a current extremity position of the first extremity portion; and

if not greater than the reliability threshold, serving the extremity position previously tracked as the current extremity position of the first extremity portion.

5. The method for tracking positions of human extremities as claimed in claim 2, wherein the outline tracking reliability is inversive to the first distance.

6. The method for tracking positions of human extremities as claimed in claim 1, wherein the outline candidate position reliability is inversive but is not linear to the second extremity position of the second extremity portion.

7. The method for tracking positions of human extremities as claimed in claim 1, wherein the first extremity portion is the head, hand, leg, elbow, finger, or other extremity portions, or a mark on one of the described extremity portions.

8. The method for tracking positions of human extremities as claimed in claim 1, wherein a first extremity position of the first extremity portion, the outline candidate position, the depth candidate position, or the second extremity position of the second extremity portion is a two-dimensional image and is represented by image coordinates or image blocks.

9. The method for tracking positions of human extremities as claimed in claim 1, further comprising serving a turning point of the outline or edge of the first extremity portion as the outline candidate position.

10. The method for tracking positions of human extremities as claimed in claim 1, further comprising calculating the depth information according to difference between two images of the first extremity portion.

11. The method for tracking positions of human extremities as claimed in claim 1, further comprising calculating the depth information according to difference between partial regions of two images of the first extremity portion.

12. The method for tracking positions of human extremities as claimed in claim 1, further comprising calculating the depth information using a 3D reconstruction method.

13. The method for tracking positions of human extremities as claimed in claim 1, wherein second extremity portion is the head, face, neck, central point of the body, central axis of the body, waist, cross-leg, or other extremity portions for reference, or a mark on one of the described extremity portions.

14. The method for tracking positions of human extremities as claimed in claim 1, wherein the geometry relations comprise calculating a distance between the outline candidate position and the second extremity position of the second extremity portion.

15. The method for tracking positions of human extremities as claimed in claim 1, wherein the geometry relations comprise calculating a distance between the outline candidate position and the second extremity position of the second extremity portion or between the depth candidate position and the second extremity position of the second extremity portion.

16. The method for tracking positions of human extremities as claimed in claim 1, wherein the geometry relations comprise increasing probability of updating a first extremity position of the first extremity portion to a closer distance when the outline candidate position or the depth candidate position is closer to the second extremity position of the second extremity portion.

17. The method for tracking positions of human extremities as claimed in claim 1, wherein the geometry relations comprise increasing probability of updating a first extremity position of the first extremity portion by the outline candidate position when the outline candidate position is farther to position T, or decreasing the probability when closer to the position T.

18. The method for tracking positions of human extremities as claimed in claim 1, wherein the geometry relations comprise increasing probability of updating a first extremity position of the first extremity portion by the outline candidate position when difference between coordinates of the outline candidate position residing in a specific axis and that of the second extremity position of the second extremity portion residing in the specific axis is greater, or decreasing the probability when less.

19. The method for tracking positions of human extremities as claimed in claim 1, wherein the geometry relations comprise increasing probability of updating a first extremity position of the first extremity portion by the depth candidate position when difference between coordinates of the depth candidate position residing in a specific axis and that of the second extremity position of the second extremity portion residing in the specific axis is less, or decreasing the probability when greater.

20. The method for tracking positions of human extremities as claimed in claim 1, further comprising determining whether a first extremity position of the first extremity portion is replaced by the outline candidate position or the depth candidate position.

21. A system for tracking positions of human extremities, comprising:

a first picturing device;

a second picturing device; and

an operation device, retrieving a left image of a first extremity portion using the first picturing device, obtaining an outline candidate position of the first extremity portion according to feature information of the left image, retrieving a right image of the first extremity portion using the second picturing device, obtaining a depth candidate position of the first extremity portion according to depth information of the right image, calculating geometry relations between the outline candidate position and the depth candidate position and a second extremity position of a second extremity portion, and determining whether a current extremity position of the first extremity portion is required to be updated.

22. The system for tracking positions of human extremities as claimed in claim 21, wherein the operation device implements a background and edge/outline removing to the left image to obtain feature information of the first extremity portion, obtains the outline candidate position of edge/outline of the first extremity portion according to the feature information of the first extremity position, based on a extremity position previously tracked of the first extremity portion, calculates a first distance between the outline candidate position and the extremity position and then calculating outline tracking reliability of the first extremity portion, and calculates outline geometry reliability of the outline candidate position according to the second extremity position of the second extremity portion.

23. The system for tracking positions of human extremities as claimed in claim 22, wherein the operation device calculates depth information of the first extremity portion according to the left and right images, obtains the depth candidate position of the first extremity portion according to the depth information of the right image, based on the extremity position previously tracked of the first extremity portion, calculates a second distance between the depth candidate position and the extremity position and then calculating depth tracking reliability of the first extremity portion, and calculates depth geometry reliability of the depth candidate position according to the second extremity position of the second extremity portion.

24. The system for tracking positions of human extremities as claimed in claim 23, wherein the operation device compares the outline tracking reliability with the depth tracking reliability and compares the outline geometry reliability with the depth geometry reliability to select one of the reliability with greater reliability, compares the selected reliability with reliability threshold to determine whether the selected reliability is greater than the reliability threshold, if greater than the reliability threshold, updates a candidate position corresponding to the selected reliability by a current extremity position of the first extremity portion, and, if not greater than the reliability threshold, serves the extremity position previously tracked as the current extremity position of the first extremity portion.

25. The system for tracking positions of human extremities as claimed in claim 22, wherein the outline tracking reliability is inversive to the first distance.

26. The system for tracking positions of human extremities as claimed in claim 21, wherein the outline candidate position reliability is inversive but is not linear to the second extremity position of the second extremity portion.

27. The system for tracking positions of human extremities as claimed in claim 21, wherein the first extremity portion is the head, hand, leg, elbow, finger, or other extremity portions, or a mark on one of the described extremity portions.

28. The system for tracking positions of human extremities as claimed in claim 21, wherein a first extremity position of the first extremity portion, the outline candidate position, the depth candidate position, or the second extremity position of the second extremity portion is a two-dimensional image and is represented by image coordinates or image blocks.

29. The system for tracking positions of human extremities as claimed in claim 21, wherein the operation device serves a turning point of the outline or edge of the first extremity portion as the outline candidate position.

30. The system for tracking positions of human extremities as claimed in claim 21, wherein the operation device calculates the depth information according to difference between two images of the first extremity portion.

31. The system for tracking positions of human extremities as claimed in claim 21, wherein the operation device calculates the depth information according to difference between partial regions of two images of the first extremity portion.

32. The system for tracking positions of human extremities as claimed in claim 21, wherein the operation device calculates the depth information using a 3D reconstruction method.

33. The system for tracking positions of human extremities as claimed in claim 21, wherein second extremity portion is the head, face, neck, central point of the body, central axis of the body, waist, cross-leg, or other extremity portions for reference, or a mark on one of the described extremity portions.

34. The system for tracking positions of human extremities as claimed in claim 21, wherein the geometry relations comprise calculating a distance between the outline candidate position and the second extremity position of the second extremity portion.

35. The system for tracking positions of human extremities as claimed in claim 21, wherein the geometry relations comprise calculating a distance between the outline candidate position and the second extremity position of the second extremity portion or between the depth candidate position and the second extremity position of the second extremity portion.

36. The system for tracking positions of human extremities as claimed in claim 21, wherein the geometry relations comprise increasing probability of updating a first extremity position of the first extremity portion to a closer distance when the outline candidate position or the depth candidate position is closer to the second extremity position of the second extremity portion.

37. The system for tracking positions of human extremities as claimed in claim 21, wherein the geometry relations comprise increasing probability of updating a first extremity position of the first extremity portion by the outline candidate position when the outline candidate position is farther to position T, or decreasing the probability when closer to the position T.

38. The system for tracking positions of human extremities as claimed in claim 21, wherein the geometry relations comprise increasing probability of updating a first extremity position of the first extremity portion by the outline candidate position when difference between coordinates of the outline candidate position residing in a specific axis and that of the second extremity position of the second extremity portion residing in the specific axis is greater, or decreasing the probability when less.

39. The system for tracking positions of human extremities as claimed in claim 21, wherein the geometry relations comprise increasing probability of updating a first extremity position of the first extremity portion by the depth candidate position when difference between coordinates of the depth candidate position residing in a specific axis and that of the second extremity position of the second extremity portion residing in the specific axis is less, or decreasing the probability when greater.

40. The system for tracking positions of human extremities as claimed in claim 21, wherein the operation device determines whether a first extremity position of the first extremity portion is replaced by the outline candidate position or the depth candidate position.