Distributed-processing motion tracking system for tracking individually modulated light points

• US 6,324,296 B1
• Filed: 12/04/1997
• Issued: 11/27/2001
• Est. Priority Date: 12/04/1997
• Status: Expired due to Term

• Current Assignee: Phasespace, Inc.
• Original Assignee: Phasespace, Inc.
• Inventors: Johnson, Jack L., McSheery, Tracy D., Nollet, Scott R., Jivan, Vinay C., Black, John R.
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Werner, Brian P.
• Application Number:
  US08/985,354
• Time in Patent Office: 1,454 Days
• Field of Search: 382/103, 382/106, 382/107, 382/118, 382/153, 382/154, 382/288, 382/291, 382/304, 382/323, 345/358, 356/3.14, 356/3.15, 356/375, 356/139.03, 356/152.1, 356/620, 356/623, 250/203.2, 250/203.3, 250/559.32, 250/559.35, 250/206.2, 250/206.3, 901/47, 348/77, 348/119, 348/139, 348/169, 348/172, 341/56, 702/142, 702/152, 702/153, 702/172
• US Class Current: 382/107
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

Disclosed is a distributed-processing motion capture system (and inherent method) comprising: plural light point devices, e.g., infrared LEDs, in a motion capture environment, each providing a unique sequence of light pulses representing a unique identity (ID) of a light point device; a first imaging device for imaging light along a first and second axis; and a second imaging device for imaging light along a third and fourth axis. Both of the imaging devices filter out information not corresponding to the light point devices, and output one-dimensional information that includes the ID of a light point device and a position of the light point device along one of the respective axes. The system also includes a processing device for triangulating three-dimensional positions of the light point devices based upon the one-dimensional information. The system is very fast because the necessary processing is distributed to be maximally parallel. The motion capture system uses a cylindrical collimating (CC) optics sub-system superimposed on a cylindrical telecentric (CT) optics sub-system. The outputs of the plural light point devices are modulated to provide a unique sequence of light pulses representing a unique identifier (ID) for each of the light point devices according to a predetermined cycle of modulation intervals based upon synchronization signals provided via RF communication. At least two of the light point devices concurrently provide light during the cycle.

Claims

1. A processing motion capture system comprising:

a plurality of light point devices attached to an object to be tracked in a motion capture environment, each being operable to provide a continuous non-zero output over an interval corresponding to a plurality of pulse periods so as to define a plurality of continuous non-zero light pulses which vary in amplitude over said interval in a sequence that represents a unique plural bit digital identity (ID) of a light point device, each bit in said digital ID corresponding to an amplitude level within one of said pulse periods;

said amplitude level changing at least once over said interval; and

a plurality of image capture devices synchronized with said light pulses and capturing a sequence of images of said pulses substantially corresponding to said plurality of light point devices; and

processing circuitry to recognize the identities of, and to track positions of, substantially all of said plurality of light point devices based upon said light pulses appearing within said sequence of images, respectively.

2. The system of claim 1, wherein said processing circuitry is operable to track positions of said plurality of light point devices simultaneously.

3. The system of claim 1, wherein said processing circuitry is distributed such that a total processing power of said processing circuitry is represented by at least a first processing stage and a second processing stage, each stage including a plurality of processing units, wherein an output of said processing circuitry is produced by said second processing stage, and said output of said second processing stage is dependent upon data generated by said first processing stage.

4. The system of claim 3, wherein said first processing stage is arranged into a first processing tier and a second processing tier, and said first processing tier has twice as many processing units as said second processing tier, and wherein said second processing tier provides said output of said second processing stage and said output of said second processing tier is dependent upon data generated by said first processing tier.

5. The system of claim 1, wherein said light pulses are modulated to represent a first logical value or a second logical value so that the digital identities are a sequence of N logical values, where N is a positive integer and 2^N total light point devices are identifiable, and wherein said processing circuitry is operable to synchronize said light pulses with image frames so as to recognize said digital identities by considering any set of N fields.

6. The system of claim 1, wherein said processing circuitry includes:

first imaging means for imaging light in said motion capture environment along a first reference axis and a second reference axis oriented at a first predetermined angle with respect to said first reference axis, for filtering out information not corresponding to said light point devices, and for outputting one-dimensional information that includes the ID of a light point device, a position of said light point device along said first reference axis and a position of said light point device along said second reference axis; and

second imaging means for imaging light in said motion capture environment along a third reference axis and a fourth reference axis oriented at a second predetermined angle with respect to said third reference axis, for filtering out information not corresponding to said light point devices, and for outputting one-dimensional information that includes the ID of a light point device, a position of said light point device along said third reference axis and a position of said light point device along said fourth reference axis;

corresponding one-dimensional information from said first and second imaging means being sufficient to yield a three-dimensional (3D) location an a light point device.

7. The system of claim 6, further comprising:

processing means for triangulating three-dimensional positions of said light point devices based upon, for a given one of said light point devices, said one-dimensional information from one of said first and second imaging means and a portion of said one-dimensional information from the other of said first and second imaging means.

8. The system of claim 7, wherein said portion of said one-dimensional information from said other of said first and second means is an ID for one of said light point devices, and a position of said one of said light point devices along a respective one of said first or third reference axis, or along a respective one of said second or fourth reference axis, respectively.

9. The system of claim 7, wherein each of said light point devices is an originating source of light.

10. The system of claim 9, wherein each of said light point devices is a light emitting diode (LED).

11. The system of claim 10, wherein the LEDs emit infrared light.

12. The system of claim 10, wherein:

said processing means includes a transmitter for transmitting synch signals; and

said motion capture system further comprises module means for receiving said synchronization signals and for modulating outputs of each of said light point devices to provide a unique sequence of light pulses representing a unique identifier (ID) for each of said light point devices according to a predetermined cycle of modulation intervals based upon said synchronization signals.

13. The system of claim 12, wherein said transmitter and module means communicate via radio frequency (RF) radiation.

14. The system of claim 6, wherein each of said first and second cameras includes:

a first cylindrical collimating and telecentric (CCT) lens system oriented perpendicularly to said first or third reference axis, respectively;

a first linear photosensor, lying along said first or third reference axis, respectively, for imaging light transferred through said first CCT lens system;

a second CCT lens system oriented perpendicularly to said second or fourth reference axis, respectively; and

a second linear photosensor, lying along said second or fourth reference axis, respectively, for imaging light transferred through said second CCT lens system;

said first linear photosensor and said second linear photosensor together defining the plane segment of the camera.

15. The system of claim 14, wherein each of said CCT lens systems is formed of a cylindrical collimating (CC) sub-system superimposed on a cylindrical telecentric (CT) sub-system.

16. The system of claim 15, wherein:

said CT sub-system includes;

a planar surface having an aperture, said planar surface being oriented parallel to a plane in which lies the linear photosensor; and

a first convex cylindrical lens, an optical axis of which is perpendicularly to a long axis of said linear photosensor;

said CC sub-system includes;

a concave cylindrical lens, an axis of symmetry of a concave surface of said concave cylindrical lens being parallel to said long axis of said linear photosensor; and

a second convex cylindrical lens, an optical axis of which being parallel to said long axis of said linear photosensor;

a path of light through said CCT lens system passing through said aperture to said concave cylindrical lens, from said concave cylindrical lens to said first convex lens, from said first convex lens to said second convex lens, and from said second convex lens to said linear photosensor.

17. The system of claim 16, wherein:

said first convex cylindrical lens is a plano-convex cylindrical lens, a planar surface of which is oriented parallel, and is closest, to said plane in which lies the linear photosensor;

said concave cylindrical lens is a concave-plano cylindrical lens, a planar surface of which is oriented parallel, and is closest, to said plane in which lies the linear photosensor; and

said second convex cylindrical lens is plano-convex cylindrical lens, a planar surface of which is oriented parallel, and is closest, to said plane in which lies the linear photosensor.

18. The system of claim 6, wherein each of said first imaging means and said second imaging means includes:

a first microprocessor for digitally processing analogs of said light imaged on a respective one of said first or third reference axis, for filtering out information not corresponding to said light point devices, and for outputting one-dimensional information including a pixel location on said one of said first and third reference axis and a corresponding amplitude of said pixel location; and

a second microprocessor for digitally processing analogs of said light imaged on a respective one of said second or fourth reference axis, for filtering out information not corresponding to said light point devices, and for outputting one-dimensional information including a pixel location on said one of said second or fourth reference axis and a corresponding amplitude of said pixel location.

19. The system of claim 18, further comprising:

a third microprocessor for processing said one-dimensional information from said first microprocessor and said second microprocessor and for outputting said one-dimensional image information for each light point device in the form of a data packet including an ID, an indicator indicating the ID as being associated with one of said first or third reference axis, or one of said second or fourth second reference axis, respectively, and a location on the corresponding reference axis.

20. The system as in claim 19, wherein said first, second and third microprocessors are digital signal processors (DSPs).

21. The system of claim 18, wherein said first and second microprocessors are operable to determine a centroid for each light point device and to incorporate said centroid into said one-dimensional information output therefrom such that said one-dimensional information output therefrom has sub-pixel resolution.

22. The system of claim 18, wherein said analogs processed by said first microprocessor represent data independent of data represented by said analogs processed by said second microprocessor such that said first and second microprocessors are operable in parallel.

23. The system as in claim 18, wherein said first and second microprocessors are field programmable gate arrays (FPGAs) and said third microprocessor is a digital signal processor (DSP).

24. The system of claim 6, wherein said one-dimensional image information output by said first and second imaging means is predictions set a predetermined number, I, of frames in the future, said predictions being based upon one-dimensional information 1D_INFO of a current frame, H, namely 1D_INFO(H), such that said prediction is 1D_INFO(H+I).

25. The system of claim 24, wherein I=3.

26. The system of claim 6, wherein said first and second predetermined angles are 90°

.

27. The system of claim 6, wherein information corresponding to said light imaged by said first imaging means is independent of information corresponding to light images by said second imaging means such that said first imaging means and said second imaging means are operable in parallel.

28. The motion capture system of claim 1, wherein all of said light point devices provide said continuous non-zero output, respectively, simultaneously during said interval corresponding to said plurality of pulse periods.

29. A processing motion capture system comprising:

at least one light point cluster structure attached to an object to be tracked in a motion capture environment, each light point cluster structure including at least three light point devices fixed in a predetermined configuration, each light point device being operable to provide a continuous non-zero output over an interval corresponding to a plurality of pulse periods so as to define a plurality of continuous non-zero light pulses which vary in amplitude over said interval in a sequence that represents a unique plural bit digital identity (ID) of a light point device, each bit in said digital ID corresponding to an amplitude level within one of said pulse periods, said amplitude level changing at least once over said interval;

imaging means, synchronized with said light point devices, for imaging light in said motion capture environment along a first reference axis and a second reference axis oriented at a first predetermined angle with respect to said first reference axis, for filtering out information NOT corresponding to said light point devices, and for outputting a sequence of one-dimensional images of said pulses corresponding to substantially all of said light point devices, said images representing, information that includes the ID of at least one of said light point devices, a position of at least one of said light point devices along said first reference axis and a position of at least one of said light point devices along said second reference axis; and

processing means for recognizing the identities of, and for tracking positions of, substantially all of said light point devices based upon said sequence of one-dimensional images.

30. The system of claim 29, wherein said processing means is operable for triangulating three-dimensional positions of said light point devices based upon, for a given one of said light point devices, said one-dimensional information from said imaging means.

31. The system of claim 29, wherein said predetermined configuration is a triangle.

32. The system of claim 29, wherein each said light point cluster includes four light point devices and said predetermined configuration is a square.

33. The motion capture system of claim 29, wherein all of said light point devices provide said continuous non-zero output, respectively, simultaneously during said interval corresponding to said plurality of pulse periods.

34. A processing method of capturing motion of a plurality of light point devices attached to an object to be tracked in a motion capture environment, each of said light point devices being operable to provide a plurality of continuous non-zero light pulses which vary in amplitude over an interval corresponding to said plurality of light pulses in a sequence that defines a unique digital identity (ID) of a light point device, the method comprising:

a) imaging, substantially for and in synchrony with each of said plurality of light point devices, a continuous non-zero output over an interval corresponding to a plurality of pulse periods so as to define a plurality of continuous non-zero light pulses which vary in amplitude over said interval in a sequence that represents a unique plural bit digital identity in said motion capture environment along a first reference axis and a second reference axis oriented at a first predetermined angle with respect to said first reference axis, each bit in said digital ID corresponding to an amplitude level within one of said pulse periods, said amplitude level changing at least once over said interval;

b) filtering out information NOT corresponding to said light point devices;

c) outputting one-dimensional images of said pulses corresponding to substantially all of said light point devices, said images representing information that includes the ID of a light point device, a position of said light point device along said first reference axis and a position of said light point device along said second reference axis;

d) imaging, substantially for and in synchrony with each of said plurality of light point devices, a continuous non-zero output over an interval corresponding to a plurality of pulse periods so as to define a plurality of continuous non-zero light pulses which vary in amplitude over said interval in said sequence that represents a unique plural bit digital identity in said motion capture environment along a third reference axis and a fourth reference axis oriented at a second predetermined angle with respect to said third reference axis, each bit in said digital ID corresponding to an amplitude level within one of said pulse periods, said amplitude level changing at least once over said interval;

e) filtering out information NOT corresponding to said light point devices;

f) outputting one-dimensional images of said pulses corresponding to substantially all of said light point devices, said images representing information that includes the ID of a light point device, a position of said light point device along said third reference axis and a position of said light point device along said fourth reference axis; and

g) processing corresponding sequences of one-dimensional images from said steps c) and f) to yield IDs and three-dimensional (3D) locations of substantially all of said plurality of light point devices.

35. The method of claim 34, further comprising:

triangulating three-dimensional positions of said light point devices based upon, for a given one of said light point devices, said one-dimensional information from one of said steps c) and f) and a portion of said one-dimensional information from the other of said steps c) and f).

36. The method of claim 35, wherein said portion of said one-dimensional information from said other of said steps c) and f) is an ID for one of said light point devices, and a position of said one of said light point devices along a respective one of said first or third reference axis, or along a respective one of said second or fourth reference axis, respectively.

37. The method of claim 36, wherein information corresponding to said light imaged by said step a) is independent of information corresponding to light imaged by said step d) such that said steps b) and c) are processed in parallel to said steps e) and f).

38. The method of claim 34, wherein each of said steps c) and f) include:

g) outputting one-dimensional information including a pixel location on a respective one of said first and third reference axis and a corresponding amplitude of said pixel location; and

h) outputting one-dimensional information including a pixel location on a respective one of said second or fourth reference axis and a corresponding amplitude of said pixel location.

39. The method of claim 38, further comprising:

i) processing said one-dimensional information from said steps g) and h) to output said one-dimensional image information for each light point device in the form of a data packet including an ID, an indicator indicating the ID as being associated with one of said first or third reference axis, or one of said second or fourth second reference axis, respectively, and a location on the corresponding reference axis.

40. The method of claim 34, wherein said one-dimensional image information output by said steps c) and f) is a predictions set of a predetermined number, I, of frames in the future, said predictions being based upon one-dimensional information 1D_INFO of a current frame, H, namely 1D_INFO(H), such that said prediction is 1D_INFO(H+I).

41. The method of claim 40, wherein I=3.

42. The method of claim 34, wherein analogs imaged by said step a) represent data independent of data represented by analogs imaged by said step d) such that said steps b) and c) are operable in parallel.

43. The method of claim 34, further comprising:

transmitting synch signals; and

receiving said synchronization signals; and

modulating outputs of each of said light point devices to provide a unique sequence of light pulses representing a unique identifier (ID) for each of said light point devices according to a predetermined cycle of modulation intervals based upon said synchronization signals.

44. The method of claim 34, wherein said light pulses are modulated to represent a first logical value or a second logical value so that the digital identities are a sequence of N logical values, where N is a positive integer and 2^N total light point devices are identifiable, and wherein imaged frames are synchronized with said light pulses to recognize said digital identities by considering any set of N fields.

45. The method of claim 34, wherein all of said light point devices provide said continuous non-zero output, respectively, simultaneously during said interval corresponding to said plurality of pulse periods.

46. A motion capture method comprising:

imaging a plurality of light point devices attached to an object to be tracked in a motion capture environment, each being operable to provide a continuous non-zero output over an interval corresponding to a plurality of pulse periods so as to define a plurality of continuous non-zero light pulses which vary in terms of an optical characteristic over said interval in a sequence that represents a unique plural bit digital identity (ID) of a light point device, each bit in said digital ID corresponding to a state of said optical characteristic within one of said pulse periods, said state of said optical characteristic changing at least once over said interval;

capturing a sequence of images of said pulses corresponding to substantially all of said plurality of light point devices; and

recognizing the identities of, and tracking the positions of, substantially all of said plurality of light point devices based upon said light pulses appearing within said sequence of images, respectively.

47. The method of claim 46, wherein all of said light point devices provide said continuous non-zero output, respectively, simultaneously during said interval corresponding to said plurality of pulse periods.

48. A motion capture system comprising:

a plurality of light point devices attached to an object to be tracked in a motion capture environment, each being operable to provide a continuous non-zero output over an interval corresponding to a plurality of pulse periods so as to define a plurality of continuous non-zero light pulses which vary in terms of an optical characteristic over said interval in a sequence that represents a unique plural bit digital identity (ID) of a light point device, each bit in said digital ID corresponding to a state of said optical characteristic within one of said pulse periods, said state of said optical characteristic changing at least once over said interval;

means for capturing a sequence of images of said pulses corresponding substantially to all of said plurality of light point devices; and

processing circuitry to recognize the identities of, and to track positions of, substantially all of said plurality of light point devices based upon said light pulses appearing within said sequence of images, respectively.

49. The system of claim 48, wherein said variable optical characteristic is pulse amplitude.

50. The motion capture system of claim 48, wherein all of said light point devices provide said continuous non-zero output, respectively, simultaneously during said interval corresponding to said plurality of pulse periods.