AUTOMATED COLLECTIVE CAMERA CALIBRATION FOR MOTION CAPTURE

• US 20110025853A1
• Filed: 07/30/2010
• Published: 02/03/2011
• Est. Priority Date: 07/31/2009
• Status: Active Grant

• Current Assignee: Naturalpoint Incorporated (Leyard Optoelectronic Co., Ltd.)
• Original Assignee: Naturalpoint Incorporated (Leyard Optoelectronic Co., Ltd.)
• Inventors: Richardson, James D.
• Granted Patent: US 9,019,349 B2
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A motion-capture system is provided. The motion-capture system includes a host computing system and a plurality of motion-capture cameras that are operatively coupled with the host computing system. Each of the motion-capture cameras is disposed in a different location and orientation relative to a motion-capture space and includes a marker-tracking optical filter to provide a marker-tracking mode and thereby relatively enhance light from markers on a moving body in the motion-capture space. One or more of the motion-capture cameras is remotely controllable to selectively interchange the marker-tracking optical filter with a scene-view optical component, so as to selectively transition the motion-capture camera between the marker-tracking mode and a scene mode, in which light from the markers is less enhanced than in the marker-tracking mode.

Claims

1. A motion-capture system, comprising:

a host computing system; and

a plurality of motion-capture cameras operatively coupled with the host computing system, where each of the motion-capture cameras is disposed in a different location and orientation relative to a motion-capture space and includes a marker-tracking optical filter to provide a marker-tracking mode and thereby relatively enhance light from markers on a moving body in the motion-capture space, and where one or more of the motion-capture cameras is remotely controllable to selectively interchange the marker-tracking optical filter with a scene-view optical component, so as to selectively transition the motion-capture camera between the marker-tracking mode and a scene mode, in which light from the markers is less enhanced than in the marker-tracking mode.

2. The system of claim 1, where each of the plurality of motion-capture cameras includes a light source to provide illumination to passive markers.

3. The system of claim 2, where the light source is infrared.

4. The system of claim 3, where for each of the plurality of motion-capture cameras, the marker-tracking optical filter relatively enhances transmission of infrared light, and where for the one or more of the motion-capture cameras that are selectively transitionable between the marker-tracking mode and the scene mode, the scene-view optical component is an infrared-cut filter.

5. The system of claim 1, where the marker-tracking mode for each of the plurality of motion-capture cameras is operable to track markers via infrared light, and where a polychromatic lens is provided on the one or more of the motion-capture cameras that are selectively transitionable between the marker-tracking mode and the scene mode.

6. The system of claim 1, where for each of the plurality of motion-capture cameras, the marker-tracking optical filter and marker-tracking mode are configured to provide tracking of active markers.

7. The system of claim 1, where for the one or more of the motion-capture cameras that are selectively transitionable between the marker-tracking mode and the scene mode are configured to, when in the scene mode, compress image data prior to transmitting such image data to the host computing system.

8. The system of claim 1, where for the one or more of the motion-capture cameras that are selectively transitionable between the marker-tracking mode and the scene mode, the marker-tracking optical filter and the scene view optical component are configured so that focus in the motion-capture camera is substantially unchanged as a result of transitions between the marker-tracking mode and the scene mode.

9. The system of claim 1, where the host computing system is configured to substantially simultaneously calibrate the plurality of motion-capture cameras in response to the plurality of motion-capture cameras tracking movement of a reference object in the motion-capture space.

10. The system of claim 9, where during said substantially simultaneous calibration, one or more of the motion-capture cameras that are selectively transitionable between the marker-tracking mode and the scene mode is/are placed in the marker-tracking mode.

11. The system of claim 9, where subsequent to said substantially simultaneous calibration, one or more of the motion-capture cameras that are selectively transitionable between the marker-tracking mode and the scene mode is/are placed in the scene mode.

12. The system of claim 9, where said substantially simultaneous calibration further includes presenting a marker-defined reference plane to the plurality of motion-capture cameras.

13. The system of claim 9, where subsequent to said substantially simultaneous calibration, the host computing system is configured to create a composite view including a visual representation of the markers that are relatively enhanced by light from the marker-tracking optical filter that is overlaid on at least a portion of a scene view that includes the moving body to thereby align the visual representation of the markers that are relatively enhanced by the marker-tracking optical filter with the moving body in the composite view.

14. The system of claim 13, where the host computing system is configured to perform a video analysis operation on the composite view to track a position of a non-marker object in the composite view relative to a position of the visual representation of the markers.

15. The system of claim 13, where tracking the position of the non-marker object includes tracking a movement of the non-marker object subsequent to interaction with the moving body based on a movement of the visual representation of the markers.

16. The system of claim 15, where the composite view includes a virtual object overlaid at a position that tracks the movement of the non-marker object.

17. A motion-capture camera apparatus comprising:

an image sensor configured to receive incident light from a motion-capture space and thereby generate an image of the motion-capture space;

a marker-tracking optical filter;

a scene-view optical component;

a switching mechanism actuatable to positionally interchange the marker-tracking optical filter and the scene-view optical component relative to the image sensor; and

a controller, operable in a marker-tracking mode, to control the switching mechanism so that the marker-tracking optical filter is positioned relative to the image sensor to relatively enhance light received at the image sensor from a plurality of markers on a moving body in the motion-capture space, and operable in a scene mode, to control the switching mechanism to positionally interchange the marker-tracking optical filter and the scene-view optical component so that the scene-view optical component is positioned relative to the image sensor to enhance light received at the image sensor from the plurality of markers to a lesser extent than in the marker-tracking mode.

18. The apparatus of claim 17, where the plurality of markers each include retro-reflective material, and the apparatus further comprises:

a light source configured to outwardly direct light in a direction that is substantially coaxially aligned with an axis extending perpendicularly outward from the image sensor, to enhance light reflected by the retro-reflective material of each of the plurality of markers toward the image sensor.

19. The apparatus of claim 17, where the image sensor detects light in a range of visible light to infrared light, and the apparatus further comprises:

a polychromatic lens to focus light received from the motion-capture space that is substantially anywhere in the range toward the image sensor.

20. The apparatus of claim 17, where the marker-tracking optical filter includes an infrared-pass filter and the scene-view optical component includes an infrared-cut filter.

21. The apparatus of claim 17, where the marker-tracking optical filter and the scene-view optical component are configured so that focus in the motion-capture camera is substantially unchanged as a result of positional transitions between the marker-tracking mode and the scene mode.

22. The apparatus of claim 17, where the controller adjusts an operating parameter of the motion-capture camera apparatus in response to switching between the marker-tracking mode and the scene mode to thereby enhance light during the marker-tracking mode and enhance light to a lesser extent than in the marker-tracking mode during the scene mode.

23. The apparatus of claim 17, further comprising:

a communication link; and

the controller further includes a video compression engine to compress video frames of images generated by the image sensor when operating in the scene mode to thereby reduce a data rate for transmission via the communication link, and where video frames of images generated by the image sensor when operating in the marker-tracking mode are not compressed by the video compression engine.

24. A method for remotely controlling a plurality of motion-capture cameras each disposed in a different location and orientation in a motion-capture space, each of the plurality of motion-capture cameras being operable in a marker-tracking mode where each of the plurality of motion-capture cameras relatively enhances light from a plurality of markers on a moving body in the motion-capture space via use of a marker-tracking optical filter, and where one or more motion-capture cameras of the plurality of motion-capture cameras is remotely controllable to selectively positionally interchange the marker-tracking optical filter with a scene-view optical component, so as to selectively transition the motion-capture camera between the marker-tracking mode and a scene mode, where the one or more motion-capture cameras enhances light from the plurality of markers to a lesser extent than in the marker-tracking mode, the method comprising:

setting the plurality of motion-capture cameras to the marker-tracking mode;

receiving substantially simultaneously, from each of the plurality of motion-capture cameras, position data for a reference object as the reference object moves in the motion-capture space, the position data forming a calibration data set;

substantially simultaneously calibrating the plurality of motion-capture cameras based on a solution to the calibration data set calculated according to a minimization technique;

setting one or more, but less than all, of the motion-capture cameras to the scene mode;

creating a marker-tracking view based on marker-tracking data received from motion-capture cameras that are set to the marker-tracking mode, the marker-tracking view including a visual representation of the plurality of markers that are relatively enhanced by the marker-tracking optical filter;

creating a scene view based on scene data received from the one or more motion-capture cameras that are set to the scene mode, the scene view including the moving body; and

creating a composite view where at least a portion of the marker-tracking view is overlaid on at least a portion of the scene view based on the solution to the calibration data set to thereby align the visual representation of the plurality of markers with the moving body.

25. The method of claim 24, where the video data is compressed by a compression engine included in the one or more motion-capture cameras that are set to the scene mode.

26. The method of claim 24, where the position data further includes information for a marker-defined reference plane;

and

substantially simultaneously calibrating includes the plurality of motion-capture cameras sharing a frame of reference to a world coordinate system.

27. The method of claim 24, where the composite view includes a virtual object overlaid on the moving body based on a position of the visual representation of the plurality of markers.

28. The method of claim 27, further comprising:

changing, in the composite view, a position of one or more of the markers of the visual representation of the plurality of markers in response to the virtual object being out of alignment with the moving body to thereby align the virtual object with the moving body.

29. The method of claim 24, further comprising:

performing a video analysis operation on the composite view to track a position of a non-marker object in the composite view relative to a position of the plurality of markers as indicated by the marker-tracking data.

30. The method of claim 29, where tracking the position of the non-marker object includes determining a movement of the non-marker object subsequent to interaction with the moving body based on the marker-tracking data.

31. The method of claim 30, where the composite view includes a virtual object overlaid at a position that tracks the movement of the non-marker object