System and method for tracking

• US 9,747,697 B2
• Filed: 03/16/2016
• Issued: 08/29/2017
• Est. Priority Date: 08/31/2010
• Status: Active Grant

• Current Assignee: CAST Group of Companies, Inc.
• Original Assignee: CAST Group of Companies, Inc.
• Inventors: Densham, Gilray, Eichel, Justin
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): PEREZ FUENTES, LUIS M
• Application Number:
  US15/071,401
• Publication Number:
  US 20160196667A1
• Time in Patent Office: 531 Days
• Field of Search: 348169
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/586 from multiple light sources, e.g. photometric stereo
• CPC Class Codes: H04N 23/15 Image signal generation with circuitry for avoiding or correcting image misregistration
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 9/315 Modulator illumination systems general applications of lighting devices F21V; general optical systems G02B; lamp houses for projectors not peculiar to the presence of an electronic spatial light modulator G03B21/20
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

Systems and methods are provided for generating calibration information for a media projector. The method includes tracking at least position of a tracking apparatus that can be positioned on a surface. The media projector shines a test spot on the surface, and the test spot corresponds to a known pixel coordinate of the media projector. The system includes a computing device in communication with at least two cameras, wherein each of the cameras are able to capture images of one or more light sources attached to an object. The computing device determines the object'"'"'s position by comparing images of the light sources and generates an output comprising the real-world position of the object. This real-world position is mapped to the known pixel coordinate of the media projector.

Claims

1. A kit of parts for generating calibration information for a media projector, comprising:

a media projector that shines at least one test spot on a surface, the test spot corresponding to a known pixel coordinate of the media projector;

a tracking apparatus able to be positioned on the surface at the test spot, the tracking apparatus comprising;

one or more light sources that strobe according to a strobe pattern associated with an object ID; and

a wireless radio for communicating with a computing device;

at least two cameras, each of the cameras able to capture images of the one or more light sources; and

a computing device able to obtain at least the images from the at least two cameras, the computing device configured to at least;

analyze the images to determine the strobe pattern of the one or more light sources;

identify the object ID based on the strobe pattern;

determine a real-world X,Y,Z-coordinate position of the one or more light sources by at least determining a pixel location of the one or more light sources in the images;

generate a mapping between the known pixel coordinate of the test spot and the real-world position of the one or more light sources; and

output the mapping as calibration information for the media projector.

2. The kit of parts of claim 1 wherein the media projector shines multiple tests spots on the surface, each one of the test spots corresponding to a respective known pixel coordinate of the media projector;

the tracking apparatus able to be moved to different positions on the surface at the multiple test spots;

the at least two cameras capturing images of the one of more light sources of the tracking apparatus at the different positions corresponding to the respective test spots; and

the computing device is further configured to obtain the images from the at least two cameras to determine real-world X,Y,Z-coordinate positions for the tracking apparatus corresponding to the respective test spots, generate a mapping matrix comprising mappings between the known pixel coordinates of the respective tests spots and the respective real-world X,Y,Z-coordinate positions, and output the mapping matrix as calibration information for the media projector.

3. The kit of parts of claim 1 wherein the tracking apparatus further comprises an inertial measurement unit, and the wireless radio transmits at least measurements obtained from the inertial measurement unit and the object ID.

4. The kit of parts of claim 3 wherein the computing device further comprises a state machine calculator for computing a current state of the object, the current state comprising the position and angular orientation of the object.

5. The kit of parts of claim 3 wherein the inertial measurement unit measures at least inertial acceleration.

6. The kit of parts of claim 3 wherein the inertial measurement unit measures at least angular orientation, which comprises roll, pitch, and yaw.

7. The kit of parts of claim 1 wherein the computing device is further configured to ignore a light source that does not strobe when analyzing the images.

8. The kit of parts of claim 3 wherein the measurements include inertial acceleration data, and the computing device is further configured to:

determine an acceleration and a position of a given light source of the one or more light sources by comparing a series of images of the given light source; and

, after determining that the acceleration determined from the series of images is approximately equal to the inertial acceleration data, the computing device associating the object ID obtained from the receiver with the position of the given light source.

9. The kit of parts of claim 1 wherein the computing device is further configured to:

determine if a subsequent position of the one or more light sources in subsequent images is within an expected vicinity of the position, and if so, associating the object ID with the subsequent position.

10. The kit of parts of claim 3 wherein the measurements comprise inertial acceleration data, and the computing device is further configured to:

detect that only one of the cameras is able to detect the one or more light sources, or none of the cameras are able to detect the one or more light sources;

identify a last known position of the object as determined from the images; and

determining a new position of the object by combining the inertial acceleration data with the last known position.

11. The kit of parts of claim 2 comprising multiple tracking apparatuses that are able to be positioned at different ones of the multiple test spots.

12. A method for generating calibration information for a media projector, comprising:

a media projector shining at least one test spot on a surface, the test spot corresponding to a known pixel coordinate of the media projector;

positioning a tracking apparatus on the surface at the test spot, the tracking apparatus comprising;

one or more light sources that strobe according to a strobe pattern associated with an object ID; and

a wireless radio for communicating with a computing device;

capturing images, via at least two cameras, of the one or more light sources;

the computing device obtaining at least the images from the at least two cameras;

analyzing the images to determine the strobe pattern of the one or more light sources;

identifying the object ID based on the strobe pattern;

determining a real-world X,Y,Z-coordinate position of the one or more light sources by at least determining a pixel location of the one or more light sources in the images;

generating a mapping between the known pixel coordinate of the test spot and the real-world position of the one or more light sources; and

outputting the mapping as calibration information for the media projector.

13. The method of claim 12 wherein the tracking apparatus comprises an inertial measurement unit, and the method further comprises the wireless radio transmitting at least measurements obtained from the inertial measurement unit and the object ID.

14. The method of claim 13 wherein the computing device uses a state machine calculator to compute a current state of the object, the current state comprising the position and angular orientation of the object.

15. The method of claim 13 wherein the inertial measurement unit measures at least inertial acceleration.

16. The method of claim 13 wherein the inertial measurement unit measures at least angular orientation, which comprises roll, pitch, and yaw.

17. The method of claim 12 further comprising the computing device ignoring a light source that does not strobe when analyzing the images.

18. The method of claim 13 wherein the measurements include inertial acceleration data, and the method further comprises the computing device determining an acceleration and a position of a given light source of the one or more light sources by comparing a series of images of the given light source;

and, after determining that the acceleration determined from the series of images is approximately equal to the inertial acceleration data, the computing device associating the object ID obtained from the receiver with the position of the given light source.

19. The method of claim 12 wherein the computing device is further configured to:

determine if a subsequent position of the one or more light sources in subsequent images is within an expected vicinity of the position, and if so, associating the object ID with the subsequent position.

20. The method of claim 13 wherein the measurements comprise inertial acceleration data, and the method further comprises the computing device detecting that only one of the cameras is able to detect the one or more light sources, or none of the cameras are able to detect the one or more light sources;

identifying a last known position of the object as determined from the images; and

determining a new position of the object by combining the inertial acceleration data with the last known position.

21. The method of claim 12 wherein multiple tracking apparatuses are positioned at different ones of the multiple test spots.

22. A system for generating calibration information for a media projector, comprising:

a media projector that shines at least one test spot on a surface, the test spot corresponding to a known pixel coordinate of the media projector;

a tracking apparatus able to be positioned on the surface at the test spot, the tracking apparatus comprising;

one or more light sources that strobe according to a strobe pattern associated with an object ID; and

a wireless radio for communicating with a computing device;

at least two cameras, each of the cameras able to capture images of the one or more light sources; and

a computing device able to obtain at least the images from the at least two cameras, the computing device configured to at least;

analyze the images to determine the strobe pattern of the one or more light sources;

identify the object ID based on the strobe pattern;

determine a real-world X,Y,Z-coordinate position of the one or more light sources by at least determining a pixel location of the one or more light sources in the images;

generate a mapping between the known pixel coordinate of the test spot and the real-world position of the one or more light sources; and

output the mapping as calibration information for the media projector.

23. The system of claim 22 wherein the media projector shines multiple tests spots on the surface, each one of the test spots corresponding to a respective known pixel coordinate of the media projector;

the tracking apparatus is able to be moved to different positions on the surface at the multiple test spots;

the at least two cameras capture images of the one of more light sources of the tracking apparatus at the different positions corresponding to the respective test spots; and

the computing device is further configured to obtain the images from the at least two cameras to determine real-world X,Y,Z-coordinate positions for the tracking apparatus corresponding to the respective test spots, generate a mapping matrix comprising mappings between the known pixel coordinates of the respective tests spots and the respective real-world X,Y,Z-coordinate positions, and output the mapping matrix as calibration information for the media projector.

24. The system of claim 22 wherein the tracking apparatus further comprises an inertial measurement unit, and the wireless radio transmits at least measurements obtained from the inertial measurement unit and the object ID.

25. The system of claim 24 wherein the computing device further comprises a state machine calculator for computing a current state of the object, the current state comprising the position and angular orientation of the object.

26. The system of claim 24 wherein the inertial measurement unit measures at least inertial acceleration.

27. The system of claim 24 wherein the inertial measurement unit measures at least angular orientation, which comprises roll, pitch, and yaw.

28. The system of claim 22 wherein the computing device is further configured to ignore a light source that does not strobe when analyzing the images.

29. The system of claim 24 wherein the measurements include inertial acceleration data, and the computing device is further configured to:

determine an acceleration and a position of a given light source of the one or more light sources by comparing a series of images of the given light source; and

, after determining that the acceleration determined from the series of images is approximately equal to the inertial acceleration data, the computing device associating the object ID obtained from the receiver with the position of the given light source.

30. The system of claim 22 wherein the computing device is further configured to:

determine if a subsequent position of the one or more light sources in subsequent images is within an expected vicinity of the position, and if so, associating the object ID with the subsequent position.

31. The system of claim 24 wherein the measurements comprise inertial acceleration data, and the computing device is further configured to:

detect that only one of the cameras is able to detect the one or more light sources, or none of the cameras are able to detect the one or more light sources;

identify a last known position of the object as determined from the images; and

determining a new position of the object by combining the inertial acceleration data with the last known position.

32. The system of claim 23 comprising multiple tracking apparatuses that are able to be positioned at different ones of the multiple test spots.