DETERMINING X,Y,Z,T BIOMECHANICS OF MOVING ACTOR WITH MULTIPLE CAMERAS

• US 20180315202A1
• Filed: 07/05/2018
• Published: 11/01/2018
• Est. Priority Date: 04/15/2015
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportsmedia Technology Corporation
• Inventors: Perry, Mark, Spivak, Joshua, Zander, Ryan A., Goldbeck, Graham Wei-Feng, Painter, James G.
• Granted Patent: US 10,706,566 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A plurality of high speed tracking cameras is pointed towards a routine hovering area of an in-the-field sports participant who routinely hovers about that area. Spots within the hovering area are registered relative to a predetermined multi-dimensional coordinates reference frame (e.g., Xw, Yw, Zw, Tw) such that two-dimensional coordinates of 2D images captured by the high speed tracking cameras can be converted to multi-dimensional coordinates of the reference frame. A body part recognizing unit recognizes 2D locations of a specific body part in the 2D captured images and a mapping unit maps them into the multi-dimensional coordinates of the reference frame. A multi-dimensional curve generator then generates a multi-dimensional motion curve describing motion of the body part based on the mapped coordinates (e.g., Xw, Yw, Zw, Tw). The generated multi-dimensional motion curve is used to discover cross correlations between play action motions of the in-the-field sports participant and real-world sports results.

Claims

1. A system for determining multi-dimensional (mD) performance attributes of a sports participant comprising:

at least three high speed cameras configured for network communication with at least one server computer, at least one local computing device, and at least one processing facility;

wherein the at least three high speed cameras are operable to capture two-dimensional (2D) images of a sports participant;

wherein the at least one processing facility is operable to;

store the 2D images of the sports participant in a database;

develop a mD motion curve for at least two identifiable body parts of the sports participant;

identify camera frames and/or scan lines from the at least two of the at least three high speed cameras associated with spatial points on the mD motion curve; and

display the camera frames and/or the scan lines on the at least one server computer and/or the at least one local computing device.

2. The system of claim 1, wherein the at least one processing facility is operable to develop the mD motion curve using the respective temporal segment lengths of the 2D images of the sports participant.

3. The system of claim 1, wherein the at least one processing facility is operable to intertwine and temporally dispose the 2D images of the sports participant along a common timing reference and discard temporally adjacent 2D images.

4. The system of claim 1, wherein the at least one server computer and/or the at least one local computing device is operable to display camera frames corresponding to an identified spatial point on the mD motion curve.

5. The system of claim 1, wherein the at least three high speed cameras are operable to sense electromagnetic radiation outside of the range of visible light to facilitate automated recognition of skeletal features, thermal hot spots on a body of the sports participant, and/or locations of distinguishing clothing fibers on a uniform of the sports participant.

6. The system of claim 1, wherein the database stores data relating to the sports participant and data including sport event results, surrounding field effects, mental effects, and/or in-the-field biometric attributes.

7. The system of claim 6, wherein the in-the-field biometric attributes include a heart rate, a breathing rate, a perspiration level, a blood pressure, a galvanic skin response, a topical temperature, and/or cranial electrical activity of the sports participant recorded at the same time as a play action activity.

8. The system of claim 1, wherein the at least one processing facility is operable to transform the 2D images of the sports participant into four-dimensional (4D) player-biomechanical models.

9. A system for determining multi-dimensional (mD) performance attributes of a sports participant comprising:

at least three high speed cameras configured for network communication with at least one server computer, at least one local computing device, and at least one processing facility;

wherein the at least three high speed cameras are operable to capture two-dimensional (2D) images of a sports participant;

wherein the at least one processing facility is operable to;

store the 2D images of the sports participant in a database;

develop a mD motion curve for the sports participant; and

construct a four-dimensional (4D) model of the sports participant based on the 2D images of the sports participant.

10. The system of claim 9, wherein the at least three high speed cameras operate semi-asynchronously and/or asynchronously.

11. The system of claim 9, wherein the at least one processing facility is operable to intertwine and temporally dispose the 2D images of the sports participant along a common timing reference and discard temporally adjacent 2D images.

12. The system of claim 11, wherein the at least one processing facility is operable to revise the intertwined 2D images such that the intertwined 2D images conform to the mD motion curve developed for the sports participant.

13. The system of claim 9, wherein the at least one processing facility develops the mD motion curve within a mD space having at least a time (Tw) axis and three spatial and orthogonal coordinate axes (Xw, Yw, Zw).

14. The system of claim 9, wherein intrinsic and extrinsic parameters for each of the at least three high speed cameras are provided to the at least one processing facility to construct the 4D model of the sports participant.

15. The system of claim 9, wherein the at least one processing facility stores cross associating points of motion relating to environmental conditions and sport event results with the 4D model of the sports participant in the database.

16. A system for determining multi-dimensional (mD) performance attributes of a sports participant comprising:

at least three high speed cameras configured for network communication with at least one server computer, at least one local computing device, and at least one processing facility;

wherein the at least three high speed cameras are operable to capture two-dimensional (2D) images of a sports participant;

wherein the at least one processing facility is operable to;

determine a common timing reference for placing the 2D images of the sports participant;

calculate respective temporal segment lengths for the 2D images of the sports participant; and

intertwine the 2D images of the sports participant such that the 2D images of the sports participant are approximately disposed in a temporal sense along the common timing reference.

17. The system of claim 16, wherein the at least one processing facility is operable to develop a mD motion curve for identifiable body parts of the sports participant.

18. The system of claim 17, wherein the at least one processing facility is operable to identify camera frames and/or scan lines from at least two of the at least three high speed cameras that correspond with spatial points along the respective mD motion curve.

19. The system of claim 17, wherein the at least one processing facility is operable to fit the mD motion curve with curve fit optimization techniques relative to the 2D images captured by the at least three high speed cameras where a subset of the 2D images of the at least three high speed cameras is given greater weight due to closeness and/or a better point of view (POV).

20. The system of claim 16, wherein the at least three high speed cameras are configured to automatically determine the start and/or the end of a play action activity and generate a unique ID label for the play action activity.