Camera-based tracking and position determination for sporting events using event information and intelligence data extracted in real-time from position information

• US 9,185,361 B2
• Filed: 07/29/2009
• Issued: 11/10/2015
• Est. Priority Date: 07/29/2008
• Status: Active Grant

• Current Assignee: Gerald Curry
• Original Assignee: Gerald Curry
• Inventors: Curry, Gerald
• Primary Examiner(s): Murray, Daniel C
• Application Number:
  US12/511,918
• Publication Number:
  US 20100026809A1
• Time in Patent Office: 2,295 Days
• Field of Search: 348/148, 348/157, 348/143, 348/159, 348/169, 348/162, 348/163, 348/164
• US Class Current: 1/1
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Position information of equipment at an event, such as a ball, one or more players, or other items in a game or sporting event, is used in selecting camera, camera shot type, camera angle, audio signals, and/or other output data for providing a multimedia presentation of the event to a viewer. The position information is used to determine the desired viewer perspective. A network of feedback robotically controlled Pan-Tilt-Zoom (PTZ), manually controlled cameras and stationary cameras work together with interpolation techniques to create a 2D video signal. The position information may also be used to access gaming rules and assist officiating of the event. The position information may be obtained through a transceiver(s), accelerometer(s), transponder(s), and/or RADAR detectable element(s) fitted into the ball, apparel or equipment of players, the players themselves, or other playing equipment associated with the game or sporting event. Other positioning methods that can be used include infrared video-based tracking systems, SONAR positioning system(s), LIDAR positioning systems, and digital signal processing (DSP) image processing techniques such as triangulation.

Claims

1. A method of computer controlled camera selection for viewing, the method comprising:

receiving camera outputs at a processor from a plurality of cameras, each of which is configured to provide a respective camera output showing respective field of view coverage of an ongoing event;

receiving position information at the processor, wherein the position information is associated with equipment at the ongoing event such that the received position information provides real time position information of the equipment;

storing multiple event states into memory from which the data may be read by the processor;

wherein each event state is programmed responsive to event information and intelligence data extracted from the ongoing event and/or unique logical combinations of event information and intelligence data extracted from the ongoing event in real time;

wherein the event information and intelligence data includes event participant assignment execution outcome and results, event participant decision making outcome and results, and event participant interactions with other participants;

wherein the event information and intelligence data are extracted in real time responsive to the equipment position information inputs, conditional logic responsive to the equipment position information inputs, and/or other logical combinations of equipment position information inputs;

wherein the ongoing event has a current event state, where the current event state is identified in real time as one of the preprogrammed multiple event states;

wherein the current event state is updated in real time and may involve a transition from a previous event state that is different from the current event state;

wherein the current event state transition is responsive to changes in event information and intelligence data extracted from the received position information in real time;

selecting a camera output from among the received plurality of camera outputs in response to the current event state such that the selected camera output provides its respective field of view coverage of the ongoing event and producing a processor signal comprising the selected camera output.

2. The method of claim 1, further comprising upon selecting the selected camera output in response to the current event state, combining the selected camera output with a camera output different from the selected camera output.

3. The method of claim 1, further comprising switching from outputting video frames of the selected camera output at a first rate to outputting video frames of the selected camera output at a second rate in response to the position information, at least one of the first rate or the second rate resulting in slow motion video.

4. The method of claim 1, further comprising combining audio information with the selected camera output, wherein the audio information is chosen independently of the position information.

5. The method of claim 1, further comprising combining audio information with the selected camera output, wherein the audio information is chosen in response to the position information.

6. The method of claim 1, wherein at least one camera of the plurality of cameras is configured to provide three dimensional (3D) field of view coverage of the ongoing event.

7. The method of claim 1, wherein at least two of the cameras are configured at multiple vertical distances above the ongoing event.

8. The method of claim 7, wherein the at least two of the cameras are spaced contiguously at the multiple vertical distances.

9. The method of claim 1, wherein at least two of the cameras are configured at multiple radial positions arranged radially around the ongoing event substantially in the same plane.

10. The method of claim 9, wherein the at least two of the cameras are spaced contiguously at the multiple radial positions.

11. The method of claim 1, wherein at least one camera from the plurality of cameras comprises a panoramic view camera.

12. The method of claim 1, further comprising controlling at least one function of at least one camera of the plurality of cameras in response to the position information, the at least one function including at least one of focus, zoom, pan, or tilt.

13. The method of claim 1, further comprising controlling at least one camera of the plurality of cameras to change location relative to the ongoing event in response to the position information.

14. The method of claim 1, further including receiving a manual selection of camera output from among the plurality of cameras that suspends the processor from selecting a camera output in response to the current event state.

15. The method of claim 14, further including receiving a selection for the processor to resume selecting a camera output in response to the current event state.

16. The method of claim 1, wherein the processor signal comprises camera outputs from more than one camera of the plurality of camera outputs, the method further comprising:

receiving a manual selection of a single camera output from among the plurality of camera outputs of the processor signal;

suspending selecting a camera output in response to the current event state.

17. The method of claim 1, wherein the processor signal comprises camera outputs from more than one camera of the plurality of camera outputs, the method further comprising:

transmitting the processor signal to an end user viewing system for display at the end user viewing system, wherein the end user viewing system includes a user interface such that a manual selection input to the user interface identifies a single camera output from among the plurality of camera outputs of the transmitted processor signal for display at the end user viewing system.

18. The method of claim 1, wherein the selected camera output is combined with an augmented reality presentation.

19. The method of claim 1, wherein the equipment at the ongoing event includes at least one of a player, ball, puck, gaming projectile, clothing, apparel, jersey, shorts, pants, sox, shoes, hat, headband, sweatband, armband, helmet, shoulder pads, elbow pads, or shin guards.

20. The method of claim 1, wherein at least one of the plurality of cameras is mounted on a camera pole apparatus.

21. The method of claim 1, wherein one or more mirrors are used to produce at least one of the camera outputs.

22. The method of claim 1, wherein the position information includes rate of change of position information of the equipment.

23. The method of claim 1, wherein a computer generated three dimensional environment is created using the position information, wherein the position information is associated with equipment at the ongoing event.

24. The method of claim 1, wherein the selected camera output is a different camera output from a previously selected camera output that was selected according to the previous event state.

25. The method of claim 1, wherein the current event state is determined in response to the received position information and in accordance with a predetermined three-dimensional space occupied by the equipment.

26. The method of claim 25, wherein the equipment includes two or more sensors for transmission of position data, with which the three-dimensional positional space presently occupied by the equipment at the ongoing event may be determined in real time.

27. The method of claim 25, wherein the equipment includes two or more sensors for transmission of position data, with which the three-dimensional positional space presently occupied by the ball and/or other gaming projectile may be determined in real time.

28. The method of claim 1, wherein the received position information is transmitted from equipment at the ongoing event such that the received position information provides real time position information of the equipment from which the position information was transmitted.

29. The method of claim 28, wherein the position information is transmitted as a signal comprising at least one of RADAR, LIDAR, SONAR, or infrared signals.

30. A system for computer controlled camera selection for viewing, the system comprising:

a processor that is configured to receive camera outputs from a plurality of cameras, each of which is configured to provide a respective camera output showing respective field of view coverage of an ongoing event, and that is configured to receive position information, wherein the position information is associated with equipment at the ongoing event such that the received position information provides real time position information of the equipment, and is configured to store multiple event states into memory from which the data may be read by the processor;

wherein each event state is programmed responsive to event information and intelligence data extracted from the ongoing event and/or unique logical combinations of event information and intelligence data extracted from the ongoing event in real time;

wherein the event information and intelligence data includes event participant assignment execution outcome and results, event participant decision making outcome and results, and event participant interactions with other participants;

wherein the event information and intelligence data are extracted in real time responsive to the equipment position information inputs, conditional logic responsive to the equipment position information inputs, and/or other logical combinations of equipment position information inputs;

wherein the ongoing event has a current event state, where the current event state is identified in real time as one of the preprogrammed multiple event states;

wherein the current event state is updated in real time and may involve a transition from a previous event state that is different from the current event state;

wherein the current event state transition is responsive to changes in event information and intelligence data extracted from the received position information in real time;

wherein the processor selects a camera output from among the received plurality of camera outputs in response to the current event state such that the selected camera output provides its respective field of view coverage of the ongoing event and producing a processor signal comprising the selected camera output.

31. The system of claim 30, wherein the processor is further configured to, upon selecting the selected camera output in response to the current event state, combining the selected camera output with a camera output different from the selected camera output.

32. The system of claim 30, wherein the processor is further configured to switch from outputting video frames of the selected camera output at a first rate to outputting video frames of the selected camera output at a second rate in response to the position information, at least one of the first rate or the second rate resulting in slow motion video.

33. The system of claim 30, wherein the processor is further configured to combine audio information with the selected camera output, wherein the audio information is chosen independently of the position information.

34. The system of claim 30, wherein the processor is further configured to combine audio information with the selected camera output, wherein the audio information is chosen in response to the position information.

35. The system of claim 30, wherein at least one camera of the plurality of cameras is configured to provide three dimensional (3D) field of view coverage of the ongoing event.

36. The system of claim 30, wherein at least two of the cameras are configured at multiple vertical distances above the ongoing event.

37. The system of claim 36, wherein the at least two of the cameras are spaced contiguously at the multiple vertical distances.

38. The system of claim 30, wherein at least two of the cameras are configured at multiple radial positions arranged radially around the ongoing event substantially in the same plane.

39. The system of claim 38, wherein the at least two of the cameras are spaced contiguously at the multiple radial positions.

40. The system of claim 30, wherein at least one camera from the plurality of cameras comprises a panoramic view camera.

41. The system of claim 30, wherein the processor is further configured to control at least one function of at least one camera of the plurality of cameras in response to the position information, the at least one function including at least one of focus, zoom, pan, or tilt.

42. The system of claim 30, wherein the processor controls at least one camera of the plurality of cameras to change location relative to the ongoing event in response to the position information.

43. The system of claim 30, wherein the processor receives a manual selection of camera output from among the plurality of cameras that suspends the processor from selecting a camera output in response to the current event state.

44. The system of claim 43, further including receiving a selection for the processor to resume selecting a camera output in response to the current event state.

45. The system of claim 30, wherein the processor signal comprises a signal that includes camera outputs from more than one camera of the plurality of cameras, and the system further includes a user interface of the processor that receives a manual selection of camera output from among the plurality of camera outputs and in response suspends selecting a camera output in response to the current event state.

46. The system of claim 30, wherein the processor signal comprises a signal that includes camera outputs from more than one camera of the plurality of cameras, and the system further includes a user interface that receives a user selection from an end user viewing system, the user selection identifying a single camera output from among the plurality of camera outputs comprising the processor signal such that the identified single camera output is provided for viewing at the end user viewing system.

47. The system of claim 30, wherein the processor output is combined with an augmented reality presentation.

48. The system of claim 30, wherein the equipment at the ongoing event includes at least one of a player, ball, puck, gaming projectile, clothing, apparel, jersey, shorts, pants, sox, shoes, hat, headband, sweatband, armband, helmet, shoulder pads, elbow pads, or shin guards.

49. The system of claim 30, wherein at least one of the plurality of cameras is mounted on a camera pole apparatus.

50. The system of claim 30, wherein one or more mirrors are used to produce at least one of the camera outputs.

51. The system of claim 30, wherein the position information includes rate of change of position information of the equipment.

52. The system of claim 30, wherein a computer generated three dimensional environment is created using the position information, wherein the position information is associated with equipment at the ongoing event.

53. The system of claim 30, wherein the selected camera output is a different camera output from a previously selected camera output that was selected according to the previous event state.

54. The system of claim 30, wherein the current event state is determined in response to the received position information and in accordance with a predetermined three-dimensional space occupied by the equipment.

55. The system of claim 54, wherein the equipment includes two or more sensors for transmission of position data, with which the three-dimensional positional space presently occupied by the equipment at the ongoing event may be determined in real time.

56. The system of claim 54, wherein the equipment includes two or more sensors for transmission of position data, with which the three-dimensional positional space presently occupied by the ball and/or other gaming projectile may be determined in real time.

57. The system of claim 30, wherein the received position information is transmitted from equipment at the ongoing event such that the received position information provides real time position information of the equipment from which the position information was transmitted.

58. The system of claim 57, wherein the position information is transmitted as a signal comprising at least one of RADAR, LIDAR, SONAR, or infrared signals.

59. A computer program product embodied on a non-transitory computer-readable storage medium containing computer implementable instructions executable by a computer to perform a method comprising:

receiving camera outputs at a processor from a plurality of cameras, each of which is configured to provide a respective camera output showing respective field of view coverage of an ongoing event;

receiving position information at the processor, wherein the position information is associated with equipment at the ongoing event such that the received position information provides real time position information of the equipment;

storing multiple event states into memory from which the data may be read by the processor;

wherein each event state is programmed responsive to event information and intelligence data extracted from the ongoing event and/or unique logical combinations of event information and intelligence data extracted from the ongoing event in real time;

wherein the event information and intelligence data includes event participant assignment execution outcome and results, event participant decision making outcome and results, and event participant interactions with other participants;

wherein the event information and intelligence data are extracted in real time responsive to the equipment position information inputs, conditional logic responsive to the equipment position information inputs, and/or other logical combinations of equipment position information inputs;

wherein the ongoing event has a current event state, where the current event state is identified in real time as one of the preprogrammed multiple event states;

wherein the current event state is updated in real time and may involve a transition from a previous event state that is different from the current event state;

wherein the current event state transition is responsive to changes in event information and intelligence data extracted from the received position information in real time;

wherein the method performed by the computer further comprises selecting a camera output from among the received plurality of camera outputs in response to the current event state such that the selected camera output provides its respective field of view coverage of the ongoing event, and producing a processor signal comprising the selected camera output.

60. The computer program product of claim 59, the performed method further comprising upon selecting the selected camera output in response to the current event state, combining the selected camera output with a camera output different from the selected camera output.

61. The computer program product of claim 59, the performed method further comprising switching from outputting video frames of the selected camera output at a first rate to outputting video frames of the selected camera output at a second rate in response to the position information, at least one of the first rate or the second rate resulting in slow motion video.

62. The computer program product of claim 59, the performed method further comprising combining audio information with the selected camera output, wherein the audio information is chosen independently of the position information.

63. The computer program product of claim 59, the performed method further comprising combining audio information with the selected camera output, wherein the audio information is chosen in response to the position information.

64. The computer program product of claim 59, wherein at least one of the plurality of cameras is configured to provide three dimensional (3D) field of view coverage of the ongoing event.

65. The computer program product of claim 59, wherein at least two of the cameras are configured at multiple vertical distances above the ongoing event.

66. The computer program product of claim 65, wherein the at least two cameras are spaced contiguously at the multiple vertical distances.

67. The computer program product of claim 59, wherein at least two of the cameras are configured at multiple radial positions arranged radially around the ongoing event substantially in the same plane.

68. The computer program product of claim 67, wherein the at least two cameras are spaced contiguously at the multiple radial positions.

69. The computer program product of claim 59, wherein at least one camera from the plurality of cameras comprises a panoramic view camera.

70. The computer program product of claim 59, the performed method further comprising controlling at least one function of at least one camera of the plurality of cameras in response to the position information, the at least one function including at least one of focus, zoom, pan, or tilt.

71. The computer program product of claim 59, the performed method further comprising controlling at least one camera of the plurality of cameras to change location relative to the ongoing event in response to the position information.

72. The computer program product of claim 59, the performed method further including receiving a manual selection of camera output from among the plurality of cameras that suspends the processor from selecting a camera output in response to the current event state.

73. The computer program product of claim 72, the performed method further including receiving a selection for the processor to resume selecting a camera output in response to the current event state.

74. The computer program product of claim 59, wherein the processor signal comprises camera outputs from more than one camera of the plurality of camera outputs, the performed method further comprising:

receiving a manual selection of a single camera output from among the plurality of camera outputs of the processor signal;

suspending selecting a camera output in response to the current event state.

75. The computer program product of claim 59, wherein the processor signal comprises camera outputs from more than one camera of the plurality of camera outputs, the performed method further comprising:

transmitting the processor signal to an end user viewing system for display at the end user viewing system, wherein the end user viewing system includes a user interface such that a manual selection input to the user interface identifies a single camera output from among the plurality of camera outputs of the transmitted processor signal for display at the end user viewing system.

76. The computer program product of claim 59, wherein the processor output is combined with an augmented reality presentation.

77. The computer program product of claim 59, wherein the equipment at the ongoing event includes at least one of a player, ball, puck, gaming projectile, clothing, apparel, jersey, shorts, pants, sox, shoes, hat, headband, sweatband, armband, helmet, shoulder pads, elbow pads, or shin guards.

78. The computer program product of claim 59, wherein at least one of the plurality of cameras is mounted on a camera pole apparatus.

79. The computer program product of claim 59, wherein one or more mirrors are used to produce at least one of the camera outputs.

80. The computer program product of claim 59, wherein the position information includes rate of change of position information of the equipment.

81. The computer program product of claim 59, wherein a computer generated three dimensional environment is created using the position information, wherein the position information is associated with equipment at the ongoing event.

82. The computer program product of claim 59, wherein the selected camera output is a different camera output from a previously selected camera output that was selected according to the previous event state.

83. The computer program product of claim 59, wherein the current event state is determined in response to the received position information and in accordance with a predetermined three-dimensional space occupied by the equipment.

84. The computer program product of claim 83, wherein the equipment includes two or more sensors for transmission of position data, with which the three-dimensional positional space presently occupied by the equipment at the ongoing event may be determined in real time.

85. The computer program product of claim 83, wherein the equipment includes two or more sensors for transmission of position data, with which the three-dimensional positional space presently occupied by the ball and/or other gaming projectile may be determined in real time.

86. The computer program product of claim 59, wherein the received position information is transmitted from equipment at the ongoing event such that the received position information provides real time position information of the equipment from which the position information was transmitted.

87. The computer program product of claim 86, wherein the position information is transmitted as a signal comprising at least one of RADAR, LIDAR, SONAR, or infrared signals.

88. A method of configuring a computer system, the method comprising:

establishing a network communication link from a source computer to the computer system;

transmitting program instructions from the source computer to the computer system over the network communication link, wherein the transmitted program instructions are configured to be stored in the computer system and executed, to receive camera outputs at a processor from a plurality of cameras, each of which is configured to provide a respective camera output showing respective field of view coverage of an ongoing event, receive position information at the processor,

wherein the position information is associated with equipment at the ongoing event such that the received position information provides real time position information of the equipment, store multiple event states into memory from which the data may be read by the processor;

wherein each event state is programmed responsive to event information and intelligence data extracted from the ongoing event and/or unique logical combinations of event information and intelligence data extracted from the ongoing event in real time;

wherein the event information and intelligence data includes event participant assignment execution outcome and results, event participant decision making outcome and results, and event participant interactions with other participants;

wherein the event information and intelligence data are extracted in real time responsive to the equipment position information inputs, conditional logic responsive to the equipment position information inputs, and/or other logical combinations of equipment position information inputs;

wherein the ongoing event has a current event state, where the current event state is identified in real time as one of the preprogrammed multiple event states;

wherein the current event state is updated in real time and may involve a transition from a previous event state that is different from the current event state;

wherein the current event state transition is responsive to changes in event information and intelligence data extracted from the received position information in real time; and

wherein the transmitted program instructions are further configured such that, when executed by the processor, the computer system receives the selected camera output from among the received plurality of camera outputs in response to the current event state such that the selected camera output provides its respective field of view coverage of the ongoing event and produces a processor signal comprising the selected camera output.

89. The method of computer system configuring of claim 88, wherein the selected camera output is a different camera output from a previously selected camera output that was selected according to the previous event state.

90. The method of computer system configuring of claim 88, wherein the current event state is determined in response to the received position information and in accordance with a predetermined three-dimensional space occupied by the equipment.

91. The method of claim 90, wherein the equipment includes two or more sensors for transmission of position data, with which the three-dimensional positional space presently occupied by the equipment at the ongoing event may be determined in real time.

92. The method of claim 90, wherein the equipment includes two or more sensors for transmission of position data, with which the three-dimensional positional space presently occupied by the ball and/or other gaming projectile may be determined in real time.

93. The method of computer system configuring of claim 88, wherein the received position information is transmitted from equipment at the ongoing event such that the received position information provides real time position information of the equipment from which the position information was transmitted.

94. The method of computer system configuring of claim 88, further comprising upon selecting the selected camera output in response to the current event state, combining the selected camera output with a camera output different from the selected camera output.

95. The method of computer system configuring of claim 93, wherein the position information is transmitted as a signal comprising at least one of RADAR, LIDAR, SONAR, or infrared signals.