Storage and editing of video of activities using sensor and tag data of participants and spectators

• US 10,277,861 B2
• Filed: 10/16/2017
• Issued: 04/30/2019
• Est. Priority Date: 09/10/2014
• Status: Active Grant

• Current Assignee: Fleye, Inc.
• Original Assignee: Fleye, Inc.
• Inventors: Miner, Cameron
• Primary Examiner(s): Nguyen, Huy T
• Application Number:
  US15/784,221
• Publication Number:
  US 20180098022A1
• Time in Patent Office: 561 Days
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/30 on the same track as the main recording
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Video from multiple locations in a venue, in which multiple individuals are engaged in activities, such as athletic performances or non-sporting activities, is stored and edited, using sensor data from participants and tag data from spectators, to create individualized video programs of athletic performances of individuals. Each camera provides a video feed that is continuously stored. Each video feed is stored in a sequence of data files in computer storage, which data files are created in regular time intervals. Each file is accessible using an identifier of the camera and a time interval. Similarly, data from sensors and tagging devices are continuously received and stored in a database. The database stores, for each sensor, an identifier of each individual detected in the proximity of the sensor and the time at which the individual was detected. Each sensor is associated with one or more cameras. Tag data is similarly stored from tagging devices manipulated by spectators.

Claims

1. A computer-implemented process for generating an edited video program for an event in a venue, wherein the venue includes one or more cameras that capture video of activity of a set of individuals participating in the event in the venue, comprising:

continuously receiving video data from the one or more cameras during periods of time during the event;

storing the received video data for each camera in one or more data files, each data file including video data from the camera for a period of time during the event;

receiving, and storing in a tag database, tag data from a plurality of user devices associated with users who are not the individuals participating in the event, the tag data including a plurality of tag hits including, for each tag hit, a time and an identifier of the user device providing the tag hit, wherein a tag hit indicates interest by a user in video for the activity occurring in the venue at the time of the tag hit;

storing, in computer storage, data associating at least one of the users with the activity;

receiving an indication of an individual participating in the activity and an indication of a first user;

determining whether the first user is associated with the activity in the data stored in the computer storage;

in response to a determination that the first user is associated with the activity in the data stored in the computer storage, accessing the stored tag data, using the indication of the individual, to retrieve times associated with tag data associated with the individual and accessing the stored video data to retrieve, for the first user, clips of the stored video data according to the retrieved times for tag hits associated with the individual.

2. The computer-implemented process of claim 1, wherein the indication of the individual is the identifier of the user device used by the user to enter tag hits.

3. The computer-implemented process of claim 1, further comprising:

accessing data associating permissions with each of the users; and

wherein accessing the stored video data by a user is conditioned upon the permissions associated with the user.

4. The computer-implemented process of claim 1, wherein the tag database comprises a database table comprising a row for each tag hit including at least a first column indicating a time for the tag hit.

5. The computer-implemented process of claim 4, wherein the database table is provided for each user device.

6. The computer-implemented process of claim 4, wherein the database table further includes a second column for an identifier of the user device.

7. The computer-implemented process of claim 4, wherein the tag database includes a table including rows relating identifiers of user devices to information used to control access to the stored video data.

8. A computer-implemented process for generating an edited video program using video captured from one or more cameras and tag data indicating times at which tag hits are received on user devices, the process comprising:

receiving and storing video data from the one or more cameras during periods of time, the received video data for each camera being stored in one or more data files, each data file including video data from the camera for a period of time;

receiving, and storing in a tag database, tag data from a plurality of user devices associated with users who are not participants in the activity, the tag data including a plurality of tag hits including, for each tag hit, a time and an identifier of the user device, wherein a tag hit indicates interest by a user in video for the activity occurring at the time of the tag hit;

storing, in computer storage, data associating at least one of the users with the activity;

receiving an indication of an individual participating in the activity and an indication of a first user;

determining whether the first user is associated with the activity in the data stored in the computer storage;

in response to a determination that the first user is associated with the activity in the data stored in the computer storage;

accessing the stored tag data, using an indication of the first user, to retrieve times at which tag hits associated with the first user and the activity were received;

identifying one or more cameras corresponding to the tag data and stored video data from the identified cameras which include video data from the retrieved times for the tag data; and

accessing the stored video data for the identified cameras to retrieve, for the first user, clips of the stored video data from the identified cameras according to the retrieved times from the tag data.

9. The computer-implemented process of claim 8, wherein the indication of the individual is the identifier of the user device used by the user to enter tag hits.

10. The computer-implemented process of claim 8, further comprising:

accessing data associating permissions with each of the users; and

wherein accessing the stored video data by a user is conditioned upon the permissions associated with the user.

11. The computer-implemented process of claim 8, wherein the tag database comprises a database table comprising a row for each tag hit including at least a first column indicating a time for the tag hit.

12. The computer-implemented process of claim 11, wherein the database table is provided for each user device.

13. The computer-implemented process of claim 11, wherein the database table further includes a second column for an identifier of the user device.

14. The computer-implemented process of claim 11, wherein the tag database includes a table including rows relating identifiers of user devices to information used to control access to the stored video data.

15. A computer system for generating an edited video program for an individual from a set of individuals participating in an event in a venue, wherein the venue includes a plurality of cameras that capture video of the individuals and a plurality of sensors, each sensor associated with one or more cameras of the plurality of cameras, each of the sensors at least detecting proximity of an individual near the sensor, the computer system comprising:

a processing unit, a computer storage medium and a database, and interfaces connected to receive data from the plurality of sensors and to receive video data from the plurality of cameras;

the processing unit executing computer program instructions to be operative to;

continuously receive the video data from the plurality of cameras during periods of time during the event;

store the received video data for each camera in a sequence of data files in the computer storage, each data file in the sequence being created in regular time intervals, each data file having as associated time interval and including video data from the camera for a period of time during the event in the associated time interval, each data file in the sequence being accessible using an identifier of the camera providing the received video data and data representing the time interval;

continuously receive the data from the plurality of sensors during periods of time during the event, the received data, from each sensor, being indicative of individuals and times at which proximity of the individuals to the sensor is detected by the sensor;

store the received data from each sensor in the database such that the database stores, for each sensor, an identifier of each individual detected in the proximity of the sensor by the sensor, and a time at which the individual was detected in the proximity of the sensor by the sensor;

receiving, and storing in a tag database, tag data from a plurality of user devices associated with users who are not participants in the activity, the tag data including a plurality of tag hits including, for each tag hit, a time and an identifier of the user device, wherein a tag hit indicates interest by a user in video for the activity occurring at the time of the tag hit;

store, in computer storage, data associating at least one of the users with the activity and individuals participating in the activity;

receive an indication of a first user and a first activity;

determine whether the first user is associated with the activity in the data stored in the computer storage;

in response to a determination that the first user is associated with the activity in the data stored in the computer storage;

determine an indication of an individual associated with the first user;

access, from the database, the stored data from the sensors, using the indication of the individual, to retrieve times from the database at which the individual was detected in the proximity of one or more of the sensors by the one or more of the sensors, and identifiers of the one or more of the sensors; and

access the stored video data from the computer storage to retrieve, for the first user, clips of the stored video data received from cameras associated with the one or more sensors, according to the retrieved times from the database at which the individual was detected in the proximity of one or more sensors by the one or more of the sensors and the times associated with the tag hits from the user device associated with the first user.

16. The computer system of claim 15, wherein the indication of the individual is the identifier of the user device used by the user to enter tag hits.

17. The computer system of claim 15, further comprising stored data associating permissions with each of the users, and wherein the processing system is further operative to condition access to the stored video data by a user upon the permissions associated with the user.

18. The computer system of claim 15, wherein the tag database comprises a database table comprising a row for each tag hit including at least a first column indicating a time for the tag hit.

19. The computer system of claim 18, wherein the database table is provided for each user device.

20. The computer system of claim 18, wherein the database table further includes a second column for an identifier of the user device.