Selecting and correlating physical activity data with image data

• US 10,179,263 B2
• Filed: 06/29/2016
• Issued: 01/15/2019
• Est. Priority Date: 02/17/2011
• Status: Active Grant

• Current Assignee: Nike, Inc.
• Original Assignee: Nike, Inc.
• Inventors: Burroughs, Brandon S., Engelberg, Richard J., Rice, Jordan M., Sweet, Bob, Weast, Aaron B., Convay, Wade, Rooke-Ley, Joshua
• Primary Examiner(s): Mazumder, Saptarshi
• Application Number:
  US15/197,014
• Publication Number:
  US 20170014684A1
• Time in Patent Office: 930 Days
• Field of Search: None
• CPC Class Codes: A63B 2225/50 Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 24/0062 Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance
• CPC Class Codes: A63B 43/00 Balls with special arrangements
• CPC Class Codes: A63B 43/06 with illuminating devices ; with reflective surfaces
• CPC Class Codes: A63B 71/0622 Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Example embodiments may relate systems, methods, apparatuses, and computer readable media configured to correlate image data of a user performing physical activity with data collected during the user'"'"'s performance. Data may include sensor data measuring, force, acceleration, speed, and/or processed sensor data from one or more sensors. Certain embodiments may determine whether the user is within a performance zone based on user attributes. Correlation of the image data with physical activity data may be based, at least in part, whether the user is within a performance zone.

Claims

1. A computer-implemented method comprising:

receiving a first set of physical activity data corresponding to an athletic performance comprising at least a first activity performed by a user, wherein the first set of physical activity data is generated by or derived from a first sensor, and wherein the first set of physical activity data further comprises at least one of;

a shoe type, a shoe color, or a shoe size associated with the user during the athletic performance;

capturing, by an image capturing device, image data corresponding to at least the first activity performed by the user;

determining, by one or more processors, whether a portion of the first set of physical activity data indicates that the user has satisfied a first activity threshold;

correlating, in response to determining that the portion of the first set of physical activity data indicates that the user has satisfied the first activity threshold, the image data in accordance with the portion of the first set of physical activity data;

dynamically modifying, based on user input selections, the portion of the first set of physical activity data to be overlaid with the correlated image data;

joining the correlated image data and the portion of the first set of physical activity data into a single file; and

transmitting the file to a computing device for display to the user.

2. The computer-implemented method of claim 1, wherein the image data comprises a plurality of frames, and wherein the correlating the image data further comprises:

associating a first number of frames of the image data with a corresponding time period of the first set of physical activity data.

3. The computer-implemented method of claim 1, wherein the image data comprises a plurality of frames, the computer-implemented method further comprising:

determining, by the one or more processors, a plurality of activity values for a first activity metric corresponding to the portion of the first set of physical activity data; and

adjusting the plurality of activity values to be associated with at least a first frame of the image data.

4. The computer-implemented method of claim 1, further comprising:

establishing a communication channel with the first sensor, wherein the first sensor comprises a force sensor; and

in response to establishing the communication channel, outputting, to a display device, a communication prompting the user to perform the first activity.

5. The computer-implemented method of claim 1, further comprising:

generating, by the one or more processors, a summary video segment associated with the image data; and

outputting, to a display device, an overlay for the summary video segment, wherein the overlay comprises origination data for the first set of physical activity data.

6. The computer-implemented method of claim 1, further comprising:

receiving an input selection identifying the user; and

in response to receiving the input selection, initiating the correlation of the image data in accordance with the portion of the first set of physical activity data.

7. The computer-implemented method of claim 1, further comprising:

determining, based on the portion of the first set of physical activity data, a second activity performed by the user during the athletic performance; and

in response to the determining the second activity, initiating the correlation of the image data in accordance with the portion of the first set of physical activity data.

8. The computer-implemented method of claim 1, further comprising:

receiving a second set of physical activity data corresponding to one or more previous athletic performances where the user performed the first activity; and

correlating the image data in accordance with the second set of physical activity data.

9. The computer-implemented method of claim 1, further comprising:

determining, based on a second portion of the first set of physical activity data, that the user has satisfied a performance zone threshold; and

in response to the determining the performance zone threshold is satisfied, initiating the capturing of the image data in accordance with the second portion of the first set of physical activity data.

10. One or more non-transitory computer readable media storing instructions that, when executed by at least one processor, cause the at least one processor to:

receive a first set of physical activity data corresponding to an athletic performance comprising at least a first activity performed by a user, wherein the first set of physical activity data is generated by or derived from a first sensor, and wherein the first set of physical activity data further comprises at least one of;

a shoe type, a shoe color, or a shoe size associated with the user during the athletic performance;

determine that one or more portions of the first set of physical activity data indicate that the user has satisfied a first activity threshold;

in response to the determining that the one or more portions of the first set of physical activity data indicates that the user has satisfied the first activity threshold, capture, by an image capturing device, image data corresponding to at least the first activity performed by the user;

determine, based on the one or more portions of the first set of physical activity data, a second activity performed by the user during the athletic performance;

in response to the determining that the user performed the second activity, initiate the correlation of the image data in accordance with the one or more portions of the first set of physical activity data;

dynamically modify, based on user input selections, the one or more portions of the first set of physical activity data to be overlaid with the correlated image data;

join the correlated image data and the one or more portions of the first set of physical activity data into a single file; and

transmit the file to a computing device for display to the user.

11. The one or more non-transitory computer readable media of claim 10, wherein the instructions, when executed, further cause the at least one processor to:

receive a second set of physical activity data corresponding to one or more previous athletic performances where the user performed the first activity; and

correlate the image data in accordance with the second set of physical activity data.

12. The one or more non-transitory computer readable media of claim 10, wherein the instructions, when executed, further cause the at least one processor to:

establish a communication channel with the first sensor, wherein the first sensor comprises a force sensor; and

in response to establishing the communication channel, output, to a display device, a communication prompting the user to perform the first activity.

13. The one or more non-transitory computer readable media of claim 12, wherein the instructions, when executed, further cause the at least one processor to:

generate a summary video segment associated with the image data; and

output, to a display device, an overlay for the summary video segment, wherein the overlay comprises origination data for the first set of physical activity data.

14. The one or more non-transitory computer readable media of claim 10, wherein the instructions, when executed, further cause the at least one processor to correlate the image data in accordance with the portion of the first set of physical activity data by:

determining that the portion of the physical activity data indicates the user performed a first type of athletic activity; and

in response to determining that the user performed the first type of athletic activity, initiate the correlating the image data with the portion of the first set of physical activity data.