VIDEO AND MOTION EVENT INTEGRATION SYSTEM

• US 20150154452A1
• Filed: 11/20/2014
• Published: 06/04/2015
• Est. Priority Date: 08/26/2010
• Status: Active Grant

• Current Assignee: Newlight Capital, LLC
• Original Assignee: Blast Motion Incorporated
• Inventors: Bentley, Michael, Kaps, Ryan, Bose, Bhaskar
• Granted Patent: US 9,235,765 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G11B 27/022 Electronic editing of analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/17 using electrical sensing means see provisionally also G11B27/13
• CPC Class Codes: G11B 31/006 with video camera or receiver
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Enables intelligent synchronization and transfer of generally concise event videos synchronized with motion data from motion capture sensor(s) coupled with a user or piece of equipment. Greatly saves storage and increases upload speed by uploading event videos and avoiding upload of non-pertinent portions of large videos. Provides intelligent selection of multiple videos from multiple cameras covering an event at a given time, for example selecting one with least shake. Enables near real-time alteration of camera parameters during an event determined by the motion capture sensor, and alteration of playback parameters and special effects for synchronized event videos. Creates highlight reels filtered by metrics and can sort by metric. Integrates with multiple sensors to save event data even if other sensors do not detect the event. Also enables analysis or comparison of movement associated with the same user, other user, historical user or group of users.

Claims

1. A video and motion event integration system comprising:

at least one motion capture element configured to couple with a user or piece of equipment or mobile device coupled with the user, wherein said at least one motion capture element comprises

a memory;

a sensor configured to capture any combination of values associated with an orientation, position, velocity and acceleration of said at least one motion capture element;

a radio;

a microcontroller coupled with said memory, said sensor and said radio, wherein said microcontroller is configured to

collect data that comprises sensor values from said sensor;

store said data in said memory;

optionally analyze said data and recognize an event within said data to determine event data;

transmit said data or said event data associated with said event via said radio;

at least one computer comprising

a wireless communication interface configured to communicate with said radio to obtain said event data associated with said event;

wherein said computer is coupled with wireless communication interface, wherein said computer is configured to

receive

said data from said wireless communication interface and analyze said data and recognize an event within said data to determine event data

or

said event data from said wireless communication interface

or

both said data and said event data;

analyze said event data to form motion analysis data;

store said event data, or said motion analysis data, or both said event data and said motion analysis data;

obtain an event start time and an event stop time from said event data;

obtain at least one video start time and at least one video stop time associated with at least one video from at least one camera;

synchronize said event data, said motion analysis data or any combination thereof with said at least one video based on

a first time associated with said data or said event data obtained from said at least one motion capture element coupled with said user or said piece of equipment or said mobile device coupled with the user and

at least one time associated said at least one video to create at least one synchronized event video;

command at least one camera to transfer said at least one synchronized event video captured at least during a timespan from within said event start time to said event stop time to another computer without transferring at least a portion of the video that occurs outside of said at least one video that occurs outside of said timespan from within said event start time to said event stop time to said another computer.

2. The system of claim 1 wherein said at least one computer is further configured to overlay a synchronized event video comprising both of

said event data, said motion analysis data or any combination thereof that occurs during said timespan from said event start time to said event stop time and

said video captured during said timespan from said event start time to said event stop time.

3. The system of claim 1 wherein said at least one computer is further configured to discard at least a portion of said at least one video outside of said event start time to said event stop time.

4. The system of claim 1 wherein said at least one computer is a mobile device, said camera, a desktop computer or server computer or any combination of any number of a mobile device, said camera, desktop computer and a server computer.

5. The system of claim 1 wherein said at least one computer is further configured to accept a metric associated with said at least one synchronized event video;

display a plurality of synchronized event videos or plurality of corresponding thumbnails thereof along with a value associated with said metric for each of said plurality of synchronized event videos or said plurality of corresponding thumbnails.

6. The system of claim 5 wherein said at least one computer is further configured to accept a selection criteria for said metric;

display only those synchronized event videos or said thumbnails thereof that pass said selection criteria.

7. The system of claim 6 wherein said at least one computer is further configured to order said only those synchronized event videos or said thumbnails thereof according to a value of said metric.

8. The system of claim 6 wherein said at least one computer is further configured to generate a highlight reel of said plurality of synchronized event videos that pass said selection criteria.

9. The system of claim 1 wherein said at least one computer is further configured to select a first synchronized event video to display from said at least one synchronized event video for a particular time span that are associated with different cameras based on image analysis.

10. The system of claim 1 wherein said at least one computer is further configured to select a first synchronized event video to display from said at least one synchronized event video for a particular time span that are associated with different cameras based on image analysis wherein said image analysis comprises minimal shake detection.

11. The system of claim 1 wherein said at least one computer is further configured to send a control message to said at least one camera to modify video recording parameters of said at least one video based on said data or said event data or said motion analysis data.

12. The system of claim 11 wherein said video recording parameters comprise one or more of frame rate, resolution, color depth, color or grayscale, compression method, compression quality, and recording on or off.

13. The system of claim 1 wherein said at least one computer is further configured to accept a sound track;

analyze said sound track to determine time points of high audio amplitude associated with beats;

synchronize said beats with said event.

14. The system of claim 1 wherein said at least one computer is configured to display said at least one synchronized event video;

and to

modify playback characteristics of said at least one synchronized event video based on said event data, said motion analysis data, or both,

wherein said modifications to playback characteristics may occur one or more times throughout the playback of said at least one synchronized event video.

15. The system of claim 14 wherein said modifications to playback characteristics comprise one or more of changes to playback speed, image brightness, image colors, image focus, image resolution, flashing special effects, use of graphic overlays or borders.

16. The system of claim 14 wherein said modifications to playback characteristics comprise slower motion playback as said event video approaches a sub-event of interest or as said event video approaches a peak value of a selected metric associated with said motion analysis data or said event data or both.

17. The system of claim 14 wherein said at least one computer is further configured to accept a sound track;

analyze said sound track to determine time points of high audio amplitude associated with beats;

play said sound track along with said at least one synchronized event video;

synchronize said beats with said event; and

,

wherein said modifications to playback characteristics comprise one or more of modifying one or more of the volume, tempo, tone, or audio special effects of said sound track.

18. The system of claim 1 wherein said at least one computer is further configured while a communication link is open between said at least one motion capture sensor and said at least one computer to discard at least a portion of said video outside of said event start time to said event stop time and save said video from said event start time to said event stop time with said motion analysis data that occurs from said event start time to said event stop time.

19. The system of claim 1 wherein said at least one computer is further configured while a communication link is not open between said at least one motion capture sensor and said at least one computer to save video and after said event is received after said communication link is open, then discard at least a portion of said video outside of said event start time to said event stop time and save said video from said event start time to said event stop time with said motion analysis data that occurs from said event start time to said event stop time.

20. The system of claim 1 wherein said at least one computer is further configured to synchronize said motion analysis data with said video based on image analysis to more accurately determine a start event frame or stop event frame in said video or both, that is most closely associated with said event start time or said event stop time or both.

21. The system of claim 1 wherein said at least one computer is further configured to generate a metric from an object within said at least one synchronized event video based on image analysis.

22. The system of claim 1 wherein said at least one computer is further configured to access previously stored event data or motion analysis data associated with said user or piece of equipment;

display information comprising a presentation of said event data associated with said at least one user on a display based on

said event data or motion analysis data associated with said user or piece of equipment

and

said previously stored event data or motion analysis data associated with said user or piece of equipment

or

said previously stored motion capture data or motion analysis data associated with at least one other user or other piece of equipment.

23. The system of claim 1 wherein said microcontroller in said at least one motion capture element is configured to transmit said event to at least one other motion capture sensor or said at least one computer or any combination thereof, and wherein said at least one other motion capture sensor or said at least one other mobile device or any combination thereof is configured to save data or transmit data or both save data and transmit data associated with said event even if said at least one other motion capture sensor has not detected said event.

24. The system of claim 1 wherein said at least one computer is further configured to request or broadcast a request for camera locations proximal to said event or oriented to view said event or both proximal to and oriented to view said event.

25. The system of claim 1 wherein said at least one computer is further configured to request said video from said at least one camera proximal to said event wherein said video contains said event.

26. The system of claim 1 wherein the microcontroller is coupled with a temperature sensor and wherein said microcontroller is configured to transmit a temperature obtained from the temperature sensor as a temperature event.

27. The system of claim 1 wherein said at least one computer is further configured to obtain sensor values from sensors proximal to said event and wherein said sensors comprise any combination of temperature, humidity, wind, elevation, light, sound and physiological sensors and save said sensor values that occur within said timespan from said event start time and event stop time with said event data.

28. The system of claim 1 wherein said event comprises

motion associated with said at least one motion capture sensor coupled with any combination of said user, or said piece of equipment or said mobile device

or

motion indicative of standing, walking, falling, heat stroke, a seizure, violent shaking, a concussion, a collision, abnormal gait, abnormal or non-existent breathing

or

any combination thereof.