MULTI-SENSOR EVENT DETECTION SYSTEM

• US 20150318015A1
• Filed: 07/16/2015
• Published: 11/05/2015
• Est. Priority Date: 08/26/2010
• Status: Active Grant

• Current Assignee: Newlight Capital, LLC
• Original Assignee: Blast Motion Incorporated
• Inventors: BOSE, Bhaskar, BENTLEY, Michael, KAPS, Ryan
• Granted Patent: US 9,406,336 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/216 using geographical information, e.g. location of the game device or player using GPS
• CPC Class Codes: A63F 13/217 using environment-related information, i.e. information generated otherwise than by the player, e.g. ambient temperature or humidity
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 13/828 Managing virtual sport teams
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 21/0492 Sensor dual technology, i.e. two or more technologies collaborate to extract unsafe condition, e.g. video tracking and RFID tracking
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 2020/10592 specifically adapted for recording or reproducing multichannel signals
• CPC Class Codes: G11B 27/031 : Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/765 : Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 7/188 : Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A sensor event detection system including a motion capture element with a memory, sensor, microprocessor, first communication interface and another sensor. The sensor captures values associated with an orientation, position, velocity and acceleration of the motion capture element. The first communication interface receives other values associated with a temperature, humidity, wind and elevation, and the other sensor locally captures the other values. The microprocessor collects data that includes sensor values from the sensor, stores the data in the memory, and recognizes an event within the data to determine event data. The microprocessor correlates the data or event data with the other values to determine a false positive event or a type of equipment the motion capture element is coupled with, or a type of activity indicated by the data or event data, and transmits the data or event data associated with the event via the first communication interface.

Claims

1. A multi-sensor event detection system comprising:

at least one motion capture element configured to couple with a user or piece of equipment or mobile device coupled with the user, wherein said at least one motion capture element comprises

a sensor data memory;

a sensor configured to capture one or more values associated with an orientation, position, velocity, acceleration, angular velocity, and angular acceleration of said at least one motion capture element;

a first communication interface configured to receive one or more other values associated with a temperature, humidity, wind, elevation, light, sound, heart rate or

at least one other sensor configured to locally capture said one or more other values associated with said temperature, humidity, wind, elevation, light, sound, heart rate or

both said first communication interface and said at least one other sensor; and

,

a microprocessor coupled with said sensor data memory, said sensor and said first communication interface, wherein said microprocessor is configured to collect data that comprises sensor values from said sensor;

store said data in said sensor data memory;

optionally analyze said data and recognize an event within said data to determine event data;

correlate said data or said event data with said one or more other values associated with said temperature, humidity, wind, elevation, light, sound, heart rate to determine at least one of

a true event or a false positive event or quality of event or

a type of equipment that said at least one motion capture element is coupled with or

a type of activity indicated by said data or said event data; and

,

transmit at least one of said data, said event data, said true event, said false positive event, said quality of said event, said type of equipment, said type of activity via said first communication interface.

2. The system of claim 1 wherein said microprocessor is further configured to obtain confirm said event for a particular location and time by analyzing one or more of text, audio, image, and video from a server to create a confirmed event.

3. The system of claim 1 wherein said microprocessor is further configured to detect said type of equipment said at least one motion capture element is coupled with or said type of activity said at least one motion sensor senses through said correlate, when a similar motion indicates multiple types of potential activities, to differentiate a first type of activity with respect to a second type of activity based on said one or more values associated with temperature, humidity, wind, elevation, light, sound, heart rate.

4. The system of claim 2 wherein said first type of activity comprises surfing, said second type of activity comprises snowboarding or skateboarding.

5. The system of claim 1 wherein said microprocessor is further configured to recognize a location of said sensor on said piece of equipment or said user based on said data or event data.

6. The system of claim 1 wherein said microprocessor is further configured to collect data that comprises sensor values from said sensor based on a sensor personality selected from a plurality of sensor personalities, wherein the sensor personality is configured to control sensor settings to collect the data in an optimal manner with respect to a specific type of movement or said type of activity associated with a specific piece of equipment or type of clothing.

7. The system of claim 1

said microprocessor is further configured to determine said false positive event as

detect a first value from said sensor values having a first threshold value and

detect a second value from said sensor values having a second threshold value within a time window;

signify a prospective event;

compare said prospective event to a characteristic signal associated with a typical event and eliminate any false positive events;

signify a valid event if said prospective event is not a false positive event;

save said valid event in said sensor data memory comprising information within an event time window as said data.

8. The system of claim 1, wherein said microprocessor is further configured to recognize said at least one motion capture element with newly assigned locations after said at least one motion capture element is removed from said piece of equipment and coupled with a second piece of equipment of a different type based on said data or event data.

9. The system of claim 1 wherein said at least one motion capture element is contained within a motion capture element mount, said mobile device, a mobile phone, a smart phone, a smart watch, a camera, a laptop computer, a notebook computer, a tablet computer, a desktop computer, a server computer or any combination of any number of said motion capture element mount, said mobile device, said mobile phone, said smart phone, said smart watch, said camera, said laptop computer, said notebook computer, said tablet computer, said desktop computer and said server computer.

10. The system of claim 1 wherein said microprocessor is further configured to transmit said at least one of said data, said event data, said true event, said false positive event, said quality of said event, said type of equipment, said type of activity to one or more of a repository, a viewer, a server, another computer, a social media site, said mobile device, a network, and an emergency service.

11. The system of claim 1 wherein said at least one motion capture element comprises

a first motion capture element coupled to a first side of said joint of said user;

wherein said first motion capture element comprises

an accelerometer;

a magnetometer; and

a rate gyroscope;

wherein said microprocessor is configured to send an alert to another computer or to another user if an angle of rotation detected by said first motion capture element exceeds a threshold value.

12. The system of claim 1 wherein said at least one motion capture element comprises

a first motion capture element coupled to a first side of said joint of said user;

a second motion capture element coupled to a second side of said joint of said user;

wherein said first motion capture element and said second motion capture element comprise

an accelerometer;

a magnetometer; and

,

a rate gyroscope;

wherein said microprocessor is configured to send an alert to another computer or to another user if an angle of rotation detected by said first motion capture element and said second motion capture element exceeds a threshold value.

13. The system of claim 1 further comprising:

a computer comprising

a computer memory;

a second communication interface configured to communicate with said first communication interface to obtain said data or said event data associated with said event or both said data and said event data;

wherein said computer is coupled with said computer memory and is coupled with said second communication interface, wherein said computer is configured to

receive

said data from said second communication interface and analyze said data and recognize an event within said data to determine event data, or

said event data from said second communication interface, or

both said data and said event data from said second communication interface;

analyze said event data to form motion analysis data;

store said event data, or said motion analysis data, or both said event data and said motion analysis data in said computer memory;

obtain an event start time and an event stop time from said event data;

obtain at least one video start time and at least one video stop time associated with at least one video;

synchronize said event data, said motion analysis data or any combination thereof with said at least one video based on

a first time associated with said data or said event data obtained from said at least one motion capture element coupled with said user or said piece of equipment or said mobile device coupled with the user and

at least one time associated with said at least one video to create at least one synchronized event video; and

,

store said at least one synchronized event video in said computer memory without at least a portion of said at least one video outside of said event start time to said event stop time.

14. The system of claim 13 wherein said computer comprises at least one processor in said mobile device, a mobile phone, a smart phone, a smart watch, a camera, a laptop computer, a notebook computer, a tablet computer, a desktop computer, a server computer or any combination of any number of said mobile device, said mobile phone, said smart phone, said smart watch, said camera, said laptop computer, said notebook computer, said tablet computer, said desktop computer and said server computer.

15. The system of claim 14 wherein said computer is further configured to display a synchronized event video comprising both of

said event data, said motion analysis data or any combination thereof that occurs during a timespan from said event start time to said event stop time;

and,

said at least one video captured during said timespan from said event start time to said event stop time.

16. The system of claim 14 wherein said computer is further configured to discard or instruct another computer to discard or instruct a camera to discard said at least a portion of said at least one video outside of said event start time to said event stop time.

17. The system of claim 14 wherein said computer is further configured to

send a control message

locally to at least one camera coupled with said computer or

externally said at least one camera

to modify video recording parameters of said at least one video based on said data or said event data or said motion analysis data;

wherein said video recording parameters comprise one or more of frame rate, resolution, color depth, color or grayscale, compression method, compression quality, and recording on or off.

18. The system of claim 14 wherein said computer is further configured to transmit said at least one synchronized event video or a portion of said at least one synchronized event video to one or more of a repository, a viewer, a server, another computer, a social media site, said mobile device, a network, and an emergency service.

19. The system of claim 14 wherein said computer is further configured to transmit said event or said true event to one or more of a repository, a viewer, a server, another computer, a social media site, said mobile device, a network, and an emergency service.

20. The system of claim 14 wherein said computer is further configured to

accept a metric associated with said at least one synchronized event video;

accept selection criteria for said metric;

determine a matching set of synchronized event videos that have values associated with said metric that pass said selection criteria; and

,

display said matching set of synchronized event videos or corresponding thumbnails thereof along with said value associated with said metric for each of said matching set of synchronized event videos or said corresponding thumbnails.

21. The system of claim 20 wherein said computer is further configured to generate a highlight reel or fail reel of said matching set of synchronized event.

22. The system of claim 14, wherein

said sensor or said computer comprises a microphone that records audio signals;

said recognize an event comprises

determining a prospective event based on said data; and

,

correlating said data with said audio signals to determine if said prospective event is a valid event or a false positive event.

23. The system of claim 22, wherein said computer is further configured to store said audio signals in said computer memory with said at least one synchronized event video if said prospective event is a valid event.

24. The system of claim 22, wherein said computer is further configured to synchronize said motion analysis data with said at least one video based on image analysis to more accurately determine a start event frame or stop event frame in said video or both, that is most closely associated with said event start time or said event stop time or both.

25. The system of claim 14 wherein said computer is further configured to

access previously stored event data or motion analysis data associated with said user or piece of equipment;

display information comprising a presentation of said event data associated with said user on a display based on

said event data or motion analysis data associated with said user or piece of equipment; and

,

said previously stored event data or motion analysis data associated with said user or piece of equipment.

26. The system of claim 14 wherein said computer is further configured to

access previously stored event data or motion analysis data associated with at least one other user or at least one other piece of equipment;

display information comprising a presentation of said event data associated with said user on a display based on

said event data or motion analysis data associated with said user or piece of equipment; and

,

said previously stored event data or motion analysis data associated with said at least one other user or said at least one other piece of equipment.

27. The system of claim 14 wherein said microprocessor in said at least one motion capture element is configured to transmit said event to at least one other motion capture element or said computer or at least one other mobile device or any combination thereof, and wherein said at least one other motion capture element or said computer or said at least one other mobile device or any combination thereof is configured to save data or transmit data or both save data and transmit data associated with said event even if said at least one other motion capture element has not detected said event.

28. The system of claim 14 wherein said computer is further configured to

request or broadcast a request for camera locations proximal to said event or oriented to view said event or both proximal to and oriented to view said event;

and,

request said at least one video from said at least one camera proximal to said event wherein said video contains said event without at least a portion of said at least one video outside of said event start time to said event stop time.

29. The system of claim 14 wherein said computer is further configured to

said piece of equipment is configured to impact other objects during legitimate use;

said computer is further configured to

obtain one or more normal impact signatures that describe said motion analysis data for said event wherein said piece of equipment impacts said other objects during said legitimate use;

compare said one or more normal impact signatures to said motion analysis data to determine whether said piece of equipment has been used for said legitimate use.

30. The system of claim 14 wherein said computer is further configured to obtain confirm said event for a particular location and time by analyzing one or more of text, audio, image, and video from a server to create a confirmed event.