SENSOR AND MEDIA EVENT DETECTION AND TAGGING SYSTEM

• US 20160292509A1
• Filed: 06/16/2016
• Published: 10/06/2016
• Est. Priority Date: 08/26/2010
• Status: Active Grant

• Current Assignee: Newlight Capital, LLC
• Original Assignee: Blast Motion Incorporated
• Inventors: KAPS, Ryan, BENTLEY, Michael, BOSE, Bhaskar
• Granted Patent: US 9,646,209 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/42 of sport video content
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Enables detection and tagging of events using sensor data combined with data from servers such as social media sites. Sensors may measure values such as motion, temperature, humidity, wind, pressure, elevation, light, sound, or heart rate. Sensor data and event tags may be utilized to curate text, images, video, sound and post the results to social networks, for example in a dedicated feed. Event tags generated by the system may represent for example activity types, players, performance levels, or scoring results. The system may analyze social media postings to confirm or augment event tags. Users may filter and analyze saved events based on the assigned tags. The system may create highlight and fail reels filtered by metrics and by tags.

Claims

1. A sensor and media event detection and tagging system comprising:

a computer comprising

a computer memory; and

,

a first communication interface configured to obtain data, or event data comprising one or more values associated with one or more of an orientation, position, velocity, acceleration, angular velocity, angular acceleration, electromagnetic field, temperature, humidity, wind, pressure, elevation, light, sound, and heart rate associated with a user or piece of equipment or mobile device coupled with the user;

wherein said computer is coupled with said computer memory and is coupled with said first communication interface, wherein said computer is configured to

receive

said data from said first communication interface and analyze said data and recognize an event within said data to determine said event data,

or

said event data from said first communication interface,

or

both said data and said event data from said first communication interface;

store said event data in said computer memory;

obtain an event start time and an event stop time from said event data;

confirm said event for a particular location and time by analyzing one or more of text, audio, image, and video from a server to create a confirmed event, wherein said one or more of said text, audio, image, and video comprise one or more of email messages, voice calls, voicemails, audio recordings, video calls, video messages, video recordings, text messages, chat messages, postings on social media sites, postings on blogs, and postings on wikis;

determine one or more tags for said event by analyzing one or more of

said data,

said event data,

said one or more of said text, audio, image, and video from said server; and

, publish said confirmed event and said one or more tags for said event.

2. The system of claim 1, wherein said one or more tags represent one or more of

an activity type of said event;

a location of said event;

a timestamp of said event;

a stage of an activity associated with said event;

a player identity associated with said event;

a performance level associated with said event; and

,

a scoring result associated with said event.

3. The system of claim 1, further comprising

a manual tagging interface configured to

accept one or more user selected tags for said event;

publish said one or more user selected tags for said event.

4. The system of claim 1, further comprising

a database configured to store said event and said one or more tags for said event;

an event filter coupled to said database, wherein said event filter is configured to

accept one or more filter tags;

return events in said database having one or more tags that match said one or more filter tags.

5. The system of claim 1, further comprising:

at least one sensor element configured to couple with said user or said piece of equipment or said mobile device coupled with the user, wherein said at least one sensor element comprises

a sensor data memory;

at least one sensor configured to capture said one or more values associated with one or more of an orientation, position, velocity, acceleration, angular velocity, angular acceleration, electromagnetic field, temperature, humidity, wind, pressure, elevation, light, sound, and heart rate;

a second communication interface; and

,

a microprocessor coupled with said sensor data memory, said at least one sensor and said second communication interface, wherein said microprocessor is configured to

collect said data that comprises sensor values from said at least one sensor;

store said data in said sensor data memory;

optionally analyze said data and recognize an event within said data to determine said event data; and

,

transmit said data or said event data associated with said event or both said data and said event data to said computer via said second communication interface.

6. The system of claim 1, wherein said server comprises one or more of an email server, a social media site, a photo sharing site, a video sharing site, a blog, a wiki, a database, a newsgroup, an RSS server, a multimedia repository, a document repository, and a text message server.

7. The system of claim 1, wherein said analyzing one or more of said text, audio, image, and video comprises searching said text for key words or key phrases related to said confirmed event.

8. The system of claim 1, wherein said analyzing one or more of said text, audio, image, and video comprises image analysis to search for images related to said confirmed event.

9. The system of claim 1, wherein said computer obtains one or more additional sensor values associated with one or more of an orientation, position, velocity, acceleration, angular velocity, angular acceleration, electromagnetic field, temperature, humidity, wind, pressure, elevation, light, sound, and heart rate from a second sensor or second computer or from a plurality of other sensors or other computers to perform said confirm said event or to perform said determine said one or more tags for said event.

10. The system of claim 9 wherein said recognize an event comprises

determine a prospective event based on said data;

correlate said data with said one or more additional sensor values to determine if said prospective event is a valid event or a false positive event.

11. The system of claim 9 wherein said computer correlates said data or said event data with one or more of said one or more additional sensor values to determine a type of equipment that is coupled with said second sensor, said second computer or said plurality of other sensors or other computers.

12. The system of claim 9 wherein said computer correlates said data or said event data with one or more of said one or more additional sensor values to determine a type of activity for said event.

13. The system of claim 12 wherein said computer determines said type of activity for said event through a comparison of said one or more additional sensor values to typical sensor values for a first type of activity and for at least one second type of activity.

14. The system of claim 1, wherein said computer is further configured to

obtain a plurality of instances of said data or said event data or both said data and said event data from a plurality of other sensors or other computers;

correlate said plurality of instances of said data or said event data or both said data and said event data based on a time and a location of each of said instances of said data or said event data or both said data and said event data; and

,

perform said confirm said event when

the number of said plurality of instances of said data or said event data or both said data and said event data occurring within a time range and a location range exceeds an event count threshold, or,

an aggregate metric calculated from said plurality of instances of said data or said event data or both said data and said event data for the plurality of instances of said data or said event data or both said data and said event data occurring within said time range and said location range exceeds an aggregate metric threshold.

15. The system of claim 5 wherein said microprocessor is further configured to recognize a location of said at least one sensor on said piece of equipment or said user based on said data or said event data.

16. The system of claim 5 wherein said microprocessor is further configured to collect said data that comprises said sensor values from said at least one sensor based on a sensor personality selected from a plurality of sensor personalities, wherein the sensor personality is configured to control sensor settings to collect the data in an optimal manner with respect to a specific type of movement or a type of activity associated with a specific piece of equipment or type of clothing.

17. The system of claim 5 wherein said microprocessor is further configured to recognize said event within said data as

detect a first value from said sensor values having a first threshold value and

detect a second value from said sensor values having a second threshold value within a time window;

signify a prospective event;

compare said prospective event to a characteristic signal associated with a typical event and eliminate any false positive events;

signify a valid event if said prospective event is not a false positive event; and

,

save said valid event in said memory comprising information within an event time window as said data.

18. The system of claim 5, wherein said microprocessor is further configured to recognize said at least one sensor element with newly assigned locations after said at least one sensor element is removed from said piece of equipment and coupled with a second piece of equipment of a different type based on said data or said event data.

19. The system of claim 1 wherein said computer further comprises at least one processor in one or more of said mobile device, a mobile phone, a smart phone, a smart watch, a camera, a laptop computer, a notebook computer, a tablet computer, a desktop computer, and a server computer, or any combination of any number of said mobile device, said mobile phone, said smart phone, said smart watch, said camera, said laptop computer, said notebook computer, said tablet computer, said desktop computer and said server computer.

20. The system of claim 1, wherein said computer is further configured to

obtain at least one video from one or more of

a camera associated with said user,

said computer,

said mobile device,

said server, and

another server;

obtain at least one video start time and at least one video stop time associated with said at least one video;

synchronize said event data with said at least one video based on

said event start time and said event stop time, and

at least one time associated with said at least one video, to

create at least one synchronized event video comprising both of

said event data that occurs during a timespan from said event start time to said event stop time;

and,

said at least one video captured during said timespan from said event start time to said event stop time.

21. The system of claim 20 wherein said computer is further configured to display said at least one synchronized event video.

22. The system of claim 20 wherein said computer is further configured to

discard or

instruct another computer to discard or

instruct a camera to discard

at least a portion of said at least one video that occurs outside of said timespan from said event start time to said event stop time.

23. The system of claim 20 wherein said computer is further configured to

communicate with at least one camera;

send a control message

locally to said at least one camera coupled with said computer or

externally to said at least one camera,

to modify video recording parameters of said at least one video associated with said at least one camera based on said data or said event data;

wherein said video recording parameters comprise one or more of frame rate, resolution, color depth, color or grayscale, compression method, compression quality, and recording on or off.

24. The system of claim 20 wherein said computer is further configured to transmit said at least one synchronized event video or a portion of said at least one synchronized event video to one or more of a repository, a viewer, said server, said another server, another computer, a social media site, said mobile device, a network, and an emergency service.

25. The system of claim 20 wherein said computer is further configured to

accept a metric or one or more tags associated with said at least one synchronized event video;

accept selection criteria for said metric or said one or more tags;

determine a matching set of synchronized event videos that have a value or values associated with said metric or with said one or more tags that pass said selection criteria; and

,

display said matching set of synchronized event videos or corresponding thumbnails thereof along with said value or values associated with said metric or with said one or more tags for each of said matching set of synchronized event videos or said corresponding thumbnails.

26. The system of claim 25 wherein said computer is further configured to

generate a highlight reel or fail reel of said matching set of synchronized event videos.

27. The system of claim 1 wherein said computer is further configured to

access previously stored event data associated with said user or piece of equipment or with at least one other user or at least one other piece of equipment;

display information comprising a presentation of said event data associated with said user on a display based on

said event data associated with said user or piece of equipment;

and,

said previously stored event data associated with said user or piece of equipment or with said at least one other user or said at least one other piece of equipment.

28. The system of claim 5 wherein said microprocessor in said at least one sensor element is further configured to transmit said event to at least one other sensor element or other computer, and wherein said at least one other sensor element or other computer is configured to save data or transmit data or both save data and transmit data associated with said event even if said at least one sensor element has not detected said event.

29. The system of claim 1 wherein said computer is further configured to

request or broadcast a request for cameras having locations proximal to said event or oriented to view said event or both having locations proximal to and oriented to view said event;

and,

request at least one video from at least one camera of said cameras, wherein said at least one video contains said event without at least a portion of said at least one video outside of said event start time to said event stop time.