System and method for multi-device continuum and seamless sensing platform for context aware analytics

• US 10,397,355 B2
• Filed: 04/26/2016
• Issued: 08/27/2019
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: American University of Beirut Incorporated
• Original Assignee: American University of Beirut Incorporated
• Inventors: Hajj, Hazem, El Hajj, Wassim, Eid, Saeid, Bensaleh, Mohammed Sulaiman, Obeid, Abdulfattah Mohammad, Qasim, Syed Manzoor
• Primary Examiner(s): Mobin, Hasanul
• Application Number:
  US15/139,292
• Publication Number:
  US 20170124110A1
• Time in Patent Office: 1,218 Days
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/567 Integrating service provisioning from a plurality of service providers
• CPC Class Codes: H04W 4/38 for collecting sensor information

Abstract

The Multi-Device Continuum and Seamless Sensing Platform for Context Aware Analytics provide a platform for continuous sensing across multiple devices towards a unified target. The Multi-Device Continuum and Seamless Sensing Platform provides a platform for extracting, loading, integrating, and tracking related data across multiple smart devices capable of integrating with internal and external sensors, such as wearable devices. The Multi-Device Continuum and Seamless Sensing Platform develop context aware solutions, which are targeted at automated recognition of context extracted from users'"'"' devices, as people are often always interacting with a digital device such as phone, tablet, or desktop.

Claims

1. A multi-device continuum and seamless sensing platform system comprising:

a processor; and

memory storing computer readable instructions, a software application that, when executed, cause the system to;

collect sensing data for the user from a plurality of digital devices and continuously over time and space by a sensing application from a plurality of sensors including a wearable sensor, a phone sensor, and a sensor application providing user related data;

store the data for the user from the plurality of digital devices in a centric integrated system including a storage system and a processing system;

align the data for the user in a FactLog table, an Application Log Report, a query joining an Accelerometer record and a Magnetometer record, and a database Schema for an Activity Recognition Model;

synchronize the collected sensing data from the sensing application to an integrated system; and

process the aligned and synchronized sensing data across time by using a time stamp when data is collected from each sensor and space by keeping track of each device in a database represented by the FactLog Table and the database Schema for the Activity Recognition Model,

wherein the database includes data synchronization, a plurality of aggregation queries for joining sensing data in a plurality of time and fact tables represented by the FactLog table, and a trigger for inserting complementary data for a new sensing data point, and automatically keeping track of the plurality of devices and the sensing data a user is using, and installing a plurality of soft applications on the plurality of digital devices to collect digital data and receive data from sensors providing data related to the user in a device data table;

wherein the plurality of digital devices are selected from the group consisting of compute, sensing and personal devices;

a mechanism to synchronize data at different time granularity levels for data coming from different sources with different time stamps in an Application Log Report and the query joining the Accelerometer word and the Magnetometer record, wherein the mechanism provides for data alignment and that is independent of how the data is processed, a database design consisting of the FactLog table and a time granularity table joined with a logging table selected from the group of an action table or a communication table from the FactLog table, to result in an aggregation over space and time for data analysis and analytics; and

the time granularity table is populated through a trigger set on the other tables where every new record gets inserted in the logs tables, the trigger set fires and inserts the new records in the time granularity table consisting of the time stamp of the original record and divided into a respective attribute in the granularity table; and

the time granularity table is used in the aggregation queries for data mining and analytics from the query joining the Accelerometer record and the Magnetometer record.

2. The system of claim 1, wherein the sensing data is selected from the group consisting of:

bio-metric data, browsing activity, application log on the plurality of devices, keyboard log including a number of correction keystrokes (backspace and delete) in the past second, a number of navigation keystrokes in the past second and a number of alphanumerical keystrokes in the past second, communication log, emails and messages, social networking activity, mouse events including a number of mouse clicks in the past second, a number of mouse move events in the past second, and a number of mouse wheel move events in the past second, physiological data, environmental data, and digital wallet data.

3. The system of claim 2, wherein the digital devices are selected from the group consisting of:

a laptop computer, a mobile phone, a table, a health care device, a smart watch, or a sensing device; and

wherein the plurality of digital devices are data collectors and receiving ends of data, and the soft applications include a plug-in capability to enable the user to select from the plurality of sensors.

4. The system of claim 3, wherein the collected data comprises a Device ID, name, Kernel Version, and Model number, wherein the plurality of sensors are registered in the FactLog Table and saved as the personal devices of the user along with a metadata in the database;

wherein the metadata includes;

software installed, kernel version and the version of installed sensing application; and

the system provides a plurality of queries to synchronize and aggregate the data with the plurality of digital devices.

5. The system of claim 4, further comprising analyzing and reporting the collected data by the integrated system;

wherein the normalization, calibration and calculation of a plurality of statistical features are done directly on the plurality of sensing devices before data transmission and saved at the received end of the data transmission.

6. The system of claim 5, wherein the collected data is saved in the database Schema for Activity Recognition Model and the FactLog Table collected for the user;

and a database schema design is presenting a type of the user'"'"'s data and how they are installed, where the FactLog Table is provided in tables selected from the group consisting of an activity log table, an action log table, an application log table, a communication log table, a location log table, a time granularity table, a keyboard log, the FactLog table, a call log table, a magnetometer log table, an accelerometer log table, and an external sensor log table.

7. A multi-device continuum and seamless sensing platform apparatus comprising:

a processor; and

memory operatively coupled to the processor and storing computer readable instructions that, when executed, cause the apparatus to;

receive collected data for the user from a plurality of digital devices continuously over time and space from a plurality of sensors including a wearable sensor, a phone sensor, and a sensor application providing user related data;

store collected data for the user from the plurality of devices in an integrated system;

align the data for the user in a FactLog table, an Application Log Report, a query joining an Accelerometer record and a Magnetometer record, and a database Schema for an Activity Recognition Model;

synchronize the collected data from the sensing application across the plurality of digital devices and the integrated system; and

process the aligned and synchronized sensing data across time by using a time stamp when data is collected from each sensor and space by keeping track of each device in a database represented by FactLog Table and the database Schema for the Activity Recognition Model,

wherein the database includes data synchronization, a plurality of aggregation queries for joining sensing data in a plurality of time and fact tables represented by the FactLog table, and a trigger for inserting complementary data for a new sensing data point; and

an apparatus to automatically keeping track of the plurality of devices and the sensing data a user is using, and installing a plurality of soft applications on the plurality of digital devices to collect digital data and receive data from sensors providing data related to the user in a device data table;

wherein the plurality of digital devices are selected from the group consisting of compute, sensing and personal devices

a mechanism to synchronize data at different time Granularity levels for data coming from different sources with different time stamps in an Application Log Report and the query joining the Accelerometer record and the Magnetometer record, wherein the mechanism provides for data alignment and that is independent of how the data is processed, a database design consisting of the FactLog table and a time granularity table joined with a logging table selected from the group of an action table or communication table from the FactLog table, to result in an aggregation over space and time for data analysis and analytics; and

the time granularity table is populated through a trigger set on the other tables where every new record gets inserted in the loos tables, the tripper set fires and inserts the new records in the time granularitv table consisting of the time stamp of the original record and divided into a respective attribute in the granularity table; and

the time granularity table is used in the aggregation queries for data minim and analytics from the query joining the Accelerometer record and the Magnetometer record.

8. The apparatus of claim 7, wherein the collected data is the device data table selected from the group consisting of:

bio-metric data, browsing activity, application log on the plurality of devices, keyboard log including a number of correction keystrokes (backspace and delete) in the past second, a number of navigation keystrokes in the past second and a number of alphanumerical keystrokes in the past second, communication log, emails and messages, social networking activity, mouse events including a number of mouse clicks in the past second, a number of mouse move events in the past second, and a number of mouse wheel move events in the past second, physiological data, environmental data, and digital wallet data.

9. The apparatus of claim 8, wherein the digital devices are selected from the group consisting of:

a laptop computer, a mobile phone, a table, a health care device, a smart watch, or a sensing device; and

wherein the plurality of digital devices are collectors and receiving ends, and the soft applications include a plug-in capability to enable the user to select from the plurality of sensors.

10. The apparatus of claim 9, wherein the collected data comprises a Device ID, name, Kernel Version, and Model number, wherein sensors are registered and saved in the FactLog Table as the personal devices of the user along with a metadata in the database;

wherein the metadata includes;

software installed, kernel version and the version of installed sensing application; and

the system provides a plurality of queries to synchronize and aggregate the data with the plurality of digital devices.

11. The apparatus of claim 10, wherein the integrated system further analyzes and reports the collected data to the plurality of digital devices;

wherein the normalization, calibration and calculation of a plurality of statistical features are done directly on the plurality of sensing devices before data transmission and saved at the received end of the data transmission.

12. The apparatus of claim 11, wherein the collected data is saved in an elaborate description for the various types of tables collected for the user;

and a database schema design is presenting a type of the user'"'"'s data and how they are installed, where the database is provided in tables selected from the group consisting of an activity log table, an action log table, an application log table, a communication log table, a location log table, a time granularity table, a keyboard log, the FactLog table, a call log table, a magnetometer log table, an accelerometer log table, and an external sensor log table.