Fusing contextual inferences semantically

• US 9,336,295 B2
• Filed: 03/04/2013
• Issued: 05/10/2016
• Est. Priority Date: 12/03/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Grokop, Leonard Henry, Sadasivam, Shankar, Teague, Edward Harrison
• Primary Examiner(s): Chen, Susan
• Application Number:
  US13/784,229
• Publication Number:
  US 20140156659A1
• Time in Patent Office: 1,163 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: H04M 1/72448 with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion

Abstract

System and methods for performing context inference in a computing device are disclosed. In one embodiment, a method of performing context inference includes: determining, at a computing device, a first context class using context-related data from at least one data source associated with a mobile device; and determining, at the mobile device, a fusion class based on the first context class, the fusion class being associated with at least one characteristic that is common to the first context class and a second context class that is different from the first context class.

Claims

1. A method for performing context inference in a mobile device, the method comprising:

receiving, at a classifier implemented in one or more integrated circuits of the mobile device, sensor data from at least one data source associated with the mobile device, wherein the at least one data source comprises one or more sensors of the mobile device;

determining, by the classifier implemented in the one or more integrated circuits, a first context class for the received sensor data, the first context class corresponding to a context state indicated by the received sensor data;

determining, by the classifier implemented in the one or more integrated circuits, that a confidence value associated with the determination of the first context class is below a threshold value;

creating, by the classifier implemented in the one or more integrated circuits, a fusion class for the received sensor data at least in part by fusing the first context class for the received sensor data and at least a second context class for the received sensor data, the at least the second context class being different from the first context class, wherein the fusion class semantically encompasses the first context class and the at least the second context class and further wherein the fusion class is broader than each of the first context class and the at least the second context class;

substituting, by the classifier implemented in the one or more integrated circuits, the fusion class for the first context class; and

outputting, by the classifier implemented in the one or more integrated circuits, the fusion class as the inferred context of the mobile device.

2. The method of claim 1 wherein the at least one data source comprises one or more of an audio sensor, a transceiver, a motion sensor, a clock, a device usage sensor, a light sensor, a camera, or a calendar.

3. The method of claim 1 wherein determining the first context class further comprises determining the confidence value associated with the determination of the first context class.

4. The method of claim 1 wherein the determining the first context class is an initial classification of the received sensor data.

5. The method of claim 1 wherein the substituting the fusion class is performed in response to the confidence value associated with the determination of the first context class being below the threshold value.

6. The method of claim 4 further comprising identifying an operational property of the initial classification, the operational property including at least one of a motion, a location, an event, or an environment associated with the initial classification.

7. The method of claim 6 wherein the creating the fusion class is based on the operational property of the initial classification.

8. The method of claim 1 comprising:

receiving the sensor data over consecutive time periods;

determining the first context class for the received sensor data associated with a first time period; and

determining the at least the second context class for the received sensor data associated with at least a second time period that is after the first time period.

9. The method of claim 1 wherein determining the first context class comprises selecting the first context class from a plurality of predetermined available context classes.

10. The method of claim 1 wherein the creating the fusion class comprises selecting the fusion class from a plurality of predetermined available fusion classes.

41. The method of claim 1 wherein the creating the fusion class comprises selecting the fusion class from a plurality of predetermined available fusion classes corresponding to a particular application program.

42. The method of claim 8 comprising:

determining, by the classifier implemented in the one or more integrated circuits of the mobile device, a temporal sequence of context classes determined for the consecutive time periods, the temporal sequence of context classes including the first context class and the at least the second context class; and

creating, by the classifier implemented in the one or more integrated circuits of the mobile device, the fusion class for the received sensor data based on the temporal sequence of context classes.

11. A mobile device comprising:

means for receiving sensor data from at least one data source associated with the mobile device, wherein the at least one data source comprises one or more sensors of the mobile device;

means for determining a first context class for the received sensor data, the first context class corresponding to a context state indicated by the received sensor data;

means for determining that a confidence value associated with the determination of the first context class is below a threshold value;

means for creating a fusion class for the received sensor data at least in part by fusing the first context class for the received sensor data and at least a second context class for the received sensor data, the at least the second context class being different from the first context class, wherein the fusion class semantically encompasses the first context class and the at least the second context class and further wherein the fusion class is broader than each of the first context class and the at least the second context class;

means for substituting the fusion class for the first context class; and

means for outputting the fusion class as an inferred context for the mobile device.

12. The mobile device of claim 11 wherein the at least one data source comprises one or more of an audio sensor, a transceiver, a motion sensor, a clock, a device usage sensor, a light sensor, a camera, or a calendar.

13. The mobile device of claim 11 comprising means for determining the confidence value associated with the determination of the first context class.

14. The mobile device of claim 11 wherein the determination of the first context class is an initial classification of the received sensor data.

15. The mobile device of claim 11 wherein the means for substituting the fusion class is configured to substitute the fusion class in response to the confidence value associated with the determination of the first context class being below the threshold value.

16. The mobile device of claim 14 comprising means for identifying an operational property of the initial classification, the operational property including at least one of a motion, a location, an event, or an environment associated with the initial classification.

17. The mobile device of claim 16 comprising means for creating the fusion class based on the operational property of the initial classification.

18. The mobile device of claim 11 comprising:

means for receiving the sensor data over consecutive time periods;

means for determining the first context class for the received sensor data associated with a first time period; and

means for determining the at least the second context class for the received sensor data associated with at least a second time period that is after the first time period.

19. The mobile device of claim 11 wherein the means for determining the first context class is configured to determine the first context class by selecting the first context class from a plurality of predetermined available context classes.

20. The mobile device of claim 11 wherein the means for creating the fusion class is configured to create the fusion class by selecting the fusion class from a plurality of predetermined available fusion classes.

21. A mobile device comprising:

at least one data source comprising one or more sensors of the mobile device;

a classifier built into one or more hardware modules of the mobile device, the one or more hardware modules comprising one or more integrated circuits, and communicatively coupled to the at least one data source, wherein the classifier is configured to;

receive sensor data from the at least one data source;

determine a first context class for the received sensor data, the first context class corresponding to a context state indicated by the received sensor data;

determine that a confidence value associated with the determination of the first context class is below a threshold value;

create a fusion class for the received sensor data at least in part by fusing the first context class for the received sensor data and at least a second context class for the received sensor data, the at least the second context class being different from the first context class, wherein the fusion class semantically encompasses the first context class and the at least the second context class and further wherein the fusion class is broader than each of the first context class and the at least the second context class;

substitute the fusion class for the first context class; and

output the fusion class as an inferred context for the mobile device.

22. The mobile device of claim 21 wherein the at least one data source comprises one or more of an audio sensor, a transceiver, a motion sensor, a clock, a device usage sensor, a light sensor, a camera, or a calendar.

23. The mobile device of claim 21 wherein the classifier is configured to determine the confidence value associated with the determination of the first context class.

24. The mobile device of claim 21 wherein the determination of the first context class is an initial classification of the received sensor data.

25. The mobile device of claim 21 wherein the classifier is configured to substitute the fusion class in response to the confidence value associated with the determination of the first context class being below the threshold value.

26. The mobile device of claim 24 wherein the classifier is configured to identify an operational property of the initial classification, the operational property including at least one of a motion, a location, an event, or an environment associated with the initial classification.

27. The mobile device of claim 26 wherein the classifier is configured to create the fusion class based on the operational property of the initial classification.

28. The mobile device of claim 21 wherein the classifier is further configured to:

receive the sensor data over consecutive time periods;

determine the first context class for the received sensor data associated with a first time period; and

determine the at least the second context class for the received sensor data associated with at least a second time period that is after the first time period.

29. The mobile device of claim 21 wherein the classifier is configured to determine the first context class by selecting the first context class from a plurality of predetermined available context classes.

30. The mobile device of claim 21 wherein the classifier is configured to create the fusion class by selecting the fusion class from a plurality of predetermined available fusion classes.

31. A non-transitory processor-readable storage medium comprising processor-readable instructions configured to cause a processor to:

receive, using a classifier implemented in one or more integrated circuits of a mobile device, sensor data from at least one data source associated with a mobile device, the at least one data source comprising one or more sensors of the mobile device;

determine, using the classifier implemented in the one or more integrated circuits, a first context class for the received sensor data, the first context class corresponding to a context state indicated by the received sensor data;

determine, using the classifier implemented in the one or more integrated circuits, that a confidence value associated with the determination of the first context class is below a threshold value; and

create, using the classifier implemented in the one or more integrated circuits, a fusion class for the received sensor data at least in part by fusing the first context class for the received sensor data and at least a second context class for the received sensor data, the at least the second context class being different from the first context class, wherein the fusion class semantically encompasses the first context class and the at least the second context class and further wherein the fusion class is broader than each of the first context class and the at least the second context class;

substitute, using the classifier implemented in the one or more integrated circuits, the fusion class for the first context class; and

output, using the classifier implemented in the one or more integrated circuits, the fusion class as an inferred context for the mobile device.

32. The storage medium of claim 31 wherein the at least one data source comprises one or more of an audio sensor, a transceiver, a motion sensor, a clock, a device usage sensor, a light sensor, a camera, or a calendar.

33. The storage medium of claim 31 further comprising instructions configured to cause the processor to determine the confidence value associated with the determination of the first context class.

34. The storage medium of claim 31 wherein the determination of the first context class is an initial classification of the received sensor data.

35. The storage medium of claim 31 wherein the instructions configured to cause the processor to substitute the fusion class are configured to cause the processor to substitute the fusion class in response to the confidence value associated with the determination of the first context class being below the threshold value.

36. The storage medium of claim 34 further comprising instructions configured to cause the processor to identify an operational property of the initial classification, the operational property including at least one of a motion, a location, an event, or an environment associated with the initial classification.

37. The storage medium of claim 36 wherein instructions configured to cause the processor to create the fusion class are configured to cause the processor to create the fusion class based on the operational property of the initial classification.

38. The storage medium of claim 31 further comprising instructions configured to cause the processor to:

receive the sensor data over consecutive time periods;

determine the first context class for the received sensor data associated with a first time period; and

determine the at least the second context class for the received sensor data associated with at least a second time period that is after the first time period.

39. The storage medium of claim 31 wherein the instructions configured to cause the processor to determine the first context class are configured to cause the processor to select the first context class from a plurality of predetermined available context classes.

40. The storage medium of claim 31 wherein the instructions configured to cause the processor to create the fusion class are configured to cause the processor to select the fusion class from a plurality of predetermined available fusion classes.