Information processing using a population of data acquisition devices

• US 10,045,321 B2
• Filed: 12/30/2011
• Issued: 08/07/2018
• Est. Priority Date: 12/30/2010
• Status: Active Grant

• Current Assignee: Ambientz, ST FamTech, LLC
• Original Assignee: Staton Techiya, LLC
• Inventors: Goldstein, Steven W.
• Primary Examiner(s): Wyche, Myron K
• Application Number:
  US13/976,636
• Publication Number:
  US 20140018097A1
• Time in Patent Office: 2,412 Days
• CPC Class Codes: G06F 11/3013 where the computing system is an embedded system, i.e. a combination of hardware and software dedicated to perform a certain function in mobile devices, printers, automotive or aircraft systems testing or monitoring of control systems or parts thereof G05B23/02
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 50/80 for detecting, monitoring or modelling epidemics or pandemics, e.g. flu
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

Distributed systems, controllers and methods for processing information from a plurality of devices are provided. A distributed system includes a plurality of devices distributed in an environment. Each device has at least a communication capability for interchanging information with others of the devices and/or with a communication system. Each of at least some of the devices has one or more sensors for acquiring sensor data related to the environment proximate to the device. At least one of the communication system or one or more of the devices is configured as a controller configured to: select a subset of devices from among the plurality of devices, receive information based on the acquired sensor data of the selected subset, and combine the received information from the selected subset to determine a characteristic of the environment proximate to one or more of the devices.

Claims

1. A system comprising:

a device among a plurality of devices distributed in an environment, wherein the device has a communication capability for interchanging information with others among the plurality of devices as part of a mobile ad hoc network, a communication system, or a combination thereof, and

wherein the device has a sensor for acquiring sensor data related to the environment proximate to the device including at least one microphone used for detecting keywords or used for detecting non-speech acoustic events;

wherein one or more of the devices among the plurality including the device is configured as a controller on an ad hoc basis, wherein the controller performs operations comprising;

selecting a subset of devices from among the plurality of devices for collection of the sensor data,

receiving information based on the acquired sensor data of the selected subset of devices using the mobile ad hoc network and a location of the one or more devices, and

aggregating the received information including audio information from the at least one microphone for detecting keywords or non-speech acoustic events and at least sensor data from at least one non-acoustic sensor from the selected subset of devices to determine a characteristic of the environment proximate to the one or more of the devices;

adjusting a manner in which the subset of devices operate based on the characteristic of the environment and the information aggregated from the at least one microphone and the at least one non-acoustic sensor; and

wherein the system adaptively enables or configures the selected subset of devices according to triggering events detected at one or more of the plurality of devices using the at least one microphone, and at least a biometric sensor among a heart rate sensor, a blood pressure sensor, a toxin detection sensor, a nutrient detection sensor or a pheromone detection sensor or at least one among a location sensor, a light detection sensor, a proximity sensor, a gravity detection sensor, a magnetic field detection sensor, an electrical field detection sensor, a vibration sensor, a pressure sensor, a humidity sensor, or a moisture sensor.

2. The system of claim 1, wherein the system adaptively enables or configures the subset of devices according to triggering events detected at one or more of the plurality of devices using the at least one microphone, the biometric sensor, the location sensor and at least one among the light detection sensor, the proximity sensor, the gravity detection sensor, the magnetic field detection sensor, the electrical field detection sensor, the vibration sensor, the pressure sensor, the humidity sensor, or the moisture sensor.

3. The system of claim 1, wherein at least one of the devices includes at least one of a wireless device or a wired device and wherein the system enables the switching of a subset of devices on or off based on information about context, including position, orientation in space, social-graph membership, and demographics and the processing and analyzing of information obtained from the sensors, and further setting of triggers to enable or disable sensing, processing or analysis.

4. The system of claim 1, wherein at least one of the devices includes a fixed device a mobile device, or a combination thereof.

5. The system of claim 1, wherein at least one of the devices includes a mobile telephone device, an ear-piece, a hearing aid, a navigation device, a television remote, a car key fob, a portable game controller, GPS device, a computer or sensor module having at least one sensor, or a combination thereof.

6. The system of claim 1, wherein the sensor includes the at least one microphone, the biometric sensor, and at least one among a motion-related sensor, a temperature sensor, a compass, or an image sensor.

7. The system of claim 1, wherein at least one of the devices includes a position module for determining a position of the respective device.

8. The system of claim 1, wherein each of at least some of the devices include a local data analysis module for processing the acquired sensor data to determine at least one event, the event being used by the controller selected by the ad hoc network for determining the characteristic of the environment.

9. The system of claim 8, wherein the local data analysis module includes an acoustic analysis module to process audio information including processing of speech and non-speech events.

10. The system of claim 8, wherein the local data analysis module includes a sensor analysis module to process non-audio related information.

11. The system of claim 1, wherein the controller selected by the ad hoc network is configured to determine locations for at least some of the devices.

12. The system of claim 11, wherein the controller selected by the ad hoc network is configured to select the subset of devices according to at least the determined locations and the received information aggregated from among the plurality of devices.

13. The system of claim 1, wherein the controller selected by the ad hoc network is further configured to pass configuration information to the devices for acquiring the sensor data, processing of the sensor data at the respective devices, or a combination thereof.

14. The system of claim 1, wherein the mobile ad hoc network is used to fine tune acoustical characteristics in a concert hall based on data acquired from at least a subset of devices from the mobile ad hoc network.

15. The system of claim 2, wherein the mobile ad hoc network is used to estimate traffic flow from noise occurring the environment.

16. The system of claim 1, wherein the environment includes an audio scene and a characteristic of the audio scene includes speech events, non-speech events to determine an emotional state, or a combination thereof.

17. The system of claim 16, wherein when the system is a distributed system that records speech from a user not associated with the distributed system, the distributed system provides a warning indication to all devices in a vicinity of the device recording the speech to indicate that audio information is being recorded.

18. The system of claim 17 wherein the distributed system further determines trend information across a population of users of the devices from the aggregated information.

19. The system of claim 17, wherein the distributed system further monitors localization of events, tracking of events, topics and triggering events, or a combination thereof, across a population of users of the devices from the aggregated information.

20. The system of claim 17, wherein the distributed system further determines aggregated motion-related characteristics related to multiple ones of the devices from the aggregated information.

21. The system of claim 20, wherein the aggregated motion-related characteristics comprise human or vehicle flow characteristics.

22. The system of claim 1, wherein at least one of the devices enables the fine tuning of an acoustical characteristic using a subset of devices from the ad hoc network according to a triggering event detected at one or more of the devices of the mobile ad hoc network.

23. The system of claim 1, wherein the controller selected by the ad hoc controller receives the information from at least one of the devices in the subset of devices responsive to a confirmation indication from the respective device, the confirmation indication indicating an allowance to release the respective information to the controller.

24. An ad hoc controller for interacting with a plurality of devices distributed in an environment, wherein each device has a communication capability for interchanging information with others of the devices as part of a mobile ad hoc network, the ad hoc controller, with a communication system, or a combination thereof, wherein each of at least some of the devices has a sensor for acquiring sensor data related to the environment proximate to the device, wherein the ad hoc controller comprises:

a selection/acquisition module configured on an ad hoc basis to select a subset of devices from among the plurality of devices using the mobile ad hoc network and the sensor and to receive information based on the acquired sensor data of the selected subset of devices, the acquired sensor data being acquired from at least one or more microphones used for detecting at least keywords, and from at least a biometric sensor among a heart rate sensor or a blood pressure sensor; and

a scene analysis module configured to combine the information received from the selected subset of devices to determine a characteristic of the environment proximate to one or more of the devices and configured to adjust a manner in which the subset of devices operate based on the characteristic of the environment and the information combined from the at least one or more microphones and the biometric sensor.

25. The ad hoc controller of claim 24, wherein the ad hoc controller is configured to determine locations for at least some of the devices.

26. The ad hoc controller of claim 25, wherein the selection/acquisition module is configured to select the subset of devices according to the determined locations, the received information from among the plurality of devices, or a combination thereof.

27. The ad hoc controller of claim 24, wherein the selection/acquisition module is further configured to pass configuration information to the devices for acquiring the sensor data, processing of the sensor data at the respective devices, or a combination thereof.

28. The ad hoc controller of claim 24, wherein the selection/acquisition module is configured to adjust a selection of devices in the subset of devices, a configuration of the devices in the subset, or a combination thereof, in response to the characteristic of the environment determined by the scene analysis module.

29. The ad hoc controller of claim 24, further comprising a sensor module including at least one of the one or more microphones, the biometric sensor, and a gravity detection sensor, a pheromone detection sensor, or a combination thereof.

30. The ad hoc controller of claim 29, further comprising a local data analysis module for processing sensor data from the sensor module, the processed sensor data being combined with the information received from the selected subset of devices to determine the characteristic of the environment.

31. The ad hoc controller of claim 24, wherein the ad hoc controller is configured to mitigate, amplify, or pass-through information to one or more of the plurality of devices responsive to the characteristic of the environment.

32. A method at an ad hoc controller for processing information from a plurality of devices in a mobile ad hoc network distributed in an environment, the method comprising:

selecting a subset of devices from among the plurality of devices by the ad hoc controller on an ad hoc basis using the mobile ad hoc network and based on sensor data collected, wherein the ad hoc controller includes a communication system, a device, or a combination thereof, wherein each device has a communication capability for interchanging information with others among the devices the communication system, or a combination thereof, wherein each of at least some of the devices has a sensor for acquiring sensor data related to the environment proximate to the device including at least one microphone used for detecting keywords and used for detecting non-speech acoustic events, and at least a biometric sensor;

acquiring sensor data by the selected subset of devices;

receiving information based on the acquired sensor data of the selected subset of devices using the mobile ad hoc network;

combining, by the ad hoc controller, the received information from the selected subset of devices to determine a characteristic of the environment proximate to one or more of the devices; and

adjusting a manner in which the selected subset of devices operate based on the characteristic of the environment, an aggregation of the acquired sensor data from the at least one microphone used for detecting keywords and used for detecting non-speech acoustic events, and the biometric sensor.

33. The method of claim 32, wherein the receiving of the information includes receiving the information buffered at the devices prior to receiving a request from the ad hoc controller for the information.

34. The method of claim 32, wherein the acquiring of the sensor data includes processing the acquired sensor data to determine at least one event, the processed data being used to determine the characteristic of the environment.

35. The method of claim 32, the method further including determining trend information across a population of users of the devices from the combined information.

36. The method of claim 32, the method further including monitoring localization of events, tracking of events, topics and triggering events, or a combination thereof, across a population of users of the devices from the combined information.

37. The method of claim 32, the method further including determining aggregated motion-related characteristics related to multiple ones of the devices from the combined information.

38. The method of claim 32, the method further including adjusting a selection of devices in the subset of devices, a configuration of the devices in the subset, or a combination thereof, responsive to the characteristic of the environment.

39. The method of claim 32, wherein the information from at least one of the devices in the subset of devices is received responsive to a confirmation indication from the respective device, the confirmation indication indicating an allowance to release the respective information to the ad hoc controller.