Sensory enhancement systems and methods in personal electronic devices

• US 8,390,445 B2
• Filed: 03/01/2012
• Issued: 03/05/2013
• Est. Priority Date: 02/01/2006
• Status: Active Grant

• Current Assignee: Innovation Specialists LLC
• Original Assignee: Innovation Specialists LLC
• Inventors: Betts, William L., Betts, Carol
• Primary Examiner(s): Wu, Daniel
• Assistant Examiner(s): Terrell, Emily C
• Application Number:
  US13/409,220
• Publication Number:
  US 20120154144A1
• Time in Patent Office: 369 Days
• Field of Search: 340/539.11, 340/539.26
• US Class Current: 340/539.26
• CPC Class Codes: G01N 33/0075 for multiple spatially distributed sensors, e.g. for environmental monitoring transmission systems for measured values G08C
• CPC Class Codes: G06K 7/10366 the interrogation device being adapted for miscellaneous applications
• CPC Class Codes: G08B 13/1427 with transmitter-receiver for distance detection
• CPC Class Codes: G08B 17/00 Fire alarms; Alarms responsive to explosion
• CPC Class Codes: G08B 21/0263 System arrangements wherein the object is to detect the direction in which child or item is located
• CPC Class Codes: G08B 21/0275 Electronic Article Surveillance [EAS] tag technology used for parent or child unit, e.g. same transmission technology, magnetic tag, RF tag, RFID
• CPC Class Codes: G08B 21/0286 Tampering or removal detection of the child unit from child or article
• CPC Class Codes: G08B 21/0415 detecting absence of activity per se
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 21/0446 worn on the body to detect changes of posture, e.g. a fall, inclination, acceleration, gait
• CPC Class Codes: G08B 21/088 by monitoring a device worn by the person, e.g. a bracelet attached to the swimmer
• CPC Class Codes: G08B 21/12 responsive to undesired emission of substances, e.g. pollution alarms
• CPC Class Codes: G08B 25/009 Signalling of the alarm condition to a substation whose identity is signalled to a central station, e.g. relaying alarm signals in order to extend communication range
• CPC Class Codes: G08B 25/016 Personal emergency signalling and security systems emergency non-personal manually actuated alarm activators G08B25/12
• CPC Class Codes: G08B 29/181 due to failing power supply
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: H04M 1/026 Details of the structure or mounting of specific components
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/10 : including a GPS signal receiver
• CPC Class Codes: H04M 2250/12 : including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04R 2430/20 : Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 3/005 : for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Disclosed are personal electronic devices (PEDs) having a sensory enhancement (SE) system for monitoring environmental conditions and detecting environmental events, for example but not limited to, changes in acoustic, thermal, optical, electromagnetic, chemical, dynamic, wireless, atmospheric, or biometric conditions. The detection of such events can be used to invoke a notification, an alert, a corrective action, or some other action, depending upon the implementation to the PED user or another party.

Claims

1. A non-transitory computer readable medium comprising computer program code instructions for a personal electronic device (PED) that can be transported with a user, the PED having a computer based architecture with a processor designed to execute the computer program code instructions, the PED designed to perform a first electronic based intelligence function, the first electronic based intelligence function enabling the user to initiate and receive wireless telephone calls over a telephone network, the computer program code instructions designed to enable the PED to perform a second electronic based intelligence function that is different than the first electronic based intelligence function, the second electronic based intelligence function involving detection of an event in an environment associated with the PED, the PED comprising a transducer designed to sense an environmental condition, the computer program code instructions comprising:

instructions designed to detect the event in the environment associated with the PED by comparing sensed data from the transducer with reference data and determining detection of the event based upon the sensed data and the reference data, the computer program code instructions comprising instructions for initiating a notification upon detection of the event, the computer program code instructions comprising instructions for acquiring the reference data from the environment associated with the PED with the transducer and storing the reference data in memory associated with the PED prior to detecting the event;

detection engine software comprising instructions that correlate the sensed data and the reference data to produce a correlation value that is indicative of a relationship between the sensed data and the reference data; and

instructions that initiate a detection indication when the correlation value is above a predefined threshold; and

wherein the correlation is performed by storing the reference data in the frequency domain;

storing the sensed data in the time domain;

converting the sensed data from the time domain to the frequency domain by performing a fast Fourier transform on the sensed data that is stored in the time domain;

comparing the frequency domain sensed data with the frequency domain reference data to produce a correlation result;

determining detection of the event when the correlation result surpasses the predefined threshold, which indicates that the frequency domain sensed data matches the frequency domain reference data; and

wherein the frequency domain sensed data and the frequency domain reference data are each numerical arrays of magnitudes or vectors.

2. The medium of claim 1, wherein the sensed data and the reference data are each numerical arrays of magnitudes or vectors that can be compared and that were derived by sampling an environmental signal over respective time periods.

3. The medium of claim 1, wherein the computer program code instructions further comprise instructions designed to suspend operation of the first electronic based intelligence function while the notification is produced.

4. The medium of claim 1, wherein the computer program code instructions further comprise user interface instructions that enable the user to input the predefined threshold.

5. The medium of claim 1, wherein the computer program code instructions further comprise:

instructions that collect and store a historical trend of the sensed data;

instructions that accumulate the historical trend data over a predefined time period;

instructions that differentiate the historical trend data to determine rate of change over a predefined time period;

instructions that compare the trend data or accumulated trend data with predefined thresholds; and

instructions that initiate a detection indication based upon one of the comparison.

6. The medium of claim 1, wherein the data sensed by the transducer is the sound pressure level within the ear of the user and the computer program code instructions comprise:

instructions to automatically reduce the power of an audio device driven by the first electronic based intelligence function while maintaining safe audible sound to the user from the audio device driven by the first electronic based intelligence function.

7. The medium of claim 1, wherein the PED is a wireless telephone that is designed to operate in a plurality of different modes of operation that are selectable by a user via a user interface comprising a keyboard and a display, wherein the first electronic based intelligence function is performed during a first mode of operation that can be selected by the user via the user interface, and wherein the second electronic based intelligence function is performed during a second mode of operation that can be selected by the user via the user interface.

8. The medium of claim 1, wherein the computer program code instructions further comprise:

instructions designed to enable the PED to communicate with one or more other PEDs;

instructions designed to exchange information related to the event between or among the PEDs; and

wherein the instructions designed to detect the event perform the detection based at least in part upon information received from the one or more other PEDs.

9. The medium of claim 1, wherein the environmental event is a moving object, wherein the transducer is a microphone, and wherein the computer program code instructions further comprise:

instructions designed to determine a Doppler shift associated with the moving object; and

instructions designed to determine a speed of the moving object based upon the Doppler shift.

10. The medium of claim 1, wherein the computer program code instructions further comprises instructions designed to determine a direction associated with an event based upon data sensed by at least two transducers, the at least two transducers including at least two microphones.

11. The medium of claim 1, wherein the environmental event is a change in an acoustic, thermal, optical, dynamic, or chemical condition and wherein the computer program code instructions further comprise instructions designed to define a range around the PED, inside of which the environmental event may be detected and outside of which the environmental event cannot be detected.

12. The medium of claim 11, wherein the transducer designed to sense an environmental condition is a radio frequency identification (RFID) tag or transponder and wherein the computer program code instructions further comprise:

instructions designed to determine the proximity or distance of the RFID tag from the PED; and

/or

instructions designed to measure one or more environmental conditions in the vicinity of the RFID tag.

13. The medium of claim 12, wherein the RFID tag is attached to a child.

14. The medium of claim 12, wherein the associated RFID tag is concealed.

15. The medium of claim 12, wherein the associated RFID tag is attached with an interlocking mechanism that has a positive indication of attachment and the computer program code instructions further comprise:

instructions that compute the proximity or distance of the RFID tag from the PED;

instructions for initiating a notification if the distance of the RFID tag exceeds a predefined threshold;

instructions that detect any interruption of the positive indication of attachment; and

instructions for initiating a notification upon interruption of the positive indication of attachment.

25. The medium of claim 1, further comprising instructions enabling the user to associate an identifier for the reference data after acquisition.

26. The medium of claim 25, wherein the identifier is an image indicative of the event.

16. A non-transitory computer readable medium comprising computer program code instructions for a personal electronic device (PED) that can be transported with a user, the PED having a computer based architecture with a processor designed to execute the computer program code instructions, the PED having a user interface comprising a keyboard and a display, the PED designed to be operable in a plurality of modes of operation that can be selected by a user via the user interface, the PED designed to perform a first electronic based intelligence function during a first mode of operation, the first electronic based intelligence function enabling the user to initiate and receive wireless telephone calls over a telephone network, the computer program code instructions designed to enable the PED to perform a second electronic based intelligence function that is different than the first electronic based intelligence function during a second mode of operation, the second electronic based intelligence function involving detection of an event in an environment associated with the PED, the PED comprising a plurality of transducers designed to sense environmental conditions, at least one of the plurality of transducers being operable during each mode of operation, the computer program code instructions comprising:

instructions designed to detect the event in the environment associated with the PED by comparing sensed data from the transducer with reference data and determining detection of the event based upon the sensed data and the reference data, the computer program code instructions comprising instructions for initiating a notification upon detection of the event, wherein the computer program code instructions comprise instructions designed to engage in a communication session with a remote computer and access the reference data from the remote computer;

detection engine software comprising instructions that correlate the sensed data and the reference data to produce a correlation value that is indicative of a relationship between the sensed data and the reference data; and

instructions that initiate a detection indication when the correlation value is above a predefined threshold; and

wherein the correlation is performed by storing the reference data in the frequency domain;

storing the sensed data in the time domain;

converting the sensed data from the time domain to the frequency domain by performing a fast Fourier transform on the sensed data that is stored in the time domain;

comparing the frequency domain sensed data with the frequency domain reference data to produce a correlation result;

determining detection of the event when the correlation result surpasses the predefined threshold, which indicates that the frequency domain sensed data matches the frequency domain reference data; and

wherein the frequency domain sensed data and the frequency domain reference data are each numerical arrays of magnitudes or vectors.

17. The medium of claim 16, further comprising computer program code instructions designed to perform the first electronic based intelligence function.

18. The medium of claim 16, wherein the computer program code instructions enable the user to select which of the at least one of the plurality of transducers is operational via the user interface.

19. The medium of claim 16, wherein the computer program code instructions further comprise instructions designed to suspend operation of the first electronic based intelligence function while the notification is produced.

20. The medium of claim 16, wherein the computer program code instructions further comprise user interface instructions that enable the user to input the predefined threshold via the user interface.

21. The medium of claim 16, wherein the computer program code instructions further comprise:

instructions designed to enable the PED to communicate with one or more other PEDs;

instructions designed to exchange information related to the event between or among the PEDs; and

wherein the instructions designed to detect the event perform the detection based at least in part upon information received from the one or more other PEDs.

22. The medium of claim 16, wherein the event is a moving object and wherein the computer program code instructions further comprise:

instructions designed to determine a Doppler shift associated with the moving object; and

instructions designed to determine a speed of the moving object based upon the Doppler shift.

23. The medium of claim 16, wherein the computer program code instructions further comprises instructions designed to determine a direction associated with an event based upon data sensed by at least two transducers.

24. The medium of claim 16, wherein the environmental event is a change in an acoustic, thermal, optical, dynamic, or chemical condition and wherein the computer program code instructions further comprise instructions designed to define a range around the PED, inside of which the environmental event may be detected and outside of which the environmental event cannot be detected.

27. A computer program having computer program code instructions stored on a non-transitory computer readable medium, the computer program for a personal electronic device (PED) that can be transported with a user, the PED having a computer based architecture with one or more processors designed to execute the computer program code instructions, the PED comprising one or more transducers designed to sense an environmental condition in a local environment associated with the PED, the PED designed to perform a first electronic based intelligence function without use of the one or more transducers, the first electronic based intelligence function enabling the user to initiate and receive wireless telephone calls over a telephone network, the computer code instructions comprising:

(a) instructions designed to enable the PED to perform a second electronic based intelligence function that is different than the first electronic based intelligence function, the second electronic based intelligence function involving detection of the environmental condition in the local environment associated with the PED;

(b) instructions designed to detect the environmental condition comprising;

(1) instructions designed to cause sensed data from a transducer to be compared with one or more of a plurality of reference signature data, each reference signature data comprising a numerical array of magnitudes or vectors that can be compared and that are representative of an environmental signal, the environmental signal corresponding to a change in an acoustic, thermal, optical, dynamic, or chemical condition;

(2) instructions designed to detect the environmental condition in the local environment based upon a sufficient degree of resemblance of the sensed data with one of the reference signature data; and

(c) instructions designed to engage in a communication session with a remote computer system and access the one or more reference signature data from the remote computer system;

(d) detection engine software comprising instructions that correlate the sensed data and the reference signature data to produce a correlation value that is indicative of a relationship between the sensed data and the reference signature data; and

instructions that initiate a detection indication when the correlation value is above a predefined threshold; and

(e) wherein the correlation is performed by storing the reference signature data in the frequency domain;

storing the sensed data in the time domain;

converting the sensed data from the time domain to the frequency domain by performing a fast Fourier transform on the sensed data that is stored in the time domain;

comparing the frequency domain sensed data with the frequency domain reference signature data to produce a correlation result;

determining detection of the event when the correlation result surpasses the predefined threshold, which indicates that the frequency domain sensed data matches the frequency domain reference signature data; and

wherein the frequency domain sensed data and the frequency domain reference signature data are each numerical arrays of magnitudes or vectors.

28. The computer program stored on the medium of claim 27, wherein the sensed data is compared with a plurality of reference signature data and further comprising instructions designed to select the one reference signature data based upon the best match.

29. The computer program stored on the medium of claim 27, further comprising:

instructions designed to compare the sensed data from the transducer with one or more local reference signature data, each local reference signature data comprising a numerical array of magnitudes or vectors that can be compared and that were derived by sampling an environmental signal over a time period; and

wherein the remote computer system is engaged in a communication session after a determination that none of the one or more local reference signature data sufficiently matches the sensed data.