Learning algorithm to detect human presence in indoor environments from acoustic signals

• US 10,388,303 B2
• Filed: 07/31/2018
• Issued: 08/20/2019
• Est. Priority Date: 06/30/2014
• Status: Active Grant

• Current Assignee: Rajeev Conrad Nongpiur
• Original Assignee: Rajeev Conrad Nongpiur
• Inventors: Nongpiur, Rajeev Conrad
• Primary Examiner(s): Sarpong, Akwasi M
• Application Number:
  US16/051,269
• Publication Number:
  US 20180336917A1
• Time in Patent Office: 385 Days
• Field of Search: 704271, 704250, 704215, 704260, 704231, 704213, 704208, 704233, 25033902, 700259, 700258
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

A system is described that constantly learns the sound characteristics of an indoor environment to detect the presence or absence of humans within that environment. A detection model is constructed and a decision feedback approach is used to constantly learn and update the statistics of the detection features and sound events that are unique to the environment in question. The learning process may not only rely on acoustic signal, but may also make use of signals derived from other sensors such as range sensor, motion sensors, pressure sensors, and video sensors.

Claims

1. A method for detecting the presence of one or more individual human in an indoor environment comprising:

detecting acoustic events in an audio signal obtained from the indoor environment;

determining that a first individual human is present in the indoor environment based at least in part on a first model associated with the first individual human being identified as having a greatest likelihood score associate with the acoustic events, the first model being part of a database of a plurality of models for determining presence of one or more individual human occupants based upon acoustic features, with each model of the plurality of models associated with a respective individual of a plurality of individuals; and

refining at least one of the models of the plurality of models based at least in part on the determining that the first individual human is present in the indoor environment, so that the models over time become more accurate and more specific to characteristics of the acoustic features of each of the respective individual human occupants and the particular indoor environment, wherein refining the models includes;

starting with generic models to detect, from acoustic features, the presence and absence of each of the individual human occupants in the indoor environment; and

during high-detection probability of presence of a particular individual human occupant, modifying a human-presence model for that particular individual human occupant so that relevant acoustic features specific to the particular individual human occupant and the indoor environment are emphasized, thereby improving accuracy of the human-presence model for that particular individual human occupant over time.

2. The method of claim 1, wherein the determining that a first individual human is present in the indoor environment comprises:

comparing the acoustic events with each of the plurality of models; and

computing a likelihood score for each of the plurality of models based at least on the comparing the acoustic events with each of the plurality of models.

3. The method of claim 2, wherein the determining that a first individual human is present in the indoor environment further comprises:

identifying a first model of the plurality of models having a greatest likelihood score, the first model associated with the first individual human; and

determining that the first individual human is present in the indoor environment based at least in part on the first model associated with the first individual human being identified as having a greatest likelihood score.

4. The method of claim 1, wherein the detecting the acoustic events in the audio signal comprises:

extracting directions of the audio signals;

extracting magnitude of the audio signals;

extracting frequency spectrogram of the audio signals;

extracting autocorrelation coefficients of the audio signals; and

extracting time information of the audio signals.

5. The method of claim 1, wherein refining at least one of the models of the plurality of models based at least in part on the determining that the first individual human is present in the indoor environment includes:

initializing a generic model for human presence or absence;

computing a probabilistic score for presence or absence of each individual human occupant from a joint probability of the acoustic events; and

updating a prior probability distribution of input features for a human-presence model or a human-absence model for each individual human occupant.

6. The method for claim 5, wherein updating the prior probability distribution of the input features for the human-presence model or the human-absence model of each individual human occupant includes:

confirmation of human presence or absence from one or more sensors that is different than a sensor that generated the audio signal; and

compute a score of presence or absence of each occupant from the human-presence model or human-absence model.

7. The method of claim 6 further comprising:

when presence or absence of an individual human occupant is confirmed or the score of presence and absence for an occupant exceeds a prescribed threshold;

updating joint probability of acoustic events and human presence or absence for the individual human occupant; and

updating a prior distribution of input features for the human-presence model or human absence model for the individual human occupant.

8. The method of claim 1, wherein computing the likelihood score for presence or absence of each occupant in the indoor environment includes:

computing, using the respective models, detection probabilities of the detected acoustic events during presence or absence of a respective individual human occupant; and

computing from a human-presence model or a human-absence model the likelihood score for the presence or absence of the respective individual human occupant, from the detection probabilities of the acoustic events.

9. The method of claim 8, wherein computing the detection probabilities of the acoustic events during presence or absence of each individual human occupant includes:

computing the acoustic features corresponding to the acoustic events from microphone array signals obtained from a microphone array in the indoor environment; and

computing the probability of acoustic features during presence and absence of each occupant by taking into account the prior probability distributions of acoustic features during determined presence and absence of each respective individual human occupant in the indoor environment.

10. A computer implemented method comprising:

identifying one or more events in one or more signal obtained from an indoor environment;

determining that a first individual is present in the indoor environment based at least in part on a first model of a plurality of models for determining presence of one or more individual occupants based upon one or more events being identified in the indoor environment; and

refining at least one of the models of the plurality of models based at least in part on the determining that the first individual is present in the indoor environment, the refining such that the models become more accurate in determining the presence of specific individuals being present in the indoor environment, the refining of a model of the plurality of models including;

during high-detection probability of presence of a particular individual occupant, modifying an occupant-presence model for that particular individual occupant so that relevant features specific to the particular individual occupant and the indoor environment are emphasized to improve accuracy of the occupant-presence model for that particular individual occupant.

11. The computer implemented method of claim 10, wherein each model of the plurality of models is associated with a respective individual of a plurality of individuals.

12. The computer implemented method of claim 10, wherein the determining that the first individual is present in the indoor environment is based at least in part on the first model associated with the first individual being identified as having a greatest likelihood score for being present in the indoor environment.

13. The computer implemented method of claim 10, wherein refining at least one of the models of the plurality of models based at least in part on the determining that the first individual is present in the indoor environment, refining the at least one model a plurality of successive instances over time to become increasingly more accurate and more specific to characteristics of the features of each of the respective individual occupants and the particular indoor environment.

14. The computer implemented method of claim 10, wherein determining that a first individual is present in the indoor environment is based at least in part on determining a separate probability score for:

the absence of each of the individual occupants in the indoor environment, and

the presence of each of the individual occupants in the indoor environment.

15. The computer implemented method of claim 10, wherein the refining of a model of the plurality of models further includes starting with a generic occupant-presence model to detect the presence of at least one specific individual in the indoor environment and during high-detection probability of presence of a particular individual occupant, modifying the generic occupant-presence model for the particular individual occupant so that relevant features specific to the particular individual occupant and the indoor environment are emphasized to improve accuracy of the occupant-presence model for that particular individual occupant.

16. The computer implemented method of claim 10, wherein the one or more signal obtained from the indoor environment comprises a signal obtained from at least one of a range sensor, motion sensor, pressure sensor, and video sensor.

17. The computer implemented method of claim 10, wherein refining at least one of the models of the plurality of models comprises determining a pitch and vocal-tract length of a plurality of individuals.

18. The computer implemented method of claim 10, wherein refining at least one of the models of the plurality of models based at least in part on the determining that the first individual is present in the indoor environment includes:

initializing a generic model for individual occupant presence or absence;

computing a probabilistic score for presence or absence of each individual occupant from a joint probability of the events; and

updating a prior probability distribution of input features for a presence model or an absence model for each individual occupant.

19. The computer implemented method of claim 18, wherein updating the prior probability distribution of the input features for the presence model or the absence model of each individual occupant includes:

confirmation of individual occupant presence or absence from one or more sensors that is different than a sensor that obtained the audio signal; and

computing a score of presence or absence of each occupant from the presence model or absence model.

20. The computer implemented method of claim 19, further comprising:

when presence or absence of an individual occupant is confirmed or the score of presence and absence for an occupant exceeds a prescribed threshold;

updating joint probability of events and presence or absence for the individual occupant; and

updating a prior distribution of input features for the presence model or absence model for the individual occupant.