Security surveillance via three-dimensional audio space presentation

• US 9,892,743 B2
• Filed: 10/18/2016
• Issued: 02/13/2018
• Est. Priority Date: 12/27/2012
• Status: Active Grant

• Current Assignee: Arlington Technologies, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Avaya Incorporated
• Inventors: Seligmann, Doree Duncan, John, Ajita, Sammon, Michael J., Skiba, David
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US15/296,238
• Publication Number:
  US 20170040028A1
• Time in Patent Office: 483 Days
• Field of Search: 348 1408, 348143, 348159, 348485, 348 47, 348156, 367 93, 367127, 379158, 381 1, 381 17, 381 56, 381 61, 381 80, 381 92, 381303, 381307, 700 94, 700245, 700 91, 704232, 704233, 704274, 715850, 345419, 703 6
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 21/028 using properties of sound source
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/03 Application of parametric coding in stereophonic audio systems
• CPC Class Codes: H04S 3/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence
• CPC Class Codes: H04S 7/304 For headphones
• CPC Class Codes: H04S 7/40 Visual indication of stereophonic sound image

Abstract

Embodiments are directed to using a three-dimensional sound space to analyze security surveillance information. According to one embodiment, the three-dimensional sound space can comprise part of a security surveillance system in which sound sources related to security surveillance information can be presented and a user can efficiently navigate even a large number of sound sources in the three-dimensional sound space. Effective audio surveillance relies on the ability of the surveillance personnel to efficiently identify calls that need further analysis and calls that need no further analysis without introducing too many false negative or false positive conditions. Utilization of three-dimensional space described herein can increase the ease with which security analysts review audio content and identify relevant audio content that requires further analysis.

Claims

1. A method comprising:

collecting, by a processor, security surveillance information from one or more data sources, the security surveillance information comprising at least in part audio data from at least one of the one or more data sources;

processing, by the processor, the collected security surveillance information into a plurality of sound sources based on one or more attributes of the security surveillance information;

presenting, by the processor, the plurality of sound sources in a three-dimensional sound space having the plurality of sound sources playing at a same time, wherein each of the plurality of sound sources is assigned a respective location in the three-dimensional sound space relative to one another, and wherein a user is assigned a current location in the three-dimensional sound space relative to each respective location;

receiving, by the processor, an input from the user to navigate the user to a new location in the three-dimensional sound space; and

based on the input, changing, by the processor, the current location of the user in the three-dimensional sound space relative to each respective location of the plurality of sound sources in the three-dimensional sound space.

2. The method of claim 1, wherein the security surveillance information further comprises non-audio data from at least one or the one or more data sources and wherein processing the collected security surveillance information into a plurality of sound sources further comprises converting the non-audio data to audio data.

3. The method of claim 1, wherein processing the collected security surveillance information into a plurality of sound sources further comprises selecting sound sources from at least a portion of the security surveillance information based on the one or more attributes of the security surveillance information.

4. The method of claim 3, wherein processing the collected security surveillance information into a plurality of sound sources further comprises grouping the selected sound sources based on the one or more attributes of the security surveillance information.

5. The method of claim 4, wherein processing the collected security surveillance information into a plurality of sound sources further comprises assigning the grouped sound sources to the respective location in the three-dimensional sound space relative to one another based on the one or more attributes of the security surveillance information.

6. The method of claim 1, wherein the attributes comprise information related to one or more of a keyword in the security surveillance information, a phrase in the security surveillance information, a sentiment expressed in in the security surveillance information, a context of a conversation in the security surveillance information, a location from which the security surveillance information was obtained, or a time at which the security surveillance information was obtained.

7. The method of claim 1, wherein the attributes comprise information related to one or more of identity of speakers in the security surveillance information, identity of a dominant speaker in the security surveillance information, an amount of turn taking occurring in a conversation in the security surveillance information, timing of a conversation in the security surveillance information, location of parties to a conversation in the security surveillance information, or recent or current events.

8. The method of claim 2, wherein the non-audio data comprises the attributes of the security surveillance information.

9. The method of claim 8, wherein presenting the plurality of sound sources in a three-dimensional sound space comprises presenting the converted attributes for the sound source or a group of sound sources.

10. A system comprising:

a processor; and

a memory coupled with the processor and storing a set of instructions which, when executed by the processor, causes the processor to;

collect security surveillance information from one or more data sources, the security surveillance information comprising at least in part audio data from at least one of the one or more data sources,

process the collected security surveillance information into a plurality of sound sources based on one or more attributes of the security surveillance information,

present the plurality of sound sources in a three-dimensional sound space having the plurality of sound sources playing at a same time, wherein each of the plurality of sound sources is assigned a respective location in the three-dimensional sound space relative to one another, and wherein a user is assigned a current location in the three-dimensional sound space relative to each respective location,

receive an input from the user to navigate the user to a new location in the three-dimensional sound space, and

based on the input, change the current location of the user in the three-dimensional sound space relative to each respective location of the plurality of sound sources in the three-dimensional sound space.

11. The system of claim 10, wherein the security surveillance information further comprises non-audio data from at least one or the one or more data sources and wherein processing the collected security surveillance information into a plurality of sound sources further comprises converting the non-audio data to audio data.

12. The system of claim 10, wherein processing the collected security surveillance information into a plurality of sound sources further comprises:

selecting sound sources from at least a portion of the security surveillance information based on the one or more attributes of the security surveillance information;

grouping the selected sound sources based on the one or more attributes of the security surveillance information; and

assigning the grouped sound sources to the respective location in the three-dimensional sound space relative to one another based on the one or more attributes of the security surveillance information.

13. The system of claim 10, wherein the attributes comprise information related to one or more of a keyword in the security surveillance information, a phrase in the security surveillance information, a sentiment expressed in in the security surveillance information, a context of a conversation in the security surveillance information, a location from which the security surveillance information was obtained, a time at which the security surveillance information was obtained, identity of speakers in the security surveillance information, identity of a dominant speaker in the security surveillance information, an amount of turn taking occurring in a conversation in the security surveillance information, timing of a conversation in the security surveillance information, location of parties to a conversation in the security surveillance information, or recent or current events.

14. The system of claim 11, wherein the non-audio data comprises the attributes of the security surveillance information and wherein presenting the plurality of sound sources in a three-dimensional sound space comprises presenting the converted attributes for the sound source or a group of sound sources.

15. The system of claim 11, wherein the non-audio data comprises the attributes of the security surveillance information and wherein presenting the plurality of sound sources in a three-dimensional sound space comprises presenting the converted attributes for the sound source or a group of sound sources.

16. A non-transitory computer-readable medium comprising a set of instruction stored therein which, when executed by a processor, causes the processor to:

collect security surveillance information from one or more data sources, the security surveillance information comprising at least in part audio data from at least one of the one or more data sources;

process the collected security surveillance information into a plurality of sound sources based on one or more attributes of the security surveillance information;

present the plurality of sound sources in a three-dimensional sound space having the plurality of sound sources playing at a same time, wherein each of the plurality of sound sources is assigned a respective location in the three-dimensional sound space relative to one another, and wherein a user is assigned a current location in the three-dimensional sound space relative to each respective location;

receiving an input from the user to navigate to a new location in the three-dimensional sound space; and

based on the input, changing the current location of the user in the three-dimensional sound space relative to each respective location of the plurality of sound sources in the three-dimensional sound space.

17. The non-transitory computer-readable medium of claim 16, wherein the security surveillance information further comprises non-audio data from at least one or the one or more data sources and wherein processing the collected security surveillance information into a plurality of sound sources further comprises converting the non-audio data to audio data.

18. The non-transitory computer-readable medium of claim 16, wherein processing the collected security surveillance information into a plurality of sound sources further comprises:

selecting sound sources from at least a portion of the security surveillance information based on the one or more attributes of the security surveillance information;

grouping the selected sound sources based on the one or more attributes of the security surveillance information; and

assigning the grouped sound sources to the respective location in the three-dimensional sound space relative to one another based on the one or more attributes of the security surveillance information.