CONVERSATION DETECTION

• US 20150302867A1
• Filed: 04/17/2014
• Published: 10/22/2015
• Est. Priority Date: 04/17/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Keibler, Evan Michael, Scott, Jason, Brown, Cameron, Plumb, Jonathan William, Tomlin, Arthur Charles, Paulovich, Jonathan
• Granted Patent: US 10,529,359 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Various embodiments relating to detecting a conversation during presentation of content on a computing device, and taking one or more actions in response to detecting the conversation, are disclosed. In one example, an audio data stream is received from one or more sensors, a conversation between a first user and a second user is detected based on the audio data stream, and presentation of a digital content item is modified by the computing device in response to detecting the conversation.

Claims

1. On a computing device, a method for detecting a conversation between users, the method comprising:

receiving an audio data stream from one or more sensors;

detecting a conversation between a first user and a second user based on the audio data stream; and

modifying presentation of a digital content item in response to detecting the conversation.

2. The method of claim 1, wherein detecting the conversation between the first user and the second user includes

detecting voice activity in the audio data stream, determining that the voice activity includes segments of human speech, and

determining that the segments of human speech alternate between different source locations.

3. The method of claim 2, wherein the one or more sensors include a microphone array comprising a plurality of microphones, and wherein determining a source location of a segment of human speech includes applying a beamforming spatial filter to a plurality of audio samples of the microphone array to estimate the different source locations.

4. The method of claim 2, wherein detecting the conversation between the first user and the second user further includes determining that the segments of human speech occur within a designated cadence range.

5. The method of claim 2, wherein detecting the conversation between the first user and the second user further includes determining that the segments of human speech alternate between different source locations within a threshold period of time.

6. The method of claim 2, further comprising:

determining that one or more segments of human speech are provided by an electronic audio device, and

ignoring those one or more segments of human speech provided by the electronic audio device when determining that the segments of human speech alternate between different source locations.

7. The method of claim 1, wherein the digital content item includes one or more of an audio content item and a video content item, and wherein modifying presentation of the digital content item includes pausing presentation of the audio content item or the video content item.

8. The method of claim 1, wherein the digital content item includes an audio content item, and wherein modifying presentation of the digital content item includes lowering a volume of the audio content item.

9. The method of claim 1, wherein the digital content item includes one or more visual content items, and wherein modifying presentation of the digital content item includes one or more of hiding the one or more visual content items from view on a display, moving the one or more visual content items to a different position on the display, changing a translucency of the one or more visual content items, and changing a size of the one or more visual content items on the display.

10. A storage machine holding instructions executable by a logic machine to:

receive an audio data stream from one or more sensors;

detect a conversation between a first user and a second user based on the audio data stream; and

modify presentation of a digital content item in response to detecting the conversation.

11. The storage machine of claim 10, wherein detecting the conversation between the first user and the second user includes

detecting voice activity in the audio data stream,

determining that the voice activity includes segments of human speech, and

determining that the segments of human speech alternate between different source locations.

12. The storage machine of claim 11, wherein detecting the conversation between the first user and the second user further includes determining that the segments of human speech alternate between different source locations within a threshold period of time or the segments of human speech occur within a designated cadence range.

13. The storage machine of claim 11, further holding instruction executable by the logic machine to

determine that one or more segments of human speech are provided by an electronic audio device, and

ignore those one or more segments of human speech provided by the electronic audio device when determining that the segments of human speech alternate between different source locations.

14. The storage machine of claim 10, wherein the digital content item includes one or more of an audio content item and video content item, and wherein the instructions are executable to modify presentation of the digital content item by pausing presentation of the one or more of the audio content item or video content item.

15. The storage machine of claim 10, wherein the digital content item includes an audio content item, and wherein the instructions are executable to modify presentation of the digital content item by lowering a volume of the audio content item.

16. The storage machine of claim 10, wherein the digital content item includes one or more visual content items, and wherein the instructions are executable to modify presentation of the digital content item by one or more of hiding the one or more visual content items from view on a display, moving the one or more visual content items to a different position on the display, changing a translucency of the one or more visual content items, and changing a size of the one or more visual content items on the display.

17. A head-mounted display device comprising:

one or more audio sensors configured to capture an audio data stream;

an optical sensor configured to capture an image of a scene;

a see-through display configured to display a digital content item;

a logic machine; and

a storage machine holding instructions executable by the logic machine to

while the digital content item is being displayed via the see-through display, receive the stream of audio data from the one or more audio sensors,

detect human speech segments alternating between a wearer of the head-mounted display device and another person based on the audio data stream,

receive the image of the scene including the other person from the optical sensor,

confirm that the other person is speaking to the wearer based on the image,

detect a conversation between the wearer and the other person based on the audio data stream and the image, and

modify presentation of the digital content item via the see-through display in response to detecting the conversation.

18. The head-mounted display device of claim 17, wherein the digital content item includes one or more of an audio content item and a video content item, and wherein the instructions are executable to modify presentation of the digital content item by pausing presentation of the audio content item or the video content item.

19. The head-mounted display device of claim 17, wherein the conversation is detected by determining that human speech segments are spoken by the wearer before and after a human speech segment spoken by the other person or human speech segments are spoken by the other person before and after a human speech segment spoken by the wearer.

20. The head-mounted display device of claim 17, wherein the digital content item includes a plurality of visual content items presented at different positions on the see-through display, and wherein the instructions are executable to modify presentation of the digital content item by moving a visual content item of the plurality of visual content items away from a position on the see-through display that corresponds with a direction of a source location of a segment of human speech of the other person.