Conversation, presence and context detection for hologram suppression

• US 9,922,667 B2
• Filed: 01/16/2015
• Issued: 03/20/2018
• Est. Priority Date: 04/17/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Tomlin, Arthur Charles, Hill, Dave, Paulovich, Jonathan, Keibler, Evan Michael, Scott, Jason, Brown, Cameron G., Forsythe, Thomas, Kohler, Jeffrey A., Murphy, Brian
• Primary Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US14/598,578
• Publication Number:
  US 20150302869A1
• Time in Patent Office: 1,159 Days
• Field of Search: 704231-257, 704270-275
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Various embodiments relating to detecting at least one of conversation, the presence and the identity of others during presentation of digital content on a computing device. When another person is detected, one or more actions may be taken with respect to the digital content. For example, the digital content may be minimized, moved, resized or otherwise modified.

Claims

1. A head-mounted display device comprising:

one or more audio sensors configured to capture an audio data stream;

an optical sensor configured to capture an image data stream from a scene;

a see-through display configured to display a digital content item;

a logic machine; and

a storage machine holding instructions executable by the logic machine to

receive the stream of audio data from the one or more audio sensors while the digital content item is being displayed via the see-through display,

receive the stream of image data from the optical sensor while the digital content item is being displayed via the see-through display,

detect at least one of a presence and identity of a person from at least one of the audio data stream captured by the one or more audio sensors and image data stream captured by the optical sensor, and

modify presentation of the digital content item via the see-through display in response to detecting at least one of the presence and identity of the person by at least one of the one or more audio sensors and the optical sensor, modification of the presentation comprising at least one of providing a cutout in the presentation around at least a portion of the person, pausing the content and repositioning the content elsewhere away from the detected person.

2. The head-mounted display device of claim 1, wherein the digital content item includes one or more of an audio content item and a video content item, and wherein the instructions are executable to modify presentation of the digital content item by providing a cutout entirely around the person, the cutout obscuring virtual content items otherwise overlapping with the person.

3. The head-mounted display device of claim 1, wherein the digital content item includes one or more of an audio content item and a video content item, and wherein the instructions are executable to modify presentation of the digital content item by providing a cutout around a head of the person, the cutout obscuring virtual content items otherwise overlapping with the person.

4. The head-mounted display device of claim 3, wherein the cutout moves with the person.

5. On a computing device, a method for detecting when to suppress digital content, the method comprising:

receiving at least one of an audio data stream and an image data stream from one or more sensors;

detecting a conversation between a first person receiving the digital content and a second person; and

modifying presentation of a digital content item when the conversation is detected, the modification of the presentation comprising suppressing a first portion of the content around at least a portion of the person while leaving a second portion of the content unaltered.

6. The method of claim 5, wherein detecting the conversation comprises:

detecting voice activity in the audio data stream, determining that the voice activity includes segments of human speech, and

determining that the segments of human speech alternate between different source locations.

7. The method of claim 6, wherein the one or more sensors include a microphone array comprising a plurality of microphones, and wherein determining a source location of a segment of human speech includes applying a beam-forming spatial filter to a plurality of audio samples of the microphone array to estimate the different source locations.

8. The method of claim 5, wherein the digital content item includes one or more of an audio content item and a video content item, and wherein modifying presentation of the digital content item further includes pausing presentation of the audio content item or the video content item.

9. The method of claim 5, wherein the digital content item includes an audio content item, and wherein modifying presentation of the digital content item further includes lowering a volume of the audio content item.

10. The method of claim 5, wherein the digital content item further includes one or more visual content items, and wherein modifying presentation of the digital content item includes one or more of hiding the one or more visual content items from view on a display, moving the one or more visual content items to a different position on the display, changing a translucency of the one or more visual content items, and changing a size of the one or more visual content items on the display.