Media presentation playback annotation

• US 9,342,516 B2
• Filed: 05/18/2011
• Issued: 05/17/2016
• Est. Priority Date: 05/18/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Matthews, Joseph H. III, Padmanabhan, Udiyan, Peters, Jannes Paul
• Primary Examiner(s): Patel, Manglesh M
• Application Number:
  US13/110,292
• Publication Number:
  US 20120297284A1
• Time in Patent Office: 1,826 Days
• Field of Search: 715/202, 715/200, 715/233
• US Class Current: 1/1
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G10L 25/57 for processing of video signals

Abstract

Annotations can be automatically added to a media presentation during playback of the presentation without a user having to manually interact with the playback device. The playback device determines whether an annotation is to be added to the media presentation based on characteristics of voice input received at the device, such as voice input signal strength or variances in the voice input signal strength. Characteristics of video input received at the device can be used to determine whether a user is speaking to the computing device as well. The device can handle a new annotation overlapping an existing annotation by either removing the existing annotation or by shifting the existing annotation until there is no more overlap. A media presentation can comprise multiple annotation tracks.

Claims

1. A method of annotating a media presentation, the method comprising:

playing at least a portion of the media presentation at a computing device comprising one or more audio input devices;

monitoring the one or more audio input devices for voice input received during the playing; and

upon detecting voice input and without manual interaction with the computing device;

upon determining that a user is speaking in the direction of the computing device, wherein the determining that the user is speaking in the direction of the computing device is based at least in part on a signal strength of the voice input received at the one or more audio input devices being above a first threshold value and variations in the signal strength of the voice input received at the one or more audio input devices being below a second threshold value;

determining that the voice input comprises an annotation to be added to the media presentation;

recording the annotation during the playing; and

adding the annotation to the media presentation; and

upon determining that the user is not speaking in the direction of the computing device;

determining that the voice input does not comprise an annotation to be added to the media presentation.

2. The method of claim 1, wherein the monitoring begins in response to the playing at least a portion of the media presentation beginning.

3. The method of claim 1, wherein the variations comprise variations in the signal strength of the voice input received across a plurality of the one or more audio input devices.

4. The method of claim 1, wherein the variations comprise variations in the signal strength of the voice input received at one of the one or more audio input devices.

5. The method of claim 1, wherein the annotation comprises a portion of the voice input received by the one or more audio inputs.

6. The method of claim 5, the computing device further comprising one or more video input devices, the method further comprising receiving video input at the one or more video input devices during the playing, the annotation comprising a portion of the video input received by the one or more video input devices.

7. The method of claim 1, the computing device further comprising one or more video input devices, the determining that the voice input comprises an annotation being based at least in part on video input received by the one or more video input devices.

8. The method of claim 7, wherein the determining that the voice input comprises an annotation further comprises determining that the user is positioned in front of the computing device.

9. The method of claim 1, wherein the computing device comprises an input device capable of determining a distance that a user is positioned from the computing device, the determining that the voice input comprises an annotation further comprising determining that the user is positioned within a specified distance range in front of the computing device.

10. The method of claim 1, the computing device comprising a display, the media presentation being played back at the display during the playing, user interface elements related to annotation being absent from the display during the playing.

11. The method of claim 1, the determining that the voice input comprises an annotation being based only on the voice input.

12. The method of claim 1, the computing device further comprising one or more video input devices, the method further comprising receiving video input at the one or more video input devices, the determining that the voice input comprises an annotation being based only on the voice input and the video input.

13. The method of claim 1, the determining that the voice input comprises an annotation further comprising capturing the annotation while a media element of the media presentation is playing, and the method further comprising:

determining that the capturing of the annotation extends past a time that the media element stops playing in the media presentation; and

modifying the media presentation such that the media element is scheduled to be played for the duration of the annotation.

14. The method of claim 1, the method further comprising:

after the adding the annotation to the media presentation, playing the media presentation at the computing device or another computing device, the playing comprising playing the annotation.

15. At least one computing device programmed to carry out a method, the method comprising:

playing at least a portion of a media presentation at a computing device, the media presentation being defined at least in part by a master file comprising annotation data indicating a time when an existing annotation is to be played during playback of the media presentation, the computing device comprising one or more audio input devices;

monitoring the one or more audio input devices for voice input during the playing; and

upon detecting voice input and without manual interaction with the computing device;

determining that the voice input comprises a new annotation to be added to the media presentation, wherein determining that the voice input comprises a new annotation comprising determining that a user is speaking in the direction of the computing device, wherein the determining that the user is speaking in the direction of the computing device is based at least in part on a signal strength of the voice input received at the one or more audio input devices being above a first threshold value and variations in the signal strength of the voice input received at the one or more audio input devices being below a second threshold value;

determining that the new annotation overlaps the existing annotation by an overlap amount;

recording the new annotation during the playing;

modifying the annotation data to remove the existing annotation from the media presentation if the overlap amount exceeds an overlap threshold, and modifying the annotation data to shift the time at which the existing annotation begins playing during playback of the media presentation so that the existing annotation no longer overlaps with the new annotation if the overlap amount is less than the overlap threshold;

adding the new annotation to the media presentation, the adding comprising adding annotation data to the master file indicating the time at which the new annotation is to start playing during playback of the media presentation and a reference to an annotation file storing the new annotation; and

after the adding the new annotation to the media presentation, playing the media presentation, the playing comprising playing the new annotation.

16. The at least one computing device of claim 15, the modifying the annotation data to shift the time at which the existing annotation begins playing during playback of the media presentation comprising:

splitting the existing annotation into a first annotation segment and a second annotation segment;

shifting the first annotation segment forward in time to avoid overlapping the new annotation; and

shifting the second annotation segment back in time to avoid overlapping with the new annotation.

17. One or more computer-readable memory or storage devices storing computer readable code that when executed by a computing device performs a method of annotating a media presentation, the method comprising:

playing at least a portion of the media presentation at the computing device comprising one or more audio input devices;

monitoring the one or more audio input devices for voice input received during the playing; and

upon detecting voice input and without manual interaction with the computing device;

upon determining that a user is speaking in the direction of the computing device, wherein the determining that the user is speaking in the direction of the computing device is based at least in part on a signal strength of the voice input received at the one or more audio input devices being above a first threshold value and variations in the signal strength of the voice input received at the one or more audio input devices being below a second threshold value;

determining that the voice input comprises an annotation to be added to the media presentation;

recording the annotation during the playing; and

adding the annotation to the media presentation; and

upon determining that the user is not speaking in the direction of the computing device;

determining that the voice input does not comprise an annotation to be added to the media presentation.