Visual interface for identifying positions of interest within a sequentially ordered information encoding

• US 8,966,389 B2
• Filed: 09/21/2007
• Issued: 02/24/2015
• Est. Priority Date: 09/22/2006
• Status: Expired due to Fees

• Current Assignee: Limelight Networks, Inc.
• Original Assignee: Limelight Networks, Inc.
• Inventors: Castro, Alexander, Thompson, Jonathan, Thompson, Daniel
• Primary Examiner(s): VU, THANH T
• Application Number:
  US11/903,279
• Publication Number:
  US 20080077583A1
• Time in Patent Office: 2,713 Days
• Field of Search: 715/730, 715/772, 715/780, 715/727
• US Class Current: 715/772
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/64 Browsing; Visualisation therefor generation of a list or set of audio data G06F16/638
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/7854 using shape G06F16/7837 takes precedence
• CPC Class Codes: G06F 16/90344 by using string matching techniques

Abstract

Embodiments of the present invention provide intuitive, easily used, and efficient visual representation of positions of interest to a user within a sequentially ordered information encoding. In particular, various embodiments of the present invention provide a heat-map-like representation of the relatedness, at each location or position within a sequentially ordered information encoding, of the contents of the information encoding at or near the position to a specified search criterion. The heat-map visual representation allows a user to identify positions of particular interest, with respect to the specified search criterion, and to directly access the information at those positions, allowing the user to avoid time-consuming and inefficient hit-or-miss searching techniques.

Claims

1. A visual interface system that facilitates identifying probable positions within a sequentially-played content at which audio information related to a search criterion can be found, the visual interface system comprising:

a processing unit configured to be coupled to a display device, the processing unit configured to;

cause the display device to display a user interface, the user interface configured to allow a user to enter a search-criterion, wherein the search criterion is directed to audio information within the sequentially-played content; and

cause the display device to display a graphical representation of a relatedness function that displays graphical indications of the relatedness of audio information within the sequentially-played content to the search criterion at each position within a range of positions of the sequentially-played content, wherein;

the relatedness function represents gradations between exact and unrelated matches to the search criterion that are aligned in time with the sequentially-played content;

the gradations for each position within the range of positions of the sequentially-played content represent a magnitude of a relatedness metric computed as a numerical average of intermediate scores computed for each word in the search criterion for which a matching or related word is found within a window about the position; and

each intermediate score is computed as a product of a position-offset factor and a closeness-of-match factor.

2. The visual interface system of claim 1 wherein the graphical representation of the relatedness function provides a visual indication at each position within the range of positions of the magnitude of the relatedness metric computed for audio information of the sequentially-played content within the window about the position with respect to the search criterion.

3. The visual interface system of claim 2 wherein the relatedness metric is a single numerical value that directly specifies, or that is used to compute, one of:

a grayscale value;

a shading type; and

a color.

4. The visual interface system of claim 2 wherein the relatedness metric comprises two or more numerical values that directly specify, or that are used to compute, one of:

a shading type; and

a color.

5. The visual interface system of claim 2 wherein the relatedness function comprises two or more components that specify, or are used to compute, two or more parameters that together specify a color in a color-encoding system.

6. The visual interface system of claim 1 wherein the relatedness function assigns a plurality of windows, one window about each position within the range of positions of the sequentially played content, and computes the relatedness metric for each of the plurality of windows.

7. The visual interface system of claim 1 wherein the relatedness function smooths peaks by fitting a parametric smoothing function to each peak.

8. The visual interface system of claim 1 wherein the relatedness function further receives a relatedness-metric threshold and returns a minimum relatedness metric rather than returning the relatedness metric with a magnitude less than the relatedness-metric threshold.

9. The visual interface system of claim 8 wherein the relatedness function renormalizes computed relatedness metrics above the relatedness-metric threshold so that the renormalized relatedness metrics fall within the same range of magnitudes as they would were the relatedness-metric threshold equal to the minimum relatedness metric.

10. The visual interface system of claim 1 wherein the relatedness function applied to a particular sequentially-played content, at a particular, specified position or range of positions within the sequentially-played content, with respect to a specified search-criterion, additionally depends implicitly or explicitly on one or more of the following additional parameters:

a type of sequentially-played content to which the relatedness function is applied;

a type of specified search criterion;

a type of relatedness metric to be returned by the relatedness function;

a subject-matter domain to which the sequentially-played content and search criterion pertain;

a degree of smoothing to apply to computed relatedness metrics;

characteristics of a window used for comparing the search criterion to audio information at or near a position for which a relatedness metric is computed; and

a relatedness-metric threshold.

11. The visual interface system of claim 10 wherein the user interface further includes a threshold-selection feature that allows a user to specify a relatedness-metric threshold for use in computing the relatedness metrics displayed by the graphical representation of the relatedness function.

12. The visual interface system of claim 10 wherein the user interface further includes a degree-of-smoothing-selection feature that allows a user to specify a degree of smoothing for use in computing the relatedness metrics displayed by the graphical representation of the relatedness function.

13. The visual interface system of claim 10 wherein the processing unit is further configured to cause the display to display a visual representation of a high-resolution, constricted-domain relatedness function, in addition to the graphical representation of the relatedness function, that can be invoked by a user of the visual interface system to view the relatedness of a subrange of positions within the sequentially-played content in greater detail.

14. The visual interface system of claim 10 wherein the processing unit is further configured to cause the display to display a related-search-criteria display that can be invoked by a user of the visual interface system to view search criteria related to a specified search criterion.

15. The visual interface system of claim 1 wherein the sequentially-played content is a video program that includes a sequence of video frames, wherein positions within the video program are specified in units of time from beginning of rendering of the video program by a video-rendering application, and wherein search criterion include words and phrases that are compared to words and phrases transcribed from the audio information.

16. The visual interface system of claim 1 wherein the user interface is incorporated into a graphical user interface through which a user controls rendering of sequentially-played content.

17. The visual interface system of claim 16 wherein the graphical representation of the relatedness function is superimposed over, or included within, a progress-display component of the graphical user interface.

18. The visual interface system of claim 16 wherein the graphical user interface includes features that allow a user to select a position within the sequentially-played content at which to start or resume information-encoding rendering.

19. The visual interface system of claim 1 wherein the sequentially-played content is one of:

an encoded video recording;

an encoded multimedia presentation;

an encoded audio recording;

a text document; and

a sequence of images.

20. The visual interface system of claim 1 wherein the search criterion is one of:

words and phrases;

graphics;

portions of images; and

sounds.

21. A method for providing a visual interface that facilitates identifying probable positions within an sequentially-played content at which audio information related to specified search criterion can be found, the visual interface comprising:

receiving a search-criterion entry; and

providing a graphical representation of a relatedness function that displays graphical indications of the relatedness of audio information within the sequentially-played content to the specified search criterion at each position within a range of positions of the sequentially-played content;

wherein the range of positions of the sequentially-played content corresponds to a sequence of information blocks of the sequentially-played content, and

wherein the relatedness function represents gradations between exact and unrelated matches to the search-criterion that are aligned in time with the sequentially-played content, wherein the gradations for each position within the range of positions of the sequentially-played content represent a magnitude of a relatedness metric computed as a numerical average of intermediate scores computed for each word in the search criterion for which a matching or related word is found within a window about the position and wherein each intermediate score is computed as a product of a position-offset factor and a closeness-of-match factor.

22. A non-transitory machine-readable media having a set of instructions for identifying probable positions within an sequentially-played content at which audio information related to specified search criterion can be found, the instructions, when executed by at least one machine, cause the at least one machine to:

receive a search-criterion; and

provide a graphical representation of a relatedness function that displays graphical indications of the relatedness of audio information within the sequentially-played content to the specified search criterion at each position within a range of positions of the sequentially-played content;

wherein the sequentially-played content comprises sequential audio information blocks and is configured to provide for the sequential reception of the information blocks by a human user, and

wherein the relatedness function represents gradations between exact and unrelated matches to the search-criterion that are aligned in time with the sequentially-played content, wherein the gradations for each position within the range of positions of the sequentially-played content represent a magnitude of a relatedness metric computed as a numerical average of intermediate scores computed for each word in the search criterion for which a matching or related word is found within a window about the position and wherein each intermediate score is computed as a product of a position-offset factor and a closeness-of-match factor.