Visualization-based user interface system for exploratory search and media discovery

• US 9,245,055 B2
• Filed: 10/16/2009
• Issued: 01/26/2016
• Est. Priority Date: 10/16/2008
• Status: Active Grant

• Current Assignee: Christian Krois, Klaus Gasteier
• Original Assignee: Christian Krois, Klaus Gasteier
• Inventors: Gasteier, Klaus, Krois, Christian
• Primary Examiner(s): Hong, Stephen
• Assistant Examiner(s): ROBINSON, MARSHON L
• Application Number:
  US12/998,404
• Publication Number:
  US 20120017145A1
• Time in Patent Office: 2,293 Days
• Field of Search: 715/200, 715/205, 715/246
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9038 Presentation of query results

Abstract

A user interface system for exploratory search and media discovery includes a display unit with a specific geometrical grid for the visualization of multidimensional interrelations among media items and includes a processing unit programmed to distribute and map media items to the grid coordinates. A method includes user interaction with the user interface system for exploratory browsing, with the users interacting with the visualization grid based on an extension of the focus-plus-context principle for a fluent browsing process. A further method includes generating the specific grid on the display unit.

Claims

1. A user interface system for exploratory search and media discovery programmed for organizing media items in a data space and making these media items navigable by users, the user interface system comprising

a processing unit and a display unit,

the display unit

displaying a specific rigid 2-dimensional geometrical grid with a focal field in a center of the grid, the focal field being surrounded by at least one closed rectangular ring or by a plurality of closed rectangular rings getting repeatedly smaller towards the outside,

where each ring is subdivided regularly into rectangular fields,

resulting in the geometrical grid that subdivides any rectangular display in a regular, symmetric and centric manner as a recursive tiling,

where this specific grid is used to visualize relations between the media items placed in the focal field and related media items distributed in the fields surrounding the focal field, following a focus-plus-context principle, where the media item in the focal field is the focus and the surrounding items are the context,

from which users can select any item to become the new focus,

the processing unit

being programmed to filter and to map media items from n-dimensional data spaces to the specific grid,

as a focus-plus-context visualization that serves as a user interface for enabling a semantically coherent navigation flow,

where the user selects any media item of the user'"'"'s interest from the context, and by that defines it as the new focus, which triggers queries to retrieve related media items, which are mapped again to the grid building an updated context, from where the user again selects a media item triggering new queries and consequently updated display of focus plus context,

the processing unit being programmed to perform a dimension reduction of the n-dimensional data structure for mapping media items to the fields of the specific grid so that the proximity to the focus visualizes a degree of relevance of each context item to the item in the focus, and the arrangement along the grid'"'"'s directions is an order by similarities.

2. The user interface system according to claim 1, wherein the number of rings of the specific grid is between 1 and n.

3. The user interface system according to claim 1, wherein the processing unit is

mapping media items to the grid according to the media items'"'"' relevance in relation to the media item in the center, representing the focus,

and ordering the media items along the directions of the grid, in a way that as a whole visualizes an order comprising relevance and direction in a single view, wherein direction and relevance can be based on semantic, temporal or other data.

4. The user interface system according to claim 3, wherein the specific grid further has any number of rings, on these rings media items are mapped by the processing unit as described in claim 3.

5. The user interface system according to claim 1, wherein the specific grid is fully covering any available rectangular area on the display unit with rectangular fields as a recursive tiling.

6. The user interface system according to claim 1, the processing unit having access to media items in the data space and the function of mapping the media items to the display unit.

7. The user interface system according to claim 1, further comprising an input device.

8. The user interface system according to claim 7, wherein each field in any ring is selectable via the input device.

17. The user interface system according to claim 1, an interface for exploration and/or visualization of the media items'"'"' multidimensional interrelations wherein all media items relate to the central media item representing the focus and to the central media items'"'"' metadata.

18. The user interface system according to claim 17, wherein the processing unit performs a dimension reduction of the n-dimensional data space for mapping media items to the specific grid on the display unit.

19. The user interface system according to claim 18, wherein the processing unit is programmed to filter media items from the n-dimensional data space and to map the media items along symmetrical directions defined by a set of parameters and onto rings of the grid, so that the specific grid forms a cognitive map visualizing representatives of n-dimensional complexity with a simplified order in a single view, wherein the order can be based on semantic, temporal or other data.

20. The user interface system according to claim 19, wherein the set of parameters is pre-adjustable or generated automatically based on the data context, or is configurable via user input.

21. The user interface system according to claim 19, wherein the processing unit distributes the media items filtered from the n-dimensional data spaces to the specific grid and the grid'"'"'s directions and rings so that the resulting specific cognitive map displays the results of one or multiple queries and the queries'"'"' derivations in a single view.

25. The user interface system according to claim 1, where the use of the specific grid as a focus-plus-context user interface integrates search, navigation and reception of media items in a single view.

9. A method for user interaction executed by a processing unit of a user interface system for exploratory browsing, where users are enabled to perform a fluent discovery process through a visualization-based user interface, comprising

a focus-plus-context visualization that serves as a user interface based on a specific two-dimensional grid displayed by a display unit with a focal field in the center, surrounded by at least one closed rectangular ring or by a plurality of closed rectangular rings getting repeatedly smaller towards the outside, where each ring is subdivided regularly into rectangular fields, resulting in a geometrical grid that subdivides any rectangular display in a regular, symmetric and centric manner as a recursive tiling, where the media item in the central field is the focus and the surrounding items are the context,

an algorithm that is mapping media items to the fields of the specific grid according to their relevance in relation to the media item in the center representing the focus and ordering the media items along the directions of the grid according to the set of parameters associated with each direction,

in a way that as a whole it visualizes an order comprising relevance and direction in a single view, wherein direction and relevance can be based on semantic, temporal or other data,

a program filtering media items and mapping them from n-dimensional data spaces to the specific grid,

a user interface feature enabling users to select any field from the context, and by that defining it as the new focus,

a program filtering media items from n-dimensional data spaces based on the media item in the center representing the focus and the center media items'"'"' associated metadata,

the method for user interaction which enables users to engage in a fluent navigation process comprising the following steps:

display of media items arranged on the specific grid as focus-plus-context visualization,

selection of any media item from the context of the focus-plus-context visualization by the user,

update of visualization displaying the selected item in the center representing the focus,

query formulation based on the media item in the center representing the focus and the center media item'"'"'s associated metadata,

filtering media items from n-dimensional data spaces based on the media item in the center representing the focus and its associated metadata,

mapping media items from n-dimensional data spaces to the specific grid according to the relation of each item to the center item onto rings weighted by relevance and along directions by similarity, and

another selection of any media item from the new context of the focus-plus-context visualization by the user.

11. The method for user interaction according to claim 9, comprising a method for mapping visual representations of media items to the specific grid that is coherent and structured in the relationships the specific grid visualizes.

14. The method for user interaction according to claim 9, a classical focus-plus-context principle being extended to focus-plus-context-plus-flow to meet the requirements of browsing audiovisual media fluently by leading from one focus-plus-context visualization result to another.

15. The method for user interaction according to claim 9, a flowing, exploratory browsing being made possible, by the user interacting with the user interface system leading from one focus-plus-context visualization result to another, creating semantically coherent linear sequences of media items from a visualization of non-linear and multidimensional interrelations of media items.

16. The method for user interaction according to claim 9 enabling the user to select a visual representation of any media item from the context of visually correlated media items, and by that submitting a search query, which is formulated automatically in the background.

22. The method for user interaction according to claim 9, where n-dimensional data spaces after preparation and reduction are mapped onto the two-dimensional geometric grid that is highly regular and symmetrical.

23. The method for user interaction according to claim 9, which has an autoplay option in the user interaction process in which the user is lead from one focus-plus-context visualization result to another, where each selection of a media item to become the next focus is performed automatically by the processing unit, so that semantically coherent sequences of media items are generated and displayed, and by this users are enabled to consume media passively, and taking interaction only as an option, not as a must, which is particularly suitable for browsing audiovisual media.

26. The user interface system according to claim 9, where the mapping of media items to the visualization grid reduces the complexity of the underlying n-dimensional data space to meet the structure of the grid, which has a highly regular, centric, symmetric and stable geometry, on which content items are arranged onto rings weighted by relevance related to the focus and along semantic directions by similarities in a consistent way.

27. The method for user interaction according to claim 9, where the use of the specific grid as a focus-plus-context user interface integrates search, navigation and reception of media items in a single view.

32. The method for user interaction according to claim 9, further comprising a step of updating the display of the optional timeline showing the linear sequence of all media items the user selected.

34. The method for user interaction according to claim 9, further comprising a program performing a dimension reduction of the n-dimensional data structure.

10. A proprietary method executed by a processing unit, the method comprising steps of:

generating via the processing unit a specific grid geometry on a display unit, where the specific grid geometry is a 2-dimensional, symmetrical, rigid, recursive tiling with a focal field in the center, surrounded by a plurality of closed rectangular rings getting repeatedly smaller toward the outside,

subdividing each ring regularly into rectangular fields displayed on the display unit, and

distributing, via the processing unit, media items on the specific grid geometry on the display unit around a center media item according to the media items'"'"' multi-dimensional interrelations in data space,

wherein the length of the sides of each rectangular field in each ring is half of the length of the sides of the adjacent rectangular field in the neighboring inner ring, and

wherein the specific grid geometry serves as a mapping architecture for visualizing on the display unit the output of the processing unit programmed for organizing the media items in the data space, for filtering these media items, and for mapping these media items from n-dimensional data space to the specific grid displayed on the display unit.

12. The method according to claim 10, the method generating the specific grid on which the displayed complexity of relationships among media items is reduced in a consistent, centric and symmetrized way.

13. The method according to claim 12, the method generating the specific grid on which proximity to the focus visualizes a degree of relevance of each item represented in the context to the item in the focus, and the arrangement along the grid'"'"'s directions is an order by similarities, which can be based on semantic, temporal or other data.

24. The method according to claim 10 generates the number n of fields in a ring k according to n(k)=(3*2^k−

3)*4.

28. The method according to claim 10, where the spatial structure in the grid visualization represents a semantically weighted order without the need of displaying descriptions of relations through additional elements like lines, arrows, labels, text or whitespace.

29. The method according to claim 10, where the specific grid geometry filled with representations of media items forms a cognitive map visualizing representations of n-dimensional complexity within a simplified visual order.

30. The method according to claim 10, where the space available for the display of media items on the display unit is fully and seamlessly covered by the specific grid, on which media items are arranged by relevance and similarity.

31. The method according to claim 10, where the use of the specific grid as a focus-plus-context user interface integrates search, navigation and reception of media items in a single view.

33. The method according to claim 12, where the use of the specific grid as a focus-plus-context user interface visualizes a degree of relevance and an order by similarities in a single view.