METHOD AND SYSTEM TO ORGANIZE AND VISUALIZE MEDIA

• US 20130110838A1
• Filed: 07/15/2011
• Published: 05/02/2013
• Est. Priority Date: 07/21/2010
• Status: Abandoned Application

• Current Assignee: Spectralmind GmbH
• Original Assignee: Spectralmind GmbH
• Inventors: Jochum, Wolfgang, Peiszer, Ewald, Lidy, Thomas
• Application Number:
  US13/808,484
• Publication Number:
  US 20130110838A1
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces

Abstract

The invention concerns a method to organize and visualize electronic files (1) comprising media items (2) on an electronic device (3), characterised in that the method comprises the steps of accessing and opening the electronic files (1) and analysis of the media items (2) to extract content (4) and/or meta information (5); organization of media items (2) according to their similarity in content (4) and/or meta information (5); visualization of media items (2) as visual entities (6) laid out and/or placed on a user interface (7) according to their similarity. The invention further concerns a computer program implementing such a method and a computer readable medium comprising such a computer program, as well as an electronic device for organizing and visualizing electronic files comprising media items.

Claims

1-25. -25. (canceled)

26. Method to organize and visualize electronic files (1) comprising media items (2) on an electronic device (3), the method comprising the following steps:

a) accessing and opening the electronic files (1) and analysis of the media items (2) to extract content (4) and/or meta information (5);

b) organization of media items (2) according to their similarity in content (4) and/or meta information (5);

c) visualization of media items (2) as visual entities (6) laid out and/or placed on a user interface (7) according to their similarity,

characterised in that the media items (2) are organized in hierarchical groups (13) and grouping/clustering is performed on various levels of detail.

27. Method according to claim 26, characterised in that the content (4) comprises an audio signal, an audio waveform, a video signal, video content, text content, image content, or combinations thereof.

28. Method according to claim 26, characterised in that the meta information (5) comprises

a. file-specific information (8) such as file size,

b. attached information (9) such as ID3 tags, artist, album,

c. external information (10) such as buying statistics,

d. manually attached information (11) such as tags, or genre,

e. automatically attached information (12) such as usage statistics.

29. Method according to claim 26, characterised in that spectral features such as the rhythmic, timbral and/or visual structure are extracted from the content (4) of the media items (2) on one or more frequency bands to further assess the similarity of media items (2).

30. Method according to claim 26, characterised in that the visualization comprises the steps of placement of the media items on a grid using a dimensionality reduction method such as iterative self-organizing map training or multidimensional scaling.

31. Method according to claim 30, characterised in that the visualization comprises the steps of processing the grid data with a kernel of arbitrary, preferably radial, shape decreasing in size and peak detection to generate and place the visual entities (6).

32. Method according to claim 26, characterised in that the precomputed visualizations of predetermined zoom levels are stored in a database.

33. Method according to claim 26, characterised in that the visual entities (6) comprise circles, circular structures, rectangular structures, polygons, colored shapes, three-dimensional objects, or combinations thereof.

34. Method according to claim 26, characterised in that meta information (5) is visualized by displaying descriptive labels (14).

35. Method according to claim 34, characterised in that the labels (14) are clustered and the clusters are labeled and the number and placement of the cluster labels is determined by the following steps:

a. Estimation of the number of clusters for each possible label;

b. K-means clustering for each possible label;

c. determination of label position by its cluster center.

36. Method according to claim 34, characterised in that the labels (14) are clustered and the clusters are labeled, and the number and placement of the cluster labels are determined by the following steps:

a. Hierarchical agglomerative clustering for each possible metadata label;

b. cut off of the hierarchical tree at specific positions;

c. determination of label positions by location of centroids of remaining clusters.

37. Method according to claim 26, characterised in that the visualization is adapted and/or changed through user input or automatically.

38. Method according to claim 26, characterised in that media items (2) are selected, retrieved, visualized, and/or played back, and/or moved into a shopping basket by user interaction.

39. Method according to claim 26, characterised in that the positions of media items or groups are stored in a Geographic Information Systems database.

40. Computer program, characterised in that the computer program implements a method according to claim 26.

41. Computer readable medium, characterised in that the computer readable medium comprises a computer program according to claim 40.

42. Electronic device (3) for organizing and visualizing electronic files (1) comprising media items (2), comprising a user interface (7), a processing unit (15), and a storage unit (16), characterised in that media items (2) are hierarchically organized according to their similarity in content (4) and/or meta information (5), where grouping/clustering is performed on various levels of detail, and visualized as visual entities (6) laid out and/or placed according to their similarity.

43. Electronic device (3) according to claim 42, characterised in that the content (4) comprises an audio signal, an audio waveform, a video signal, video content, text content, image content, or combinations thereof.

44. Electronic device (3) according to claim 42, characterised in that the processing unit (15) comprises a feature extractor (17) adapted to extract spectral features from the content (4) of the media items (2) such as the rhythmic, timbral and/or visual structure of the media item (2) on one or more frequency bands to further assess the similarity of media items (2).

45. Electronic device (3) according to claim 42, characterised in that the meta information (5) comprises

a. file-specific information (8) such as file size,

b. attached information (9) such as ID3 tags, artist, album,

c. external information (10) such as buying statistics,

d. manually attached information (11) such as tags or genre, artist, album,

e. automatically attached information (12) such as usage statistics.

46. Electronic device (3) according to claim 42, characterised in that the visual entities (6) comprise circles, circular structures, rectangular structures, colored shapes, polygons, three-dimensional objects, or combinations thereof.

47. Electronic device (3) according to claim 42, characterised in that the electronic device (3) comprises a portable music player, mobile phone, smart phone, touch screen device, tablet computer, portable computer, portable digital assistant, notebook computer, personal computer, computer with a web browser, public screen, public terminal, video wall, projecting device, Hi-Fi device, television set, or interactive wall.

48. Electronic device (3) according to claim 42, characterised in that the user interface (7) comprises means to select, retrieve, visualize, and/or play back the media items (2), and/or means to move the media items (2) into a shopping basket.

49. Electronic device (3) according to claim 42, characterised in that the electronic device comprises means to access external processing units (15) and/or external databases (18).