METHOD OF ANALYZING AUDIO, MUSIC OR VIDEO DATA

• US 20100223223A1
• Filed: 06/19/2006
• Published: 09/02/2010
• Est. Priority Date: 06/17/2005
• Status: Abandoned Application

• Current Assignee: Queen Mary & Westfield College
• Original Assignee: Queen Mary & Westfield College
• Inventors: Sandler, Mark, Abdallah, Samer, Raimond, Yves
• Application Number:
  US11/917,601
• Publication Number:
  US 20100223223A1
• Time in Patent Office: Days
• US Class Current: 706/50
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/639 using playlists
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content

Abstract

Meta-data or tags are generated by analysing audio, music or video data; a database stores audio, music or video data; and a processing unit analyses the data to generate the meta-data in conformance with an ontology. Ontology-based approaches are new in this context. A logical processing unit infers knowledge from the meta-data.

Claims

1. A method of analysing audio, music or video data, comprising the steps of:

(1) a database storing audio, music or video data;

(2) a processing unit analysing the data to automatically generate meta-data in conformance with an ontology and to infer knowledge from the data and/or the meta-data.

2. The method of claim 1 in which the processing unit stores the meta-data in the database as further data, enabling the processing unit to analyse the further data to generate meta-data.

3. The method of claim 1 in which the processing unit includes a maths processing unit and a logic processing unit.

4. The method of claim 1 in which the ontology is a collection of terms specific to the creation, production, recording, editing, delivery, consumption, processing of audio, video or music data and which provide semantic labels for the audio, music or video data and the meta-data.

5. The method of claim 1 in which the ontology includes an ontology of one or more of the following:

music, time, events, signals, computation, any other ontology available on the internet or the Semantic Web.

6. The method of claim 5 in which the ontology of music includes one or more of:

(a) musical manifestations, such as opus, score, sound, signal;

(b) qualities of music, such as style, genre, form, key, tempo, metre

(c) agents, such as person, group and role, such as engineer, producer, composer, performer;

(d) instruments;

(e) events, such as composition, arrangement, performance, recording

(f) functions analysing existing data to create new data

7. The method of claim 5 in which the ontology of time includes time-point, moment, time interval, timeline, timeline mapping, co-ordinate systems.

8. The method of claim 7 in which the ontology of time uses interval based temporal logics.

9. The method of claim 5 in which the ontology of events includes event tokens representing specific events with time, place and an extensible set of other properties.

10. The method of claim 5 in which the ontology of signals includes sample, frame, signal fragment, acoustic, electronic, stereo, multi-channel, live, discrete and continuous time signals.

11. The method of claim 5 in which the ontology of computation includes Fourier transform, filtering, onset detection, hidden Markov modelling, Bayesian inference, principal and independent component analyses, Viterbi decoding, and relevant parameters, callable computation, non-deterministic function, evaluation, computational events, computation time, argument types, access modes, determinism, evaluation events.

12. The method of claim 11 in which the ontology of computation can be dynamically modified.

13. The method of claim 11 comprising the step of managing the computation by using functional tabling, in which the computations and outcomes are stored in a database, in order to contribute to future computations.

14. The method of claim 5 in which the ontology includes an ontology of semantic matching, which associates an algorithm to one or more concepts and includes some or all of the following terms:

predicate, Knowledge Machine, RDF triples, match.

15. The method of claim 1 including the step of applying temporal logic to reason about the processes and results of signal processing.

16. The method of claim 15 in which internal data models represents unambiguously temporal relationships between signal fragments in the database.

17. The method of claim 15 which builds on previous work on temporal logic by adding new types or descriptions of object.

18. The method of claim 15 which allows for multiple time lines to support definition of multiple related signals.

19. The method of claim 15 in which time-line maps are generated, handled or declared.

20. The method of claim 5 in which knowledge extracted from the Semantic Web is used in the processing to assist meta-data creation.

21. The method of claim 1 in which there are several sets of databases, processing units and logical processing units.

22. The method of claim 21 in which the several sets are each on different user computers or other appropriately enabled devices.

23. The method of claim 1 in which the database is distributed across the Internet and/or Semantic Web.

24. The method of claim 1 in which there are several sets of databases, processing units and logical processing units, co-operating on a task.

25. The method of claim 1 deployed automatically in a system used for the creation of artistic content.

26. The method of claim 25 in which the system also manages various independent instrument recordings.

27. The method of claim 26 in which the system processes related metadata to provide a single or integrated metadata representation that corresponds appropriately to a combination of the instrument recordings, whether raw or processed, that constitutes the musical work.

28. The method of claim 1 in which the meta-data analysed by the processing unit includes manually generated meta-data.

29. The method of claim 1 in which the meta-data analysed by the processing unit includes pre-existing meta-data.

30. The method of claim 1 in which the ontology includes a concept of ‘

mode’

that allows relations to be declared as strictly functional when particular attributes are treated as ‘

inputs’ and

allows reasoning about legal ways to use the relations and how to optimise its use by tabling previous computations.

31. The method of claim 30 in which the mode allows for a class of stochastic computations, where the outputs is defined by a conditional probability distribution.

32. The method of claim 1 in which information retrieval applications are built on top of a Semantic Web environment, through a layer interpreting the knowledge available in the Sematic Web.

33. A music, audio or video data file tagged with meta-data generated using the above method claim 1.

34. A method of locating music, audio or video data by searching against meta-data generated using the above method claim 1.

35. A method of purchasing music, audio or video data by locating the music, audio or video using the method of claim 34.

36. A database of music, audio, or video data tagged with meta-data generated using the above method claim 1.

37. A personal media player storing music, audio, or video data tagged with meta-data generated using the above method claim 1.

38. The personal media player of claim 36 being a mobile telephone.

39. A music, audio, or video data system that distributes files tagged with meta-data generated using the above method claim 1.

40. (canceled)

41. (canceled)