Method And Apparatus For Multimedia Data Management

• US 20070288237A1
• Filed: 12/01/2006
• Published: 12/13/2007
• Est. Priority Date: 06/07/2006
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Wu, Chung-Hsien, Lai, Yu-Sheng, Huang, Chien-Lin, Kang, Chia-Hao
• Granted Patent: US 7,739,110 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

A method and an apparatus for multimedia data management are disclosed. The method provides an indexing and retrieval scheme for digital photos with speech annotations based on image-like patterns transformed from the recognized syllable candidates. For annotated spoken content, the recognized n-best syllable candidates are transformed into a sequence of syllable-transformed patterns. Eigen-image analysis is further adopted to extract the significant information to reduce the dimensionality. Vector quantization is applied to quantize the syllable-transformed patterns into feature vectors for indexing. The invention of indexing scheme reduces the dimensionality and noise of data, and achieves better performance of 16.26% for speech annotated photo retrieval.

Claims

1. A method for multimedia data management, comprising the steps of:

generating a recognition result for a speech-annotation feature through an automatic speech recognizer;

transforming said recognition result into an image-like data, and constructing an index database by an eigen-image processing method; and

searching for said multimedia data with natural speech as input.

2. The method as claimed in claim 1, wherein said step of image-like data transformation further includes:

computing a similarity score of said recognition result; and

transforming said recognition result into said image-like data according to said similarity score.

3. The method as claimed in claim 1, wherein said step of constructing index database further includes:

extracting one or more feature parameters of said image-like data; and

quantizing said feature parameters and completing said index construction.

4. The method as claimed in claim 1, wherein said step of searching for said multimedia data with natural speech as input further includes:

inputting speech as query condition, and transforming said query condition into an index data;

comparing said index data to index information of said index data in said index database; and

finding corresponding multimedia data matching said query condition.

5. The method as claimed in claim 3, wherein said step of extracting feature parameters of said image-like data further includes:

obtaining n-best syllable candidates from said recognition result, transforming said n-best syllable candidates into said image-like data, and dividing said image-like data using a syllable candidate as a unit, where n being a natural number; and

performing eigen value analysis of said divided image data.

6. The method as claimed in claim 5, wherein said step of quantizing feature parameters and completing index construction further includes:

quantizing said syllable image-like resulted from eigen value analysis into a multidimensional vector data; and

clustering similar image-like data into a codeword sequence, and all said codeword sequences into a codebook.

7. The method as claimed in claim 1, wherein said automatic speech recognizer is a syllable-based speech recognizer.

8. The method as claimed in claim 1, wherein said eigen-image feature processing method is a principal component analysis of said image-like data.

9. An apparatus for multimedia data management, comprising:

a multimedia data index production module, for generating a recognition result for a speech-annotation feature through an automatic speech recognizer, transforming said recognition result into an image-like data, and constructing an index database by an eigen-image processing method; and

a multimedia data index searching module, for searching for said multimedia data with natural speech as input

10. The apparatus as claimed in claim 9, wherein said multimedia data index production module further includes:

an automatic speech recognizer, for generating said recognition result for said speech-annotation feature;

an image simulation module, for transforming said recognition result into an image-like data; and

an eigen capture and index construction module, for capturing one or more feature parameters of said image-like data, and constructing an index database by an eigen-image processing method.

11. The apparatus as claimed in claim 9, wherein each index in said index database is a codeword sequence.

12. The apparatus as claimed in claim 9, wherein said automatic speech recognizer is a syllable-based speech recognizer.

13. The apparatus as claimed in claim 12, wherein said image-like data is transformed from n-best syllable candidates selected from said recognition result, where n is a natural number.