System for multimedia recognition, analysis, and indexing, using text, audio, and digital video

• US 20050022252A1
• Filed: 06/04/2002
• Published: 01/27/2005
• Est. Priority Date: 06/04/2002
• Status: Abandoned Application

• Current Assignee: Tong Shen
• Original Assignee: Tong Shen
• Inventors: Shen, Tong
• Application Number:
  US10/161,920
• Publication Number:
  US 20050022252A1
• Time in Patent Office: Days
• US Class Current: 725/135
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

A new system design of multimedia recognition, processing, and indexing utilizes several new researches and technologies in the field of multi-media processing. The system integrates mature technologies being used in video security surveillance, media post-production, digital video storage and management, military visual and tacking technologies. The system makes unique integration of these existing, new, and upcoming technologies that have not been used in this combined fashion before, therefore providing new usage and applications beyond the simple sum of the functions of each technology. These technologies as components in a system that is open standard, and therefore can improve itself by modifying and replacing the technology components. The design of the system targets primarily heavily produced media contents from news, entertainment, and education and training, but not limited to these contents. Other digital contents, from live broadcast, to web broadcast, to home video, web cam, etc. can certainly use many different components of the system, and to utilize the open standard platform for various usages.

Claims

1. A multimedia application method comprising the steps of:

capturing analog source video programs and converting the analog source video programs into digital video programs;

transforming the digital video programs into selected formats;

defining modality sets of the digital video programs as tracks of audio, text, still images, moving images, and image objects in video frames;

using selected techniques for parallel processing the modality sets;

generating tags of the modality sets and storing the tags as metadata;

comparing and cross-referencing the tags, thereby defining relevance and interrelationships between the tags thereby mirroring the interrelationships of the modality sets;

thematically relating clips of the tags;

enabling addition, subtraction, combining and division of the modality sets;

establishing numerical correspondence between the parallel processes and the modality sets;

cross-comparing and cross-referencing the metadata.