APPARATUS AND METHOD FOR MULTI-MEDIA RECOGNITION, DATA CONVERSION, CREATION OF METATAGS, STORAGE AND SEARCH RETRIEVAL

• US 20070150517A1
• Filed: 01/08/2007
• Published: 06/28/2007
• Est. Priority Date: 09/30/2002
• Status: Active Grant

• Current Assignee: MyPort, Inc.
• Original Assignee: MyPort Technologies, Inc.
• Inventors: Malone, Michael
• Granted Patent: US 7,778,438 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/687 using geographical or spatial information, e.g. location
• CPC Class Codes: H04L 2209/60 Digital content management, e.g. content distribution
• CPC Class Codes: H04L 2209/80 Wireless network architectures or network communication protocols for wireless network security H04W12/00
• CPC Class Codes: H04L 9/14 using a plurality of keys or algorithms
• CPC Class Codes: H04L 9/3263 involving certificates, e.g. public key certificate [PKC] or attribute certificate [AC]; Public key infrastructure [PKI] arrangements network architectures or network communication protocols for supporting authentication of entities using certificates in a packet data network H04L63/0823
• CPC Class Codes: H04L 9/3271 using challenge-response
• CPC Class Codes: H04L 9/3297 involving time stamps, e.g. generation of time stamps
• CPC Class Codes: H04N 21/25816 involving client authentication restricting access to computer systems by authenticating users using a predetermined code G06F21/33; cryptographic authentication protocols H04L9/32; networks authentication protocols H04L63/08; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/25875 involving end-user authentication restricting access to computer systems by authenticating users using a predetermined code G06F21/33; arrangements for secret or secure communication including means for verifying the identity or authority of a user of the system H04L9/32; networks authentication protocols H04L63/08; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/26603 for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/26613 for generating or managing keys in general key distribution for secret or secure communication involving central third party, e.g. key distribution center [KDC] or trusted third party [TTP] H04L9/083; network support of key management H04L63/06; key management for network security in communication control or processing H04W12/04
• CPC Class Codes: H04N 21/2747 Remote storage of video programs received via the downstream path, e.g. from the server
• CPC Class Codes: H04N 21/4108 characterised by an identification number or address, e.g. local network address protecting specific internal or external computer components using identification number G06F21/73; network arrangements, protocols or services for addressing or naming H04L61/00
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/835 Generation of protective data, e.g. certificates protecting software against unauthorised usage in a vending or licensing environment G06F21/10
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 5/44 : Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04N 5/76 : Television signal recording
• CPC Class Codes: H04N 5/77 : between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/9201 : involving the multiplexing of an additional signal and the video signal

Abstract

This invention relates to the storage and search retrieval of all types of digital media files, whether allowing the user to create index keys, metatags within each media file(s), provides search and indexing capability. The search terms and index keys are based on contextual elements within the media, including meta data such as time, date and location, but including as well elements within the media itself, such as people or elements (car, animals, street, events, historical location and other) within a picture or a video, audio, voice, spoke word, instruments used in a musical work, or scenes in a movie. An authorized client can then retrieve the media from the remote location. Specific works can then be referenced by means of the generated search terms and index keys. When the user transmits these media files via e-mail, FTP, public server, or copies to a digital media or other distribution method, these index keys are contained within the media files, therefore allowing the third party to search and retrieve the media files based on metatags.

Claims

1. An apparatus that:

a) captures and records image, video audio, speech and other data, interprets the received audio as speech, converts the speech to text, parses the text into a set of searchable tags and embeds the tags into the image(s) video or audio for retrieval; and

b) provides automatically-derived searchable tags such as color, shape, texture, of image, video and other data, performs recognition comparison, parses the data into a set of searchable tags and embeds the tags into the image(s), video or other data for retrieval.

2. The apparatus of claim 1 in which the apparatus stores the image, video, audio and speech information in a storage medium.

3. The apparatus of claim 2 in which the apparatus uses a predetermined algorithm to identify speech elements in the recorded audio and convert those elements into text.

4. The apparatus of claim 3 in which the apparatus converts the text to searchable tags.

5. The apparatus of claim 4 in which the apparatus includes means to combine the searchable tags with the actual image data in such a way that the tag information can be recovered at a later time.

6. The apparatus of claim 1 that includes means for capturing audio data in addition to other information, means for interpreting the received audio as speech, means for converting the speech to text, means for parsing the text into a set of searchable tags, and means for embedding the tags into the image.

7. The apparatus of claim 6 in which the apparatus is operable to store the recorded information and speech in a storage medium.

8. The apparatus of claim 7 in which the apparatus uses a predetermined algorithm to identify speech elements in the recorded audio and convert those elements into text.

9. The apparatus of claim 8 in which the apparatus includes means for converting the text to searchable tags.

10. The apparatus of claim 9 in which the apparatus includes means for combining the searchable tags with the recorded data in such a way that the tag information can be recovered at a later time.

11. The apparatus of claim 5 in which the date, time, location and user data are similarly converted to tags and stored in the image dataset, such that the data contained in the tags can be recovered at a later time.

12. A method for capturing information, comprising the steps of:

capturing in a first capturing step a first set of information and converting the first set of information to a first format;

capturing in a second capturing step a second set of information temporally related to the first set of information and converting the second set of information to a second format;

combining the first set of information in the first format with the second set of information in the second format into a combined set of information in a combination format;

transmitting the combined set of information in the combination format to a remote location;

extracting the first and second set of information in the respective first and second formats from the combined set of information in the combination format;

converting the second set of information to an intermediate set of information that retains substantially all of the information in the second set of information; and

combining the intermediate set of information with the first set of information in the first format to provide a modified set of information representing the combination of the first set of information and substantially all of the first set of information.

13. The method of claim 12, wherein the step of combining the intermediate set of information with the first set of information in the first format embeds the intermediate set of information within the first set of information in the first format.

14. The method of claim 12, wherein the information in the first set of information comprises information from one of video information or image information.

15. The method of claim 14, wherein the second set of information includes audio information.

16. The method of claim 12, wherein the first and second capturing steps convert the first and second sets of information to digitally formatted data and further comprising assigning a unique file number to each first and second digitally formatted data.

17. The method of claim 16, and further comprising time stamping the digitally formatted data associated with the first and second sets of information.

18. The method of claim 12, and further comprising the step of capturing secondary information and embedding such secondary information within said first set of information.

19. A method for capturing image information, comprising the steps of:

capturing in a first capturing step image information and converting the captured image information to digitized image information in a digital image format;

capturing in a second capturing step an audio information related to the captured image information and converting the captured audio information to digitized audio information in a digitized audio format;

combining the captured image information in the digitized image format with the captured audio information in the digitized audio format into a combined set of information in a combination format;

transmitting the combined set of information in the combination format to a storage medium;

extracting the digitized image and audio information in the respective digital image and audio formats from the combined set of information in the combination format;

converting the digitized audio information to an intermediate set of information that retains substantially all of the information in the captured audio information; and

combining the intermediate set of information with the digitized image information in a digital combination format supporting the combination of digitized image information and the intermediate set of information to provide a modified digitized image with embedded information that represents substantially all of the captured audio information.

20. The method of claim 19, wherein the intermediate set of information comprises metatags.

21. The method of claim 19, wherein the digitized audio and video each have a unique file naming structure.

22. The method of claim 19, wherein the image is a still image captured with a digital camera.