System, method and computer program product for extracting metadata faster than real-time

• US 20050234985A1
• Filed: 04/09/2004
• Published: 10/20/2005
• Est. Priority Date: 04/09/2004
• Status: Abandoned Application

• Current Assignee: Nexjenn Media Incorporated
• Original Assignee: Nexjenn Media Incorporated
• Inventors: Gordon, Joseph, Hopkins, Scott, Veronis, Michael
• Application Number:
  US10/821,199
• Publication Number:
  US 20050234985A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/75 Clustering; Classification
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/908 using metadata automatically derived from the content
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A system, method and computer program product for extracting metadata from one or more content files in faster than real-time, where the content files may be received from more than one source.

Claims

1. A machine accessible medium that provides instructions, which when executed by a computing platform, cause said computing platform to perform operations comprising a method comprising:

a) receiving content from one or more sources, wherein said content includes a corresponding given realtime running time length; and

b) extracting metadata from said content in a period of time that is less than said corresponding given realtime running time length.

2. The machine accessible medium according to claim 1, wherein said content comprises at least one of audio data, video data, still-frame data, and digital data.

3. The machine accessible medium according to claim 1, wherein said metadata comprises at least one of a snapshot, a stream, a program elementary stream (PES), a track, a time code, and a scene change.

4. The machine accessible medium according to claim 1, wherein said extracting comprises at least one of:

processing content corresponding to a given time period in substantially said given time period; and

processing content corresponding to a given time period in less than said given time period.

5. The machine accessible medium according to claim 1, wherein said extracting comprises:

processing content by at least one of;

parallel processing and multi-tasking.

6. The machine accessible medium according to claim 1, wherein said (b) comprises at least one of:

1) extracting to optimize for throughput;

2) extracting to optimize for speed; and

3) extracting to optimize for quality.

7. The machine accessible medium according to claim 1, wherein said (b) comprises at least one of:

1) extracting a scene change;

2) extracting a face detection;

3) extracting a face recognition;

4) extracting an optical character recognition;

5) extracting a logo detection;

6) extracting text from audio;

7) extracting a key length value;

8) extracting geospatial data; and

9) extracting a closed captioning.

8. The machine accessible medium according to claim 1, wherein said (b) comprises:

1) extracting said metadata in a distributed manner.

9. The machine accessible medium according to claim 8, wherein said (b) (1) comprises at least one of:

i) extracting using one or more plugins;

ii) extracting using multiple streams on a server;

iii) extracting using multiple streams on more than one server;

iv) extracting using said one or more plugins on a server; and

v) extracting using said one or more plugins on more than one server.

10. The machine accessible medium according to claim 9, wherein said (b) (1) (i) comprises:

A) extracting using said one or more plug-ins, wherein said one or more plugins are of one or more configurations.

11. The machine accessible medium according to claim 1, wherein said (b) comprises:

1) extracting said metadata using deterministic analysis.

12. The machine accessible medium according to claim 11, wherein said (b) (1) comprises at least one of:

i) extracting said metadata to achieve repeatable results;

ii) extracting said metadata to analyze all frames;

iii) extracting said metadata to achieve no data loss; and

iv) extracting said metadata to achieve no lost frames.

13. The machine accessible medium according to claim 1, wherein said (b) comprises:

1) receiving external stream information; and

2) processing decisions based on said external stream information.

14. The machine accessible medium according to claim 13, wherein said external stream information includes at least one of size, resolution, encoding type, encoding parameters, frame rate, and data rate.

15. The machine accessible medium according to claim 1, wherein said content comprises compressed video and said (b) comprises at least one of:

1) identifying objects; and

2) identifying motion tracking of said objects.

16. The machine accessible medium according to claim 1, wherein said (b) comprises at least one of:

1) managing resources using load balancing;

2) managing resources using load balancing with a central registry; and

3) managing resources using fault tolerance methods.

17. The machine accessible medium according to claim 1, wherein said (b) comprises at least one of:

1) configuring a content processing engine;

2) reconfiguring said content processing engine; and

3) reconfiguring said content processing engine in real-time.

18. The machine accessible medium according to claim 1, wherein said method further comprises:

c) storing said metadata.

19. The machine accessible medium according to claim 1, wherein said method further comprises:

c) managing assets wherein said assets include at least one of said content and said metadata.

20. The machine accessible medium according to claim 19, wherein said (c) comprises at least one of:

1) receiving a search query;

2) displaying results of said search query; and

3) creating products from said results.

21. The machine accessible medium according to claim 20, wherein said (c) (1) comprises:

i) receiving a search query based on query terms.

22. The machine accessible medium according to claim 1, wherein said (b) is performed by a content processing engine, wherein said content processing engine is platform independent and written in an extensible object oriented programming language.

23. The machine accessible medium according to claim 1, wherein said (b) is performed by a global view content processing engine, and wherein (b) comprises at least one of:

1) correlating results of said data extractions intelligently from multiple input streams;

2) running multiple instances of said engine concurrently;

3) performing triggered event processing; and

4) maintaining a central registry listing availability and location of plugins.

24. The machine accessible medium according to claim 1, wherein said (b) is performed across an application programming interface using a scripted language wherein said scripted language comprises at least one of:

1) an extensible markup language;

2) an embedded language;

3) a command line based language; and

4) event handling via said scripting language.

25. The machine accessible medium according to claim 1, wherein said method further comprises:

c) displaying said metadata via an user interface.

26. The machine accessible medium according to claim 1, wherein said method further comprises:

c) clipping said content comprising at least one of;

1) segmenting said content;

2) marking a beginning and an ending of a plurality of key frames.

27. The machine accessible medium according to claim 1, wherein said content is at least one of intelligence industry content, law enforcement industry content, broadcast studio content, media asset management content, media and entertainment content, homeland defense content, distance learning content, security content, and business intelligence content.

28. A system to extract metadata comprising:

one or more tasks to receive at least one file of content, wherein said one or more tasks process said at least one file of content and extract metadata of one or more types;

one or more data sinks to filter said metadata based on said one or more types; and

a database to store said metadata, wherein said metadata is extracted in a period of time that is less than a running length of said content.

29. The system of claim 28, wherein said one or more tasks comprises at least one of:

an audio task to extract metadata about audio information from said content file;

a key frame task to extract metadata about one or more key frames in said content file;

a real producer task to extract metadata into real media format from said content file;

a synchronized multimedia integration language task to extract metadata from said content file; and

a mixed excitation linear predictive encoder task to extract metadata from said content file.

30. The system of claim 28, wherein said one or more components comprises at least one of:

a directory watcher to monitor one or more directories for said content file;

a scheduler to determine the processing operations or each of said one or more tasks; and

a task manager to line-up one or more plug-ins and allocate resources for said one or more tasks.

31. The system of claim 28, further comprising:

one or more database tools coupled to said database, wherein said one or more database tools view, produce and deliver reports, and query said database.

32. A method of processing metadata comprising:

a) receiving content from one or more sources; and

b) extracting metadata from said content faster than real-time.

33. The method according to claim 32, wherein said content comprises at least one of audio data, video data, still-frame data, and digital data.

34. The method according to claim 32, wherein said metadata comprises at least one of a snapshot, a stream, a program elementary stream (PES), a track, a time code, and a scene change.

35. The method according to claim 32, wherein said extracting in faster than real-time comprises:

processing content corresponding to a given time period in substantially said given time period.

36. The method according to claim 32, wherein said extracting in faster than real-time comprises:

processing content corresponding to a given time period in less than said given time period.

37. The method according to claim 32, wherein said step (b) comprises at least one of:

1) extracting to optimize for throughput;

2) extracting to optimize for speed; and

3) extracting to optimize for quality.

38. The method according to claim 32, wherein said step (b) comprises at least one of:

1) extracting a scene change;

2) extracting a face detection;

3) extracting a face recognition;

4) extracting an optical character recognition;

5) extracting a logo detection;

6) extracting text from audio;

7) extracting a key length value;

8) extracting geospatial data; and

9) extracting a closed captioning.

39. The method according to claim 32, wherein said step (b) comprises:

1) extracting said metadata in a distributed manner.

40. The method according to claim 39, wherein said step (b) (1) comprises at least one of:

i) extracting using one or more plugins;

ii) extracting using multiple streams on a server;

iii) extracting using multiple streams on more than one server;

iv) extracting using said one or more plugins on a server; and

v) extracting using said one or more plugins on more than one server.

41. The method according to claim 40, wherein said step (b) (1) (i) comprises:

A) extracting using said one or more plugins, wherein said one or more plugins are of one or more configurations.

42. The method according to claim 32, wherein said step (b) comprises:

1) extracting said metadata using deterministic analysis.

44. The method according to claim 32, wherein said step (b) comprises:

1) receiving external stream information; and

2) processing decisions based on said external stream information.

45. The method according to claim 44, wherein said external stream information includes at least one of size, resolution, encoding type, encoding parameters, frame rate, and data rate.

46. The method according to claim 32, wherein said content comprises compressed video and said step (b) comprises at least one of:

1) identifying objects; and

2) identifying motion tracking of said objects.

47. The method according to claim 32, wherein said step (b) comprises at least one of:

1) managing resources using load balancing;

2) managing resources using load balancing with a central registry; and

3) managing resources using fault tolerance methods.

48. The method according to claim 32, wherein said step (b) comprises at least one of:

1) configuring a content processing engine;

2) reconfiguring said content processing engine; and

3) reconfiguring said content processing engine in real-time.

49. The method according to claim 32, further comprising:

c) storing said metadata.

50. The method according to claim 32, further comprising:

c) managing assets wherein said assets include at least one of said content and said metadata.

51. The method according to claim 50, wherein said step (c) comprises at least one of:

1) receiving a search query;

2) displaying results of said search query; and

3) creating products from said results.

52. The method according to claim 51, wherein said (c) (1) comprises:

i) receiving a search query based on query terms.

53. The method according to claim 32, wherein said step (b) is performed by a content processing engine, wherein said content processing engine is platform independent and written in an extensible object oriented programming language.

54. The method according to claim 32, wherein said step (b) is performed by a global view content processing engine, and wherein step (b) comprises at least one of:

1) correlating results of said data extractions intelligently from multiple input streams;

2) running multiple instances of said engine concurrently;

3) performing triggered event processing; and

4) maintaining a central registry listing availability and location of plugins.

55. The method according to claim 32, wherein said step (b) is performed across an application programming interface using a scripted language wherein said scripted language comprises at least one of:

1) an extensible markup language;

2) an embedded language;

3) a command line based language; and

4) event handling via said scripting language.

56. The method according to claim 32, further comprising:

c) displaying said metadata via an user interface.

57. The method according to claim 32, further comprising:

c) clipping said content comprising at least one of;

1) segmenting said content;

2) marking a beginning and an ending of a plurality of key frames.

58. The method according to claim 32, wherein said content is at least one of intelligence industry content, law enforcement industry content, broadcast studio content, media asset management content, media and entertainment content, homeland defense content, distance learning content, security content, and business intelligence content.

43. The method according to claim 43, wherein said step (b) (1) comprises at least one of:

i) extracting said metadata to achieve repeatable results;

ii) extracting said metadata to analyze all frames;

iii) extracting said metadata to achieve no data loss; and

iv) extracting said metadata to achieve no lost frames.