Method and system for analyzing multimedia content

• US 8,204,955 B2
• Filed: 04/16/2008
• Issued: 06/19/2012
• Est. Priority Date: 04/25/2007
• Status: Active Grant

• Current Assignee: Miovision Technologies Incorporated
• Original Assignee: Miovision Technologies Incorporated
• Inventors: McBride, Kurtis, Fairles, Chris, Madill, Kevin, Zhang, Di, Brijpaul, Tony, Thompson, David, Silagadze, Vecheslav
• Primary Examiner(s): TRUONG, LAN DAI T
• Application Number:
  US12/104,092
• Publication Number:
  US 20080270569A1
• Time in Patent Office: 1,525 Days
• Field of Search: 709/217, 709/220, 709/223, 709/203, 709/227
• US Class Current: 709/217
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

A method and system are provided for remotely analyzing multimedia content, in particular video, and extracting information from such multimedia content. The system analyses, e.g. a video file by way of an FTP, file upload or streaming data etc., and configuration settings specified in one embodiment by a separate entity at one of multiple server entities. The system also comprises one or more remote server entities utilizing data storage and data processing capabilities. In one embodiment the client sends a video source and configuration settings over a network, the remote server accepts the video source, generates configuration settings, and a data processing module analyzes the video content to extract data from the video.

Claims

1. A method for analyzing multimedia data comprising:

a server device receiving a multimedia file generated by a remotely located multimedia client device operated for another entity, said server device comprising configuration capabilities specifying how the received multimedia file can be analyzed;

said server device providing details of said configuration capabilities to said other entity using an interface;

said server device receiving from said other entity, a request to analyze said multimedia file;

said server device receiving from said other entity, at least one input specifying at least one of said configuration capabilities and, if necessary, additional configuration capabilities required to analyze said multimedia file in accordance with said request;

said server device using said at least one input to generate configuration settings which are specific to said multimedia file, said configuration settings reflecting said at least one input and being indicative of how to analyze objects captured by said multimedia file in order to extract desired information about particular ones of said objects for said other entity;

said server device having at least a portion of said multimedia file loaded and said particular ones of said objects analyzed according to said configuration settings to extract said desired information therefrom on behalf of said other entity;

said server device obtaining extracted information resulting from analysis of said particular objects using said configuration settings; and

said server device making said extracted information available to said other entity to complete said request.

2. The method according to claim 1 further comprising said server device reviewing said extracted information with respect to said multimedia file to verify correctness of said analysis and modifying said extracted information if one or more errors are found.

3. The method according to claim 2 wherein said reviewing is performed at one or more post processing sites, a copy of said multimedia file and said extracted information being provided to said post processing sites following said analysis.

4. The method according to claim 2 wherein said reviewing comprises identifying additional configuration settings or modifications of existing configuration settings and providing configuration feedback for adjusting said configuration settings for refining said analysis of said multimedia file.

5. The method according to claim 1 wherein said multimedia file comprises video content and said analysis comprises performing computer vision techniques to identify, classify and track objects identified in said video content.

6. The method according to claim 5 wherein said video content is received as a stream of compressed data from said client device and said method comprises decompressing said compressed data as it is received.

7. The method according to claim 1 wherein said receiving is initiated through an intermediary server over a network, said intermediary server communicating with said client device to redirect said multimedia file to one or more storage devices.

8. The method according to claim 1 wherein said server device utilizes a distributed computing system comprising one or more processing nodes for performing said analysis and one or more storage nodes for storing said multimedia file and said configuration settings, said processing nodes obtaining a copy of said multimedia file and a copy of said configuration settings from said storage nodes prior to said analysis.

9. The method according to claim 8 wherein said multimedia file is redirected by an intermediary server accessed by said client device directly to said storage nodes, said intermediary server having copies of said multimedia file and said configuration settings downloaded to said processing nodes for said analysis and being responsible for redirecting copies of said multimedia file and said extracted information to one or more post processing sites for reviewing said extracted information for correctness.

10. The method according to claim 1 wherein said configuration settings are generated at one or more configuration sites, said configuration sites accessing said multimedia file prior to said analysis and utilizing a configuration tool adapted to provide a plurality of options for defining attributes associated with selected portions of said multimedia file for extracting said extracted information.

11. A system for analyzing multimedia content comprising:

at least one server entity connected to one or more remote client devices over a network, said at least one server entity comprising one or more hardware processors being operable to;

receive a multimedia file generated by a remotely located multimedia client device operated for another entity, said at least one server entity comprising configuration capabilities specifying how the received multimedia file can be analyzed;

provide details of said configuration capabilities to said other entity using an interface;

receive from said other entity, a request to analyze said multimedia file;

receive from said other entity, at least one input specifying at least one of said configuration capabilities and, if necessary, additional configuration capabilities required to analyze said multimedia file in accordance with said request;

use said at least one input to generate configuration settings which are specific to said multimedia file, said configuration settings reflecting said at least one input and being indicative of how to analyze objects captured by said multimedia file in order to extract desired information about particular ones of said objects for said other entity;

have at least a portion of said multimedia file loaded and said particular ones of said objects analyzed according to said configuration settings to extract said desired information therefrom on behalf of said other entity;

obtain extracted information resulting from analysis of said particular objects using said configuration settings; and

make said extracted information available to said other entity to complete said request.

12. The system according to claim 11, said one or more processors being further operable to:

review said extracted information to verify correctness of said analysis and;

correct said extracted information if one or more errors are found.

13. The system according to claim 12, said at least one server entity comprising one or more post processing sites, wherein said extracted information is reviewed at said one or more post processing sites, a copy of said multimedia file and said extracted information being provided to said post processing sites following said analysis.

14. The system according to claim 12 wherein said extracted information is reviewed by identifying additional configuration settings or modifications of existing configuration settings and providing configuration feedback for adjusting said configuration settings for refining analysis of related multimedia file.

15. The system according to claim 11 wherein said multimedia file comprises video content and said analysis comprises performing computer vision techniques to identify, classify and track objects identified in said video content.

16. The system according to claim 15 wherein said video content is received as a stream of compressed data from said client device and one of said at least one server entity is configured for decompressing said compressed data as it is received.

17. The system according to claim 11, said at least one server entity comprising an intermediary server connected to said client device over a network, wherein receipt of said multimedia file is initiated through said intermediary server, said intermediary server communicating with said client device to redirect said multimedia file to one or more storage devices.

18. The system according to claim 11, said at least one server entity utilizing a distributed computing system comprising one or more processing nodes for performing said analysis and one or more storage nodes for storing said multimedia file and said configuration settings, said processing nodes configured for obtaining a copy of said multimedia file and a copy of said configuration settings from said storage nodes prior to said analysis.

19. The system according to claim 18, said at least one server entity comprising an intermediary server accessed by said client device, wherein said multimedia file is redirected by said intermediary server directly to said storage nodes, said intermediary server having copies of said multimedia file and said configuration settings downloaded to said processing nodes for said analysis and being responsible for redirecting copies of said multimedia file and said extracted information to one or more post processing sites for reviewing said extracted information for correctness.

20. The system according to claim 11, said at least one server entity comprising one or more configuration sites, wherein said configuration settings are generated at said one or more configuration sites, said configuration sites configured for accessing said multimedia file prior to said analysis and utilizing a configuration tool adapted to provide a plurality of options for defining attributes associated with selected portions of said multimedia file for extracting said extracted information.