METHOD AND APPARATUS FOR AN APPLICATION CRAWLER

• US 20090216758A1
• Filed: 03/18/2009
• Published: 08/27/2009
• Est. Priority Date: 11/22/2004
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Truveo Incorporated (Warner Bros. Discovery, Inc.)
• Inventors: Tuttle, Timothy D., Kocks, Peter F., Beguelin, Adam L.
• Granted Patent: US 8,954,416 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A computer-implemented method is provided for searching for files on the Internet. In one embodiment, the method may provide an application crawler that assembles and dynamically instantiates all components of a web page. The instantiated web application may then be analyzed to locate desired components on the web page. This may involve finding and analyzing all clickable items in the application, driving the web application by injecting events, and extracting information from the application and writing it to a file or database.

Claims

1. A computer-implemented method, comprising:

loading multiple documents or applications;

assembling the multiple documents or applications;

executing the loaded and assembled multiple documents or applications to instantiate the multiple documents or applications into a completely functioning application;

indexing an object model of the multiple running, instantiated documents or applications including identifying objects that are loaded once the documents or applications have been instantiated;

applying data-query interfaces to media player objects in the instantiated documents or applications; and

using the data-query interfaces to query the media player objects for media player properties and for metadata about any downloaded audio or video streams.

2. The method of claim 1, further comprising crawling the object model of the multiple running, instantiated documents or applications, wherein crawling comprises at least one of:

following a tree of clickable items and activating items in an automated manner to simulate behavior of a human user;

or

following a seed list of pages or applications.

3. The method of claim 1, wherein indexing comprises at least one of:

traversing an object tree in the object model of the instantiated documents or applications;

traversing an object tree in the object model of the instantiated documents or applications and simulating mouse, keyboard, or other user events;

traversing an object tree in the object model of the instantiated documents or applications and recording a location and contents of an object of the object tree;

saving a plurality of uniform resource locators (URLs) associated with media into a database;

or

instantiating at least one of video files or video streams.

4. The method of claim 1, wherein indexing comprises reaching into any node in the object model of the running, instantiated documents or application and performing at least one of:

recording the node in a database;

or

saving a pointer to the node in a database.

5. The method of claim 1, further comprising crawling the object model of the multiple running, instantiated documents or applications, wherein crawling comprises at least one of:

crawling the following types of networks;

intranet, single machine, or multiple apps on a single machine;

crawling the Internet;

crawling any device on a TCP/IP network;

crawling any device on a public network;

or

crawling any device on a private network.

6. A computer-implemented method for searching for video files on a computer network, the method comprising:

loading multiple documents or applications;

assembling the multiple documents or applications;

executing the loaded and assembled multiple documents or applications to initially instantiate at least a portion of a functioning application;

at an initial point in time, crawling and indexing an object model of the multiple running, initially instantiated documents or applications including identifying objects that are loaded once the documents or applications have been initially instantiated;

relating information gathered from crawling and indexing the at least a portion of the functioning application object model of the initially instantiated documents or applications with objects that are displayed at the initial point in time;

continuing to instantiate the at least a portion of the functioning application, wherein objects are loaded, assembled and executed to subsequently instantiate the at least a portion of the functioning application;

at a subsequent point in time, crawling and indexing an object model of the multiple running, subsequently instantiated documents or applications including identifying objects that are loaded after subsequently instantiating the at least a portion of the functioning application; and

relating information gathered from crawling and indexing the at least a portion of the functioning application object model of the subsequently instantiated documents or applications with objects that are displayed at the subsequent point in time.

7. The method of claim 6, wherein crawling comprises at least one of:

following a tree of clickable items and activating items in an automated manner to simulate behavior of a human user;

or

following a seed list of pages or applications.

8. The method of claim 6, wherein indexing comprises at least one of:

traversing an object tree in the object model of the instantiated documents or applications;

loading the documents or applications;

instantiating the documents or applications;

reaching into any node in the object model of the running, instantiated documents or application and either saving or recording the node in a database;

saving a plurality of uniform resource locators (URLs) associated with media into a database;

or

simulating mouse, keyboard, or other user events.

9. The method of claim 6, wherein crawling comprises at least one of:

crawling the following types of networks;

intranet, single machine, or multiple apps on a single machine;

crawling the Internet;

crawling any device on a TCP/IP network;

crawling any device on a public network;

or

crawling any device on a private network.

10. The method of claim 6, wherein indexing also instantiates at least one of video files or video streams.

11. The method of claim 6, further comprising adding data-query interfaces to software objects in the running instantiated documents or applications to allow for interface with an application crawler.

12. A computer-implemented method for creating a searchable database, the method comprising:

crawling an object model of multiple running, instantiated documents or applications to locate video files;

indexing video files found in the object model by saving pointers to the video files in the database;

extracting metadata about the video files from the object model;

saving the metadata in the database;

detecting when a video file has been initiated;

extracting metadata in real-time as the video file is being played; and

relating the real-time metadata with objects that are displayed at the same point in time.

13. A computer-implemented method for searching for files on the Internet, the method comprising:

providing a protocol crawler for identifying video-rich web sites; and

providing an application crawler comprising;

an inspector for dynamically instantiating and assembling all components of a web page at one of said video-rich web sites to create at least one instantiated web application;

an extractor for identifying specific parts of the instantiated web application that contain useful information and providing the logic required to extract that information into a metadata record including applying data-query interfaces to media player objects in the at least one instantiated web application and using the data-query interfaces to query the media player objects for media player properties and for metadata about any downloaded audio or video streams; and

a crawler for analyzing the instantiated web application, finding and analyzing all clickable items in the application, driving the web application by injecting events, and extracting information from the application and writing it to a file or database.

14. The method of claim 13, wherein the inspector includes code for software components for at least one of the following:

a Document Object Model (DOM) implementation for one or multiple browsers;

a scripting engine capable of executing JavaScript, JScript, ECMAScript or VBScript;

an XML parsing engine;

a Cascading Style Sheet engine;

a network I/O library;

an HTML parsing and rendering engine;

an engine for executing embedded controls;

or

an engine for rendering web applications.

15. The method of claim 13 wherein the extractor includes code for software components for at least one of the following:

an XSL engine;

an XPath implementation;

a regular expression engine;

a script execution engine;

an embedded object inspector for components;

a network transport proxy;

a multimedia stream proxy;

a software bridge to process data with class libraries of external programming frameworks;

a taxonomy engine for categorizing metadata;

or

a text parsing and processing engine.

16. The method of claim 13, wherein the crawler includes code for software components for at least one of the following:

a file I/O library;

a network I/O library;

or

a library for generating and storing logfiles.

17. A computer system having a storage device having computer-executable code stored thereon and a processor, the computer system comprising:

an application crawler having programming code for crawling and indexing an object model of running, instantiated documents or applications from one or more web sites including;

at an initial point in time, crawling and indexing an object model of the multiple running, initially instantiated documents or applications including identifying objects that are loaded once the documents or applications have been initially instantiated;

relating information gathered from crawling and indexing the at least a portion of the functioning application object model of the initially instantiated documents or applications with objects that are displayed at the initial point in time;

continuing to instantiate the at least a portion of the functioning application, wherein objects are loaded, assembled and executed to subsequently instantiate the at least a portion of the functioning application;

at a subsequent point in time, crawling and indexing an object model of the multiple running, subsequently instantiated documents or applications including identifying objects that are loaded after subsequently instantiating the at least a portion of the functioning application; and

relating information gathered from crawling and indexing the at least a portion of the functioning application object model of the subsequently instantiated documents or applications with objects that are displayed at the subsequent point in time.

18. The computer system of claim 17, further comprising:

a protocol crawler for identifying one or more web sites for inspection wherein the application crawler crawls and indexes the object model of running, instantiated documents or applications from the one or more web sites.

19. The computer system of claim 18, wherein the protocol crawler identifies one or more web sites for inspection that contain at least one video file or at least one media file.

20. The computer system of claim 17, wherein the application crawler comprises at least one of:

an inspector for dynamically instantiating and assembling all components of a web page at one of the one or more web sites to create at least one instantiated web application;

an extractor for identifying specific parts of the instantiated web application that contain useful information and providing the logic required to extract that information into a metadata record;

or

a crawler for analyzing the instantiated web application, finding and analyzing all clickable items in the application, driving the web application by injecting events, and extracting information from the application and writing it to a file or database.

21. The computer system of claim 17, wherein the application crawler is configured to inspect a tree of clickable items and activate each item in an automated manner to simulate behavior of a human user.

22. The computer system of claim 17, wherein the application crawler uses a template configured for at least one of the following:

data extraction, timing of when to follow a link, depth to crawl, how to skip a commercial, where to start a crawl, finding links, location of title, location of media file metadata, temporal synchronization, or instructing the crawler to wait certain time intervals and then crawl the object model again.

23. A computer system having a storage device having computer-executable code stored thereon and a processor, the computer system comprising:

a protocol crawler for identifying video-rich websites; and

an application crawler comprising;

an inspector for dynamically instantiating and assembling all components of a web page at one of said video-rich websites to create at least one instantiated web application;

an extractor for identifying specific parts of the instantiated web application that contain useful information and providing the logic required to extract that information into a metadata record; and

a crawler for analyzing the instantiated web application, finding and analyzing all clickable items in the application, driving the web application by injecting events, detecting when a video file has been initiated, and extracting information from the application and writing it to a file or database including extracting metadata in real-time as the video file is being played and relating the real-time metadata with objects that are displayed at the same point in time.

24. The computer system of claim 23, wherein the inspector includes code for software components for at least one of the following:

a Document Object Model (DOM) implementation for one or multiple browsers;

a scripting engine capable of executing JavaScript, JScript, ECMAScript or VBScript;

an XML parsing engine;

a Cascading Style Sheet engine;

a network I/O library;

an HTML parsing and rendering engine;

an engine for executing embedded controls;

or

an engine for rendering web applications.

25. The computer system of claim 23, wherein the extractor includes code for software components for at least one of the following:

an XSL engine;

an XPath implementation;

a regular expression engine;

a script execution engine;

an embedded object inspector for components;

a network transport proxy;

a multimedia stream proxy;

a software bridge to process data with class libraries of external programming frameworks;

a taxonomy engine for categorizing metadata;

or

a text parsing and processing engine.

26. The computer system of claim 23, wherein the crawler includes code for software components for at least one of the following:

a file I/O library;

a network I/O library;

or

a library for generating and storing logfiles.