Shared media crawler database method and system

• US 9,171,003 B2
• Filed: 03/15/2013
• Issued: 10/27/2015
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: JP Morgan Chase Bank, N.A. (JP Morgan Chase & Co)
• Original Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Inventors: Ravipati, Satyanarayana, Deglurkar, Anirudha
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US13/841,789
• Publication Number:
  US 20140280010A1
• Time in Patent Office: 956 Days
• Field of Search: 707/709
• US Class Current: 1/1
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata

Abstract

The embodiments relate to transcoding, cataloging, and extracting metadata about files stored in a storage device. In one embodiment, a crawler runs on the storage device and maintains a database that is stored in the volume with the data that has been cataloged by the crawler. The crawler may discover files of any type and extract associated metadata about the files. The crawler can extract metadata about client interaction with various files, such as edits, play counts, etc. The crawler may discover files of any type and extract associated metadata about the files automatically during a scan or at the request of a client. In one embodiment, the crawler may be responsive to file system events that indicate changes to the file system, such as additions, deletions, or other types of changes. In addition, the crawler may synchronize the database with the file system so that they indicated the same state for a particular file. Furthermore, the crawler may provide notifications to various entities regarding the state of a file.

Claims

1. A method of monitoring files stored in a volume comprising at least one storage device, comprising:

monitoring, by a crawler, the at least one storage device;

maintaining, by the crawler, a catalog for files stored within the volume;

receiving, by the crawler, an event indicating a change to a file stored on the at least one storage device;

transcoding, by the crawler, the file from a first format to a second format;

extracting, by the crawler, metadata for the file;

storing, by the crawler in the volume, information indicating the change and the transcoding in the catalog of the files;

determining a set of subscribers that have requested to receive a notification of the event;

retrieving notification criteria for the set of subscribers from the catalog of files;

evaluating the retrieved notification criteria for each of the set of subscribers; and

sending a notification of the event to those subscribers whose notification criteria are satisfied.

2. The method of claim 1, wherein the catalog is specific to each volume provided by the storage device.

3. The method of claim 1, wherein the catalog accompanies the files stored on the storage device.

4. The method of claim 1, further comprising determining, by the crawler, a thumbnail indicating contents for each file.

5. The method of claim 1, further comprising transcoding, by the crawler, a format of a file serviced from the storage device.

6. The method of claim 1, wherein logging, by the crawler, changes to the files stored on the storage device comprises subscribing to a file notification service provided by an application framework running on the storage device.

7. The method of claim 1, further comprising:

providing an interface to the catalog of files stored in the storage device;

receiving a request for information for a file indicated in a presented view; and

retrieving, by the crawler, the requested information about the file.

8. The method of claim 7, further comprising providing, from the crawler, a notification indicating the information about the requested file.

9. The method of claim 1, further comprising:

receiving a notification that a new storage media has been added to a volume of the storage device; and

scanning, by the crawler, the new storage media.

10. The method of claim 1, further comprising sending, by the crawler, messages indicating changes to the files stored on the at least one storage device via a named pipe.

11. The method of claim 1, wherein logging changes to the files stored on the at least one storage device comprises receiving updates from a file notification service running as a user space application on the storage device.

12. The method of claim 11, wherein receiving updates from the file notification service comprises receiving file notification messages from a notification service running within an application framework installed on the at least one storage device.

13. The method of claim 11, wherein receiving updates from the file notification service comprises receiving file notification messages from an operating system service running on the at least one storage device.

14. A storage device comprising:

at least one volume of storage for storing files;

at least one storage medium providing one or more volumes of storage for a plurality of files; and

a controller, configured by executable program code, to execute an operating system, and a crawler application, the crawler application being configured to crawl the at least one storage medium responsive to a change in at least one of the volumes, to determine files stored in the at least one volume and metadata for each of the files, and to generate a catalog for the files stored in the at least one volume that indicates changes to the metadata and changes to the files; and

a dispatcher configured to provide information about content of the files stored in the at least one volume, the dispatcher being configured to determine a set of subscribers that have requested to receive a notification of an event, retrieve notification criteria for the set of subscribers from the catalog of files, evaluate the retrieved notification criteria for each of the set of subscribers; and

cause a notification of the event to be sent to those subscribers whose notification criteria are satisfied.

15. The storage device of claim 14, wherein the at least one storage medium comprises a plurality of disks configured as a redundant array of independent disks.

16. The storage device of claim 14, wherein the controller is configured to execute a Linux operating system.

17. The storage device of claim 16, wherein the controller is configured to execute an application framework running on the Linux operating system.

18. The storage device of claim 14, wherein the controller is configured to provide access to the catalog via an application programming interface.

19. The storage device of claim 14, wherein the controller is configured to provide access to the catalog.

20. The storage device of claim 14, wherein the crawler is configured to receive notification of changes to the files stored in the at least one volume from a notification service.

21. The storage device of claim 14, wherein the crawler is configured to determine changes to at least one of the volumes from an event provided from a disk watcher running on the storage device.