METHODS AND SYSTEMS FOR MANAGING DATA

• US 20100145949A1
• Filed: 12/11/2009
• Published: 06/10/2010
• Est. Priority Date: 06/25/2004
• Status: Active Grant

• Current Assignee: Andrew Carol, Dominic Giampaolo, Yan Arrouye
• Original Assignee: Andrew Carol, Dominic Giampaolo, Yan Arrouye
• Inventors: Carol, Andrew, Giampaolo, Dominic, Arrouye, Yan
• Granted Patent: US 8,131,775 B2
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/164 File meta data generation
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

Systems and methods for managing data, such as metadata or indexes of content of files. In one exemplary method, notifications to update a metadata database or an index database are combined into a combined notification. According to other aspects, an order among logical locations on a storage device is determined in order to specify a sequence for scanning for files to be indexed. According to another aspect, a method includes determining whether to index a file based on a path name of the file relative to a plurality of predetermined path names.

Claims

1. A machine implemented method of processing data, the method comprising:

receiving a first notification signaling that a first file has been modified;

adding the first notification to a first queue;

combining, by a data processing system, notifications into a combined notification for updating a metadata database or an index database, wherein the combining comprises dropping one of a second notification or the first notification and wherein each notification is provided by an operating system component;

updating the metadata database or the index database after receiving the combined notification.

2. A method as in claim 1 wherein, in the metadata database, the type of information in metadata for files of a first type differs from the type of information in metadata for files of a second type and wherein the combined notification further comprises data from the second notification.

3. A method as in claim 2 further comprising:

filtering the notifications before the combining of notifications, wherein notifications for files that are not to be indexed are prevented from causing the metadata database to be updated.

4. A method as in claim 3 wherein the filtering is based on a file pathname of the file and a plurality of predetermined pathnames.

46. A method as in claim 1, wherein the combining further comprises:

determining when a next notification is likely to be a last notification and adding the last notification to the first queue.

47. A method as in claim 1, wherein the combined notification comprises the last notification.

5. A method as in claim I wherein each of the notifications indicate a creation or modification of a file on a storage device.

6. A machine readable storage medium providing instructions which when executed by a data processing system cause the data processing system to perform a method of processing data, the method comprising:

receiving a first notification signaling that a first file has been modified;

adding the first notification to a first queue;

combining notifications into a combined notification for updating a metadata database or an index database, wherein the combining comprises dropping one of a second notification or the first notification and wherein each notification is provided by an operating system component;

updating the metadata database or the index database after receiving the combined notification.

7. The machine readable storage medium as in claim 6 wherein, in the metadata database, the type of information in metadata for files of a first type differs from the type of information in metadata for files of a second type and wherein the combined notification further comprises data from the second notification.

8. The machine readable storage medium as in claim 7, the method further comprising:

filtering the notifications before the combining of notifications, wherein notifications for files that are not to be indexed are prevented from causing the metadata database to be updated.

9. The machine readable storage medium as in claim 8 wherein the filtering is based on a file pathname of the file and a plurality of predetermined pathnames.

10. The machine readable storage medium as in claim 6 wherein each of the notifications indicate a creation or modification of a file on a storage device.

48. The machine readable storage medium as in claim 6, wherein the combining further comprises:

determining when a next notification is likely to be a last notification and adding the last notification to the first queue.

49. The machine readable storage medium as in claim 6, wherein the combined notification comprises the last notification.

11. A data processing system comprising:

means for receiving a first notification signaling that a first file has been modified;

means for adding the first notification to a first queue;

means for combining, by a hardware device, notifications into a combined notification for updating a metadata database or an index database, wherein the combining comprises dropping one of a second notification or the first notification and wherein each notification is provided by an operating system component;

means for updating the metadata database or the index database after receiving the combined notification.

12. A system as in claim 11 wherein, in the metadata database, the type of information in metadata for files of a first type differs from the type of information in metadata for files of a second type and wherein the combined notification further comprises data from the second notification,

13. A system as in claim 12 further comprising:

means for filtering the notifications before the combining of notifications, wherein notifications for files that are not to be indexed are prevented from causing the metadata database to be updated.

14. A system as in claim 13 wherein the filtering is based on a file pathname of the file and a plurality of predetermined pathnames.

15. A system as in claim 11 wherein each of the notifications indicate a creation as modification of a file on a storage device.

50. A system as in claim 11, wherein the combining further comprises:

means for determining when a next notification is likely to be a last notification and adding the last notification to the first queue.

51. A system as in claim 11, wherein the combined notification comprises the last notification.

16. A machine implemented method of processing data, the method comprising:

receiving a first notification signaling that a first file has been modified;

adding the first notification to a first queue;

grouping, by a data processing system, a first set of notifications into a second set of notifications for updating a metadata database or an index database, wherein the grouping comprises dropping one of a second notification or the first notification;

retaining data from the at least one notification which was dropped;

updating the metadata database or the index database after receiving the second set of notifications.

17. A method as in claim 16 wherein the second set is smaller than the first set which is at least two notifications while the second set is two or less than two.

18. A method as in claim 16 wherein, in the metadata database, the type of information in metadata for files of a first type differs from the type of information in metadata for files of a second type.

19. A method as in claim 16 wherein the grouping is in response to determining that files identified by the first set of notifications are part of a package.

20. A method as in claim 16 wherein each of the notifications indicate a creation or modification of a file on a storage device and is provided by an operating system component.

21. A method as in claim 16 further comprising:

filtering the first set of notifications before the grouping of notifications, wherein notifications for files that are not to be indexed are prevented from causing the metadata database to be updated.

22. A method as in claim 16, wherein the second set of notifications further comprises the first notification.

23. A method as in claim 22, wherein the second set of notifications further comprises data from the second notification.

24. A method as in claim 16, wherein the combining further comprises:

determining when a next notification is likely to be a last notification and adding the last notification to the first queue.

25. A method as in claim 16, wherein the second set of notifications comprises the last notification.

52. The method of claim 16, further comprising:

determining whether to update the metadata database or the index database based on the retained data.

26. A machine readable storage medium providing instructions which when executed by a data processing system cause the data processing system to perform a method of processing data, the method comprising:

receiving a first notification signaling that a first file has been modified;

adding the first notification to a first queue;

grouping a first set of notifications into a second set of notifications for updating a metadata database or an index database, wherein the grouping comprises dropping one of a second notification or the first notification;

retaining data from the at least one notification which was dropped;

updating the metadata database or the index database after receiving the second set of notifications.

27. The machine readable storage medium as in claim 26 wherein the second set is smaller than the first set which is at least two notifications while the second set is two or less than two.

28. The machine readable storage medium as in claim 26 wherein, in the metadata database, the type of information in metadata for files of a first type differs from the type of information in metadata for files of a second type.

29. The machine readable storage medium as in claim 26 wherein the grouping is in response to determining that files identified by the first set of notifications are part of a package.

30. The machine readable storage medium as in claim 26 wherein each of the notifications indicate a creation or modification of a file on a storage device and is provided by an operating system component.

31. The machine readable storage medium as in claim 26 further comprising:

filtering the first set of notifications before the grouping of notifications, wherein notifications for files that are not to be indexed are prevented from causing the metadata database to be updated.

32. The machine readable storage medium as in claim 26, wherein the second set of notifications further comprises the first notification.

33. The machine readable storage medium as in claim 32, wherein the second set of notifications further comprises data from the second notification.

34. The machine readable storage medium as in claim 26, wherein the combining further comprises:

determining when a next notification is likely to be a last notification and adding the last notification to the first queue.

35. The machine readable storage medium as in claim 26, wherein the second set of notifications comprises the last notification.

53. The medium readable storage medium of claim 26, the method further comprising:

determining whether to update the metadata database or the index database based on the retained data.

36. A data processing system comprising;

means for receiving a first notification signaling that a first file has been modified;

means for adding the first notification to a first queue;

means for grouping, by a hardware device, a first set of notifications into a second set of notifications for updating a metadata database or an index database, wherein the grouping comprises dropping one of a second notification or the first notification;

means for retaining data from the at least one notification which was dropped;

means for updating the metadata database or the index database after receiving the second set of notifications.

37. A system as in claim 36 wherein the second set is smaller than the first set which is at least two notifications while the second set is two or less than two.

38. A system as in claim 36 wherein, in the metadata database, the type of information in metadata for files of a first type differs from the type of information in metadata for files of a second type.

39. A system as in claim 36 wherein the grouping is in response to determining that files identified by the first set of notifications are part of a package.

40. A system as in claim 36 wherein each of the notifications indicate a creation or modification of a file on a storage device and is provided by an operating system component.

41. A system as in claim 36 further comprising:

means for filtering the first set of notifications before the grouping of notifications, wherein notifications for files that are not to be indexed are prevented from causing the metadata database to be updated.

42. A system as in claim 36, wherein the second set of notifications further comprises the first notification.

43. A system as in claim 42, wherein the second set of notifications further comprises data from the second notification.

44. A system as in claim 36, wherein the combining further comprises:

means for determining when a next notification is likely to be a last notification and adding the last notification to the first queue.

45. A system as in claim 36, wherein the second set of notifications comprises the last notification.

54. The system of claim 36, further comprising:

means for determining whether to update the metadata database or the index database based on the retained data.