Method and apparatus for identifying and characterizing errant electronic files

• US 6,922,781 B1
• Filed: 04/29/2000
• Issued: 07/26/2005
• Est. Priority Date: 04/30/1999
• Status: Expired due to Term

• Current Assignee: Intellectual Ventures I LLC (Intellectual Ventures LLC)
• Original Assignee: IdeaFlood, Inc.
• Inventors: Shuster, Gary Stephen
• Primary Examiner(s): Darrow, Justin T.
• Application Number:
  US09/561,751
• Time in Patent Office: 1,914 Days
• Field of Search: 713/165, 713/188, 713/200, 713/201
• US Class Current: 713/165
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 21/564 by virus signature recognition
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2221/2149 Restricted operating environment

Abstract

A computer system includes a server having a memory connected thereto. The server is adapted to be connected to a network to permit remote storage and retrieval of data files from the memory. A file identification application is operative with the server to identify errant files stored in the memory. The file identification application provides the functions of: (1) selecting a file stored in said memory; (2) generating a unique checksum corresponding to the stored file; (3) comparing said unique checksum to each of a plurality of previously generated checksums, wherein the plurality of previously generated checksums correspond to known errant files; and (4) marking the file for deletion from the memory if the unique checksum matches one of the plurality of previously generated checksums.

Claims

1. A method for identifying and characterizing stored electronic files, said method comprising:

selecting a file stored in a memory, said selecting step comprising identifying suspect files from a plurality of files stored in said memory, wherein said identifying step further comprises identifying sequential files from a directory of said memory;

generating a unique checksum corresponding to said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

2. The method according to claim 1, further comprising obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

3. The method according to claim 1, further comprising presenting said file marked for deletion for human review prior to deleting said file.

4. The method according to claim 1, further comprising automatically notifying a third party that said file has been marked for deletion.

5. A method for identifying and characterizing stored electronic files, said method comprising:

selecting a file stored in a memory, said selecting step comprising identifying suspect files from a plurality of files stored in said memory, wherein said identifying step further comprises identifying identically sized files from a directory of said memory;

generating a unique checksum corresponding to said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

6. The method according to claim 5, wherein said identifying step further comprises determining whether an aggregate size of plural identically sized files exceeds a predetermined threshold.

7. The method according to claim 5, further comprising obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

8. The method according to claim 5, further comprising presenting said file marked for deletion for human review prior to deleting said file.

9. The method according to claim 5, further comprising automatically notifying a third party that said file has been marked for deletion.

10. A method for identifying and characterizing stored electronic files, said method comprising:

selecting a file stored in a memory, said selecting step comprising identifying suspect files from a plurality of files stored in said memory, wherein said identifying step further comprises identifying files having content that fail to match their associated file type;

generating a unique checksum corresponding to said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

11. The method according to claim 10, further comprising obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

12. The method according to claim 10, further comprising presenting said file marked for deletion for human review prior to deleting said file.

13. The method according to claim 10, further comprising automatically notifying a third party that said file has been marked for deletion.

14. A method for identifying and characterizing stored electronic files, said method comprising:

selecting a file stored in a memory, said selecting step comprising identifying suspect files from a plurality of files stored in said memory, wherein said identifying step further comprises identifying files containing data that extends beyond an end of data marker;

generating a unique checksum corresponding to said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

15. The method according to claim 14, further comprising truncating said identified files after said end of data marker.

16. The method according to claim 14, further comprising obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

17. The method according to claim 14, further comprising presenting said file marked for deletion for human review prior to deleting said file.

18. The method according to claim 14, further comprising automatically notifying a third party that said file has been marked for deletion.

19. A method for identifying and characterizing stored electronic files, said method comprising:

selecting a file stored in a memory;

generating a unique checksum corresponding to said stored file, comprising generating a first checksum corresponding to a first one-thousand bytes of said stored file and a second checksum corresponding to a second portion of said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

20. The method according to claim 19, further comprising obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

21. The method according to claim 19, further comprising presenting said file marked for deletion for human review prior to deleting said file.

22. The method according to claim 19, further comprising automatically notifying a third party that said file has been marked for deletion.

23. A method for identifying and characterizing stored electronic files, said method comprising:

selecting a file stored in a memory;

generating a unique checksum corresponding to said stored file, comprising generating a first checksum corresponding to a first portion of said stored file and a second checksum corresponding to a first ten-thousand bytes of said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

24. The method according to claim 23, further comprising obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

25. The method according to claim 23, further comprising presenting said file marked for deletion for human review prior to deleting said file.

26. The method according to claim 23, further comprising automatically notifying a third party that said file has been marked for deletion.

27. A method for identifying and characterizing stored electronic files, said method comprising:

selecting a file stored in a memory;

generating a unique checksum corresponding to said stored file, comprising generating a unique checksum for graphics files based on vector graphics analysis;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

28. The method according to claim 27, further comprising obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

29. The method according to claim 27, further comprising presenting said file marked for deletion for human review prior to deleting said file.

30. The method according to claim 27, further comprising automatically notifying a third party that said file has been marked for deletion.

31. A method for identifying and characterizing stored electronic files, said method comprising:

selecting a file stored in a memory;

generating a unique checksum corresponding to said stored file, comprising dividing a graphical image file into blocks and comparing relationships between said blocks;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

32. The method according to claim 31, further comprising obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

33. The method according to claim 31, further comprising presenting said file marked for deletion for human review prior to deleting said file.

34. The method according to claim 31, further comprising automatically notifying a third party that said file has been marked for deletion.

35. A computer system, comprising:

a server having a memory connected thereto, said server being adapted to be connected to a network to permit remote storage and retrieval of data files from said memory; and

a file identification application operative with said server to identify errant files stored in said memory, said file identification application providing the functions of;

selecting a file stored in said memory, said selecting function comprising identifying suspect files from a plurality of files stored in said memory, wherein said identifying function further comprises identifying sequential files from a directory of said memory;

generating a unique checksum corresponding to said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

36. The computer system according to claim 35, wherein said file identification application further provides the function of obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

37. The computer system according to claim 35, wherein said file identification application further provides the function of presenting said file marked for deletion for human review prior to deleting said file.

38. The computer system according to claim 35, wherein said file identification application further provides the function of automatically notifying a third party that said file has been marked for deletion.

39. A computer system, comprising:

a server having a memory connected thereto, said server being adapted to be connected to a network to permit remote storage and retrieval of data files from said memory; and

a file identification application operative with said server to identify errant files stored in said memory, said file identification application providing the functions of;

selecting a file stored in said memory, said selecting function comprising identifying suspect files from a plurality of files stored in said memory, wherein said identifying function further comprises identifying identically sized files from a directory of said memory;

generating a unique checksum corresponding to said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

40. The computer system according to claim 39, wherein said identifying function further comprises determining whether an aggregate size of plural identically sized files exceeds a predetermined threshold.

41. The computer system according to claim 39, wherein said file identification application further provides the function of obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

42. The computer system according to claim 39, wherein said file identification application further provides the function of presenting said file marked for deletion for human review prior to deleting said file.

43. The computer system according to claim 39, wherein said file identification application further provides the function of automatically notifying a third party that said file has been marked for deletion.

44. A computer system, comprising:

a server having a memory connected thereto, said server being adapted to be connected to a network to permit remote storage and retrieval of data files from said memory; and

a file identification application operative with said server to identify errant files stored in said memory, said file identification application providing the functions of;

selecting a file stored in said memory, said selecting function comprising identifying suspect files from a plurality of files stored in said memory, wherein said identifying function further comprises identifying files having content that fails to match their associated file type;

generating a unique checksum corresponding to said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

45. The computer system according to claim 44, wherein said file identification application further provides the function of obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

46. The computer system according to claim 44, wherein said file identification application further provides the function of presenting said file marked for deletion for human review prior to deleting said file.

47. The computer system according to claim 44, wherein said file identification application further provides the function of automatically notifying a third party that said file has been marked for deletion.

48. A computer system, comprising:

a server having a memory connected thereto, said server being adapted to be connected to a network to permit remote storage and retrieval of data files from said memory; and

a file identification application operative with said server to identify errant files stored in said memory, said file identification application providing the functions of;

selecting a file stored in said memory, said selecting function comprising identifying suspect files from a plurality of files stored in said memory, wherein said identifying function further comprises identifying files containing data that extends beyond an end of data marker;

generating a unique checksum corresponding to said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

49. The computer system according to claim 48, wherein said identifying function further comprises truncating said identified files after said end of data marker.

50. The computer system according to claim 48, wherein said file identification application further provides the function of obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

51. The computer system according to claim 48, wherein said file identification application further provides the function of presenting said file marked for deletion for human review prior to deleting said file.

52. The computer system according to claim 48, wherein said file identification application further provides the function of automatically notifying a third party that said file has been marked for deletion.

53. A computer system, comprising:

a server having a memory connected thereto, said server being adapted to be connected to a network to permit remote storage and retrieval of data files from said memory; and

a file identification application operative with said server to identify errant files stored in said memory, said file identification application providing the functions of;

selecting a file stored in said memory;

generating a unique checksum corresponding to said stored file, comprising generating a first checksum corresponding to a first one-thousand bytes of said stored file and a second checksum corresponding to a second portion of said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

54. The computer system according to claim 53, wherein said file identification application further provides the function of obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

55. The computer system according to claim 53, wherein said file identification application further provides the function of presenting said file marked for deletion for human review prior to deleting said file.

56. The computer system according to claim 53, wherein said file identification application further provides the function of automatically notifying a third party that said file has been marked for deletion.

57. A computer system, comprising:

a server having a memory connected thereto, said server being adapted to be connected to a network to permit remote storage and retrieval of data files from said memory; and

a file identification application operative with said server to identify errant files stored in said memory, said file identification application providing the functions of;

selecting a file stored in said memory;

generating a unique checksum corresponding to said stored file, comprising generating a first checksum corresponding to a first portion of said stored file and a second checksum corresponding to a first ten-thousand bytes of said stored file;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

58. The computer system according to claim 57, wherein said file identification application further provides the function of obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

59. The computer system according to claim 57, wherein said file identification application further provides the function of presenting said file marked for deletion for human review prior to deleting said file.

60. The computer system according to claim 57, wherein said file identification application further provides the function of automatically notifying a third party that said file has been marked for deletion.

61. A computer system, comprising:

a server having a memory connected thereto, said server being adapted to be connected to a network to permit remote storage and retrieval of data files from said memory; and

a file identification application operative with said server to identify errant files stored in said memory, said file identification application providing the functions of;

selecting a file stored in-said memory;

generating a unique checksum corresponding to said stored file, wherein said generating function further comprises generating a unique checksum for graphics files based on vector graphics analysis;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

62. The computer system according to claim 61, wherein said file identification application further provides the function of obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

63. The computer system according to claim 61, wherein said file identification application further provides the function of presenting said file marked for deletion for human review prior to deleting said file.

64. The computer system according to claim 61, wherein said file identification application further provides the function of automatically notifying a third party that said file has been marked for deletion.

65. A computer system, comprising:

a server having a memory connected thereto, said server being adapted to be connected to a network to permit remote storage and retrieval of data files from said memory; and

a file identification application operative with said server to identify errant files stored in said memory, said file identification application providing the functions of;

selecting a file stored in said memory;

generating a unique checksum corresponding to said stored file, wherein said generating function further comprises dividing a graphical image file into blocks and comparing relationships between said blocks;

comparing said unique checksum to each of a plurality of previously generated checksums, said plurality of previously generated checksums corresponding to known errant files; and

marking said file for deletion from said memory if said unique checksum matches one of said plurality of previously generated checksums.

66. The computer system according to claim 65, wherein said file identification application further provides the function of obtaining a source of known errant files to generate said plurality of previously generated checksums therefrom.

67. The computer system according to claim 65, wherein said file identification application further provides the function of presenting said file marked for deletion for human review prior to deleting said file.

68. The computer system according to claim 65, wherein said file identification application further provides the function of automatically notifying a third party that said file has been marked for deletion.