Methods and apparatus for implementing host-based object storage schemes

• US 6,754,800 B2
• Filed: 11/14/2001
• Issued: 06/22/2004
• Est. Priority Date: 11/14/2001
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Wong, Thomas K., Chawla, Rajeev, Tsirigotis, Panagiotis, Radia, Sanjay R.
• Primary Examiner(s): MCLEAN MAYO, KIMBERLY N
• Application Number:
  US09/987,249
• Publication Number:
  US 20030093645A1
• Time in Patent Office: 951 Days
• Field of Search: 711/216, 711/118, 711/119, 709/203, 707/100
• US Class Current: 711/216
• CPC Class Codes: G06F 12/0864 using pseudo-associative means, e.g. set-associative or hashing
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Systems and methods are disclosed in which a computer system having main memory and persistent memory is caused to perform a method for caching related objects. The computer system receives a plurality of objects from an origin server and computes a hash value based on source information about an object. Then the computer system stores the object based on the hash value with other related objects. Additionally, a computer system consistent with the present invention may retrieve related objects from the cache by performing a batch read of related objects.

Claims

1. In a system having main memory and persistent memory, the main memory containing at least one data buffer, a method for caching related objects, the method comprising:

receiving a plurality of objects from at least one origin server;

computing a hash value corresponding to source identifying information for at least one object in the plurality of objects, wherein computing comprises;

computing a first hash value based on an object identifier associated with the object server; and

computing a second hash value based on a host identifier associated with the origin server; and

storing, in at least one data structure of a plurality of data structures, the object based on the computed hash value, wherein the object is stored with related objects having related hash values.

2. A method according to claim 1, wherein the plurality of data structures further comprises:

an index buffer capable of storing index entries;

an object buffer capable of storing and retrieving objects by logical block number; and

a metadata buffer capable of storing information about the index buffer and the object buffer.

3. A method according to claim 2, wherein the index buffer is a cyclic index buffer.

4. A method according to claim 1, wherein the object identifier is a uniform resource locator.

5. A method according to claim 4, wherein the host identifier is a domain-name-service host name associated with the origin server.

6. A method according to claim 1, wherein storing the object using a plurality of data structures further comprises:

selecting a write buffer based on the second hash value, the write buffer selected from a plurality of write buffers; and

storing information about the object in a hash table using the first hash value.

7. A method according to claim 6, wherein the write buffer is contained in fast main memory.

8. A method according to claim 6, wherein the persistent memory comprises a disk storage device.

9. A method according to claim 5, wherein storing the object further comprises:

selecting a write buffer based on the second hash value, the write buffer selected from a plurality of write buffers; and

storing information about the object in a hash table using the first hash value.

10. A method according to claim 6, further comprising:

selecting a persistent storage device on which to write objects to be written that are stored in fast main memory;

retrieving hash table entries corresponding to the objects to be written; and

modifying the hash table entries to indicate that the objects to be written are stored in persistent storage.

11. A method according to claim 10, wherein the persistent storage device is a disk storage device.

14. A method according to claim 10, further comprising:

storing information about the objects to be written in the index buffer; and

batch writing the objects to be written to persistent storage.

12. A method according to claim further comprising:

selecting a persistent storage device on which to write objects to be written that are stored in fast main memory;

retrieving hash table entries corresponding to the objects to be written; and

modifying the hash table entries to indicate that the objects to be written are stored in persistent storage.

13. A method according to claim 12, wherein the persistent storage device is a non-volatile memory storage device.

15. In a system having main memory and persistent memory, the main memory containing at least one data buffer, a method for retrieving cached related objects comprising:

receiving a request for an object, the request comprising an object identifier;

looking up the requested object in a hash table using the object identifier;

determining whether the object is stored in persistent memory;

retrieving a group of related objects from an object buffer capable of storing and retrieving objects by logical block number, when it is determined that the object is stored in persistent memory; and

discarding those retrieved objects that do not have an origin server host identifier in common with the requested object.

16. A method according to claim 15, wherein receiving a request for an object further comprises receiving a request from a web client for a web object.

17. A method according to claim 15, wherein the object identifier is a uniform resource locator.

18. A method according to claim 15, wherein retrieving a group of related objects from an object buffer further comprises:

reading an integer number of logical blocks in a single read operation.

19. A computer system having main memory and persistent memory, the main memory containing at least one data buffer, and an execution unit capable of executing program code, the computer system comprising:

a network interface capable of receiving information comprising a plurality of objects from at least one host computer;

a controller configured to calculate a hash value corresponding to source identifying information for at least one object in the plurality of objects by at least computing a first hash value based on an object identifier associated with the object, and computing a second hash value based on a host identifier associated with the origin server; and

a memory used to store, in at least one data structure of a plurality of data structures, the object based on the hash value, wherein the object is stored with related objects having related hash values.

20. A computer system comprising:

means for receiving data comprising a plurality of objects from at least one origin server;

means for computing a hash value corresponding to source identifying information for at least one object in a plurality of objects, wherein computing comprises;

computing a first hash value based on an object identifier associated with the object, and

computing a second hash value based on a host identifier associated with the origin server; and

storing, in at least one data structure of a plurality of data structures, the object based on the computed hash value, wherein the object is stored with related objects having corresponding hash values.

21. A computer-readable medium containing program code capable of causing a computer system to execute a method for caching related objects, the computer-readable medium comprising:

program code operative to receive a plurality of objects from at least one origin server;

program code operative to compute a hash value corresponding to source identifying information for at least one object in the plurality of objects by at least computing a first hash value based on an object identifier associated with the object, and computing a second hash value based on a host identifier associated with the origin server; and

program code operative to store, in at least one data structure of a plurality of data structures, the object based on the computed hash value, wherein the object is stored with related objects having corresponding hash values on a disk storage device.

22. A computer-readable medium according to claim 21, wherein the program code operative to store the object further comprises:

program code operative to select a memory region based on the second hash value, the memory region selected from a plurality of memory regions; and

program code operative to store information about the object in a hash table using the first hash value.

23. A computer-readable medium according to claim 22, further comprising:

program code operative to select a disk on which to write information comprising a plurality of objects from fast main memory;

program code operative to retrieve hash table entries corresponding to the objects to be written; and

program code operative to modify the hash table entries to indicate that the objects to be written are stored on disk.

24. A computer-readable medium according to claim 23, further comprising:

program code operative to store an identification of the objects to be written in an index buffer; and

program code operative to cause the objects to be written to disk in at least one batch write operation.

25. A computer-readable medium having main memory and persistent memory, the main memory containing at least one data buffer, and an execution unit capable of executing program code to perform a method for retrieving cached related objects, the computer-readable medium comprising:

program code operative to receive a request for an object, the request comprising an object descriptor;

program code operative to identify the requested object based on the object descriptor, using a hash table;

program code operative to determine whether the object is stored on disk;

program code operative to accept a group of related objects from an object buffer, when it is determined that the object is stored on disk; and

program code operative to discard those objects that do not have an origin server host identifier in common with the requested object.

26. A computer-readable medium according to claim 25, wherein the program code operative to accept a group of related objects from an object buffer further comprises:

program code operative to batch read information from disk.