×

Distributed processing framework file system fast on-demand storage listing

  • US 10,599,621 B1
  • Filed: 02/02/2015
  • Issued: 03/24/2020
  • Est. Priority Date: 02/02/2015
  • Status: Active Grant
First Claim
Patent Images

1. A computer-implemented method, comprising:

  • receiving a request from a requestor to provide a listing of keys for data objects in a directory of a distributed processing framework file system, wherein the data objects are stored with a storage service of a computing resource service provider;

    obtaining a set of keys from a data store of the computing resource service provider, wherein each key of the set of keys from the data store corresponds to a data object in the data objects stored with the storage service;

    determining a first starting key and a second starting key based at least in part on a scheme for partitioning the set of keys;

    generating a first sub-listing of keys by executing a first thread, wherein the first sub-listing of keys includes one or more keys in an order from the first starting key to the second starting key;

    generating a second sub-listing of keys by executing a second thread, wherein the second sub-listing of keys includes one or more keys in the order from the second starting key to an end of the data objects, and wherein the second thread executes in parallel with the first thread;

    merging the first sub-listing of keys with the second sub-listing of keys to produce the listing of keys; and

    providing the listing of keys to the requestor.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×