Method and apparatus for scalable transport processing fulfillment system

• US 20050240625A1
• Filed: 04/23/2004
• Published: 10/27/2005
• Est. Priority Date: 04/23/2004
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Wal-Mart Stores Texas LLC (WalMart Inc.)
• Inventors: Harward, Nathaniel D., Hui, Simon Wing-Lok, Groza, Yulia, Chen, Steven S., Chang, Alvin Kai-Hau
• Granted Patent: US 7,822,779 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A scalable fulfillment system is presented that supports business processes, manages the transport and processing of business-related messages or documents between a business entity and clients, such as customers, vendors, and business partners, and generally supports business document processing. The system intelligently manages the transportation of files from clients, through processing of files, to generating responses that return to the clients. To accommodate a heterogeneous interfacing and processing with different clients in a central system, a file normalization technique is used that captures a common meta-data format from the numerous heterogeneous file types used by numerous clients. The meta-data files, also referred to as messages, contain links to their associated data files and are processed separately. Once the messages are entered into the system, an intelligent queue scheduler (IQS) is used to schedule the processing of messages across a scalable resource of processing engines based on normalized file meta-data parameters. The IQS supports serial processing file requirements in a parallel processing environment. The IQS in conjunction with processing daemons implement a dynamic queue that supports initial message entry and prioritization of messages within the IQS queue during the life cycle processing of a message. The processing daemons dynamically provide processing status, timestamps, message type information, client information and other similar information that can be used by the IQS in prioritizing processing steps. Message and file processing is partitioned in a series of separate action code modules that can be programmatically selected and sequenced in a desired processing pipelined order. By using a multi-pass action interfacing method, new actions can be defined and allowed to be pluggable in the system for processing new message types and adding new clients with possibly unique file types and interfaces. The ability to define new actions and define a programmable sequence of actions allows new capabilities of message processing as the system evolves.

Claims

1. A method for heterogeneous file processing in a fulfillment system, the method comprising:

storing a set of action tasks in an action task table according to expected types of client files;

extracting from a client repository a client file and associated meta-data which is information concerning the client file;

storing the client file;

creating a client message containing the extracted meta-data and a link reference to the client file;

enqueueing the client message on a work scheduling queue containing a plurality of client messages;

dequeueing the client message from the work scheduling queue;

selecting at least one action task from the action task table based on selected meta-data information from the client message; and

processing the client message according to the selected action task.

2. The method of claim 1 wherein the selected meta-data information comprises:

a client source name;

a type of file; and

an indication of direction of file processing as either incoming or outgoing.

3. The method of claim 1 wherein the at least one selected action task comprises a list of sub-action tasks.

4. The method of claim 1 further comprising:

selecting a second action task from the action task table; and

wherein the processing step further comprises processing the client file according to the second action task.

5. The method of claim 1 wherein the fulfillment system is an online order fulfillment system comprising:

a client gateway device as a source of client files;

an order fulfillment system interface; and

an internet connection medium for interconnecting the client gateway device to an order fulfillment system interface.

6. The method of claim 5 wherein the internet connection medium is a secure connection where the medium transferred across the internet is encrypted.

7. The method of claim 1 wherein the fulfillment system is an online document processing system comprising:

a client gateway device as a source of client documents;

a document processing system interface; and

an internet connection medium for interconnecting the client gateway device to a document processing system interface.

8. A method for scalable processing of client messages in a fulfillment system, the method comprising:

storing a set of action tasks in an action task table according to expected types of client files;

enqueueing a client message on a work scheduling queue, the client message containing meta-data concerning a client file;

dequeueing a client message according to a priority based upon a first set of meta-data information;

selecting an action task from the action task table based on a second set of meta-data information; and

processing the dequeued client message according to the selected action task on an available process from a group of parallel processes.

9. The method of claim 8 wherein processing the dequeued client message further comprises:

updating the meta-data contained in the client message for a new message type and a timestamp; and

enqueueing the updated client message on the work scheduling queue.

10. The method of claim 8 wherein the action tasks are each uniquely specified by a client source name, type of file, and direction of file processing.

11. The method of claim 8 wherein the first set of meta-data information comprises at least a timestamp.

12. The method of claim 8 wherein processing the dequeued message proceeds until the processing is successfully completed or fails and an indication is provided as to the success or failure of the processing so that if the processing is unsuccessful then an ordered procedure is followed to deal with the failure and provide information as to the extent that processing was successful.

13. The method of claim 8 wherein the second set of meta-data information comprises:

a client source name;

a type of file; and

an indication of direction of file processing as either incoming or outgoing.

14. A scalable fulfillment system for transporting and processing heterogeneous data files in a business to business environment, the scalable fulfillment system comprising:

a source for client files;

a client transport interface for extracting a client file and client meta-data from the source for client files;

a transport and control interface for retrieving the client file and client meta-data from the client transport interface, for assigning a global message id to the client file and client meta-data, enqueueing the client meta-data as a client message to a work scheduling queue, and separately storing the client file; and

a scalable processing subsystem for dequeueing client messages from the work scheduling queue, communicating with a fulfillment database, and processing the client message and file according to the client source name, file type, and message direction.

15. The scalable fulfillment system of claim 14 wherein the source for client files is from a client gateway device that interfaces with the client transport interface.

16. The scalable fulfillment system of claim 14 wherein the scalable processor subsystem comprises:

a scalable pool of processors assignable as universal processing engines (UPEs);

one or more UPE daemon processes operative on one or more processors from the scalable pool of processors to request client messages from the work scheduling queue; and

one or more UPE workers operative on one or more processors from the scalable pool of processors to process the client message according to the client source name, file type, and message direction.

17. The scalable fulfillment system of claim 14 wherein the scalable processing subsystem'"'"'s performance of processing the client message and client file according to the client source name, file type, and message direction further comprises:

a table of at least one action function identifier specified for the client source name, file type, and message direction with a link address to the specified action function process; and

an action function process for processing the client message and file.

18. A scalable processing system for processing client files in an fulfillment system, the scalable processing system comprising:

a work scheduling queue for scheduling client messages for processing where a client message comprises a client source name, file type, an indication of message direction as either incoming or outgoing, and a serial/parallel flag;

a scalable pool of processor resources;

individually selectable processor resources for processing a first set of client messages in sequential order based on the serial/parallel flag indicating serial processing; and

individually selectable processor resources for simultaneously processing a second set of client messages in parallel based on the serial/parallel flag indicating parallel processing.

19. The scalable processing system of claim 18 further comprising:

an action task table containing at least a first action task and a second action task both being required for the processing of a client message, each task identified according to expected types of client files; and

an available processor resource selected from the scalable pool of processor resources for processing a first action task first the said processing comprising updating the meta-data contained in the client message for a new message type and a time stamp; and

the available processor resource enqueueing the update client message on the work scheduling queue.

20. The scalable processing system of claim 19 wherein the processing of the second action task completes the processing for the client message and provides an indication of success or failure of said processing.

21. A computer readable medium whose contents cause a computer system to perform the processing of heterogeneous files in a fulfillment system by performing the steps of:

storing a set of action tasks in an action task table according to expected types of client files;

extracting from a client repository a client file and associated meta-data which is information concerning the client file;

storing the client file;

creating a client message containing the extracted meta-data and a link reference to the client file;

enqueueing the client message on a work scheduling queue containing a plurality of client messages;

dequeueing the client message from the work scheduling queue;

selecting at least one action task from the action task table based on selected meta-data information from the client message; and

processing the client message according to the selected action task.

22. The computer readable medium of claim 21 wherein the selected meta-data information comprises:

a client source name;

a type of file; and

an indication of direction of file processing as either incoming or outgoing.

23. The computer readable medium of claim 21 whose contents further cause the computer system to perform said processing by further performing the steps of:

selecting a second action task from the action task table; and

wherein the processing step further comprises processing the client file according to the second action task.

24. The computer readable medium of claim 21 wherein the client file is a client document.