Intelligent routing method for routing items through a network

• US 10,237,167 B2
• Filed: 01/26/2016
• Issued: 03/19/2019
• Est. Priority Date: 02/08/2015
• Status: Active Grant

• Current Assignee: Bobsweep, Inc.
• Original Assignee: Ai Incorprated
• Inventors: Ebrahimi Afrouzi, Ali
• Primary Examiner(s): Truong, Lan-Huong
• Application Number:
  US15/006,434
• Publication Number:
  US 20160234098A1
• Time in Patent Office: 1,148 Days
• CPC Class Codes: H04L 43/0858 One way delays
• CPC Class Codes: H04L 45/121 by minimising delays

Abstract

A method for selecting routes for items being sent from a given beginning node to a given destination node within a network. Each segment of a trip is evaluated and assigned a negative reward based on the deviation from the ideal length of time to complete that segment. Upon arrival of the item at the destination node, the route is assigned a large positive reward. The system is configured to attempt to maximize rewards at all times by selecting the routes with the lowest likelihood of delay. As more and more trips are completed, the control system may define a policy that selects routes based on their probability of yielding the highest reward.

Claims

1. A method for determining a route from a beginning node to a destination node in a distribution network, the method comprising:

providing a plurality of nodes connected by a plurality of links that provide a plurality of routes from the beginning node to the destination node, wherein

each of the plurality of nodes represents a transfer point between links, and wherein

each link represents a segment of the plurality of routes;

providing an item for traversing the distribution network;

selecting one of the plurality of routes;

traversing the selected route by the item;

evaluating an efficiency of each segment of the selected route as traversed by the item;

calculating a reward based on the item traversing the selected route, wherein calculating the reward based on the item traversing the selected route comprises;

assigning a negative reward for each segment of the selected route based on any delay on the segment; and

assigning a positive reward for each segment of the selected route based on completion of the segment; and

summing the positive and negative rewards for all segments of the selected route;

tabulating the reward;

iteratively traversing the distribution network by other items along others of the plurality of routes and tabulating the rewards; and

selecting a preferred route corresponding with a highest reward based on tabulated rewards for future items to traverse the distribution network.

2. The method of claim 1, wherein the distribution network is selected from the group consisting of:

a pneumatic tube network, a rail network, and a conveyor belt network.

3. The method of claim 1, wherein the assigning a negative reward is calculated by the formula:

[W(estimated)−

W(actual)]R=reward, where

W (estimated) is a first amount of time to complete the segment with no delays;

W(actual) is a second amount of time to actually complete the segment; and

R is a predetermined reward rate.

4. The method of claim 1, further comprising:

assigning a value to the selected route;

calculating a policy corresponding with values of all selected routes; and

selecting the policy to maximize the reward.

5. The method of claim 4, wherein the policy defines a best route discovered represented by the formula:

π

(s)=argmax_s′

Σ

,P(s′

|s₁a)∨

(s′

)

where

π

is the policy;

P is a state transition function;

s (state) is a possible trip segment of a route; and

a is the action of selecting a route.