Load tracking utilizing load identifying indicia and spatial discrimination

• US 8,561,897 B2
• Filed: 11/17/2011
• Issued: 10/22/2013
• Est. Priority Date: 11/18/2010
• Status: Active Grant

• Current Assignee: TotalTrax, Inc.
• Original Assignee: Sky-Trax Incorporated (TotalTrax, Inc.)
• Inventors: Kunzig, Robert S., Taylor, Robert M., Maxwell, Leonard J.
• Primary Examiner(s): FRANKLIN, JAMARA ALZAIDA
• Application Number:
  US13/298,713
• Publication Number:
  US 20120126000A1
• Time in Patent Office: 705 Days
• Field of Search: 235/385
• US Class Current: 235/385
• CPC Class Codes: B66F 9/0755 Position control; Position detectors
• CPC Class Codes: B66F 9/24 Electrical devices or systems
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 10/0875 Itemisation or classification of parts, supplies or services, e.g. bill of materials
• CPC Class Codes: G06Q 50/00 Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism healthcare informatics G16H
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Methods and apparatus for determining the location of one or more unit loads in a coordinate space in a facility, comprising an integrated system that identifies a load by reading indicia on the load, determines the position of the indicia in the coordinate space, and stores the indicia position and load identity in a Label Map. A mobile subsystem on each conveying vehicle identifies the location and orientation of that vehicle using a position/orientation sensor, determines the identity of the load, establishes a Target Cube to discriminate the desired load from nearby loads, confirms acquisition of the load, and communicates the information to a fixed-base subsystem. Load identity and the location and orientation of the vehicle when a load is deposited are used to create a Load Map that contains the identity, position and orientation of all identified loads in the coordinate space.

Claims

1. A method for identifying and locating assets with one or more optical label readers, each label reader being mounted on a corresponding conveying vehicle at a known position and orientation relative to a center and a directional axis of the vehicle, each conveying vehicle having an optical image acquisition system mounted thereon for sensing position and rotational orientation within a facility having an array of position markers, each asset having a machine-readable encoded indicia of a predetermined size thereon, the method comprising the steps of:

a) repeatedly determining the position of the center of the vehicle and the rotational orientation of the directional axis of the vehicle;

b) acquiring images with the one or more label readers as each conveying vehicle moves about the facility;

c) processing each image to determine if that image comprises a label having readable indicia;

d) if the indicia is not readable, repeating steps b) and c);

e) if the indicia is readable;

1) decoding the indicia to identify the asset;

2) determining a center of the indicia on the label; and

3) determining a location of the label center relative to the label reader based upon the position and the size of the indicia within the image;

f) determining the position of the label within the facility using the location of the indicia relative to the label reader, the position and rotational orientation of the label reader relative to the center of the vehicle and the directional axis of the vehicle and the position of the center of the vehicle and directional axis of the vehicle within the facility; and

g) storing the identity and the position of each readable label within the facility in a database, called a Label Map, in a memory in a computer.

2. The method of claim 1, further comprising:

h) if a label identity is already in the Label Map, averaging the newly determined position with the position(s) already in the Label Map and storing the averaged position in the Label Map.

3. The method of claim 1, the conveying vehicle comprising a load handling mechanism having a load datum and a directional axis and a load detecting device, the method further comprising the steps of:

h) defining parameters for a Targeting Lane and a Target Cube along the directional axis of the load handling mechanism in reference to the load datum of the conveying vehicle and relative to the center point of the vehicle;

1) a Targeting Lane width to the left of the load handling directional axis;

2) a Targeting Lane width to the right of the load handling directional axis;

3) a Targeting Lane height from the load datum on the load handling mechanism;

4) a Targeting Lane depth along the load handling directional axis;

5) a Target Cube depth along the load handling directional axis;

i) projecting the Targeting Lane and the Target Cube into a virtual space which has been defined within the Label Map;

j) defining the Targeting Lane in front of the load datum of the vehicle within the virtual space based on the current location of the load datum within the facility;

k) searching the Label Map for assets lying within the Targeting Lane;

l) determining from the Label Map if any assets are in the Targeting Lane;

m) if no assets are in the Targeting Lane, repeating steps i)-l);

n) if any assets are in the Targeting Lane;

1) determining the label of the closest asset in the Targeting Lane;

2) establishing a Target Cube depth from the position of the closest asset'"'"'s label;

o) determining from the Label Map if any additional labels are within the Target Cube;

p) if any additional labels are in the Target Cube, adding the assets with the additional labels to the potential loads, otherwise go to step q);

q) determining if the load detecting device has detected a “

Load ON”

event;

r) if a “

Load ON”

event has not occurred, repeating steps i)-q);

s) if a “

Load ON”

event has occurred, then determining labels from the Label Map and the associated assets to be part of a “

Load On Board”

.

4. The method of claim 1, the conveying vehicle comprising a load handling mechanism having a load detecting device on a moveable portion thereof for detecting an asset at a predetermined close distance, further determining the position and directional orientation of each asset, comprising the steps of:

h) approaching an asset in the facility with a conveying vehicle until the load detecting device detects the asset;

i) acquiring the asset with the load handling mechanism on the conveying vehicle, thus establishing a position of the asset on the load handling mechanism;

j) transporting the asset and depositing the asset at a destination location;

k) using the position of the asset on the load handling mechanism, the relative position of the load handling mechanism to the conveying vehicle, and the position of the center of the vehicle and orientation of the directional axis of the vehicle, determining the position and the orientation of the asset within the facility when the asset is deposited;

l) storing the identity, the position and directional orientation of each asset within the facility in a database, called a Load Map, in a memory in the computer.

5. The method of claim 4, wherein, prior to step h), average sizes of assets in the facility are defined.

6. The method of claim 4, the load handling mechanism having a load datum and a directional axis, further comprising a method for locating an asset in a facility using the Load Map, comprising the steps of:

m) defining parameters for a Targeting Lane and a Target Cube along the directional axis of the load handling mechanism in reference to the load datum of the conveying vehicle and relative to the center point of the vehicle, the Targeting Lane and the Target Cube being projected into a virtual space which has been defined within the Load Map by;

1) a Targeting Lane width to the left of the load handling directional axis;

2) a Targeting Lane width to the right of the load handling directional axis;

3) a Targeting Lane height from the load datum on the load handling mechanism;

4) a Targeting Lane depth along the load handling directional axis;

5) a Target Cube depth along the load handling directional axis;

n) defining the Targeting Lane in front of the load datum of the vehicle within the virtual space based on the current location of the load datum within the facility;

o) searching the Load Map for assets lying within the Targeting Lane;

p) determining from the Load Map if any assets are in the Targeting Lane;

q) if no assets are in the Targeting Lane, repeating steps n)-p);

r) if any assets are in the Targeting Lane;

1) determining the closest asset in the Targeting Lane;

2) establishing a Target Cube depth from closest asset'"'"'s position;

s) determining from the Load Map if any additional assets are within the Target Cube;

t) if any additional assets are in the Target Cube, adding the assets to the potential loads, otherwise go to step u);

u) determining if the load detecting device has detected a “

Load ON”

event;

v) if a “

Load ON”

event has not occurred, repeating steps n)-u);

w) if a “

Load ON”

event has occurred, then determining the potential loads from the Load Map to be part of a “

Load On Board”

.

7. The method of claim 4, further comprising:

m) transmitting the identity, the position and the orientation of the deposited asset to a system controller over a wireless network; and

n) storing the identity, the position and the orientation of the deposited asset in a database, called a Global Load Map in a memory in a computer.

8. The method of claim 4, the load handling mechanism having a load datum and a directional axis, further comprising a method for identifying an asset that has no readable label in a facility using the Load Map, comprising the steps of:

m) defining parameters for a Targeting Lane and a Target Cube along the directional axis of the load handling mechanism in reference to the load datum of the conveying vehicle and relative to the center point of the vehicle the Targeting Lane and the Target Cube being projected into a virtual space which has been defined by the Load Map by;

1) a Targeting Lane width to the left of the load handling directional axis;

2) a Targeting Lane width to the right of the load handling directional axis;

3) a Targeting Lane height from the load datum on the load handling mechanism;

4) a Targeting Lane depth along the load handling directional axis;

5) a Target Cube depth along the load handling directional axis;

n) defining the Targeting Lane in front of the load datum of the vehicle within the virtual space based on the current location of the load datum within the facility;

o) determining from the Load Map if any assets are in the Targeting Lane;

p) if no assets are in the Targeting Lane, repeating steps n)-o); and

q) if any assets are in the Targeting Lane, determining the closest item in the Targeting Lane, thereby identifying the asset that has no readable label.

9. The method of claim 1, the conveying vehicle having a load detecting device on a moveable portion of a load handling mechanism for detecting an asset at a predetermined close distance, further determining the position and directional orientation of each asset, comprising the steps of:

h) approaching the asset identified in step e)

     1) with the conveying vehicle until the load detecting device detects the asset;

i) acquiring the asset with the load handling mechanism on the conveying vehicle, thus establishing a position of the asset on the load handling mechanism;

j) using the position of the asset on the load handling mechanism, the position of the center of the vehicle and the orientation of the directional axis of the vehicle, determining the position and the orientation of the asset within the facility;

k) storing the identity, the position and directional orientation of the asset within the facility in a database, called a Local Load Map, in a memory in a computer.

10. The method of claim 9 wherein, prior to step h), average sizes of assets in the facility are defined.

11. A method of creating a Global Label Map database, for identifying and locating assets within a facility having an array of position markers, each asset having a machine-readable coded indicia of a predetermined size thereon, in a system having:

i) a central system controller comprising a computer having a memory and a wireless communications unit;

ii) a mobile subsystem on a conveying vehicle, the subsystem comprising;

1) a mobile computer having a memory;

2) a wireless communications unit for communicating over a wireless network with the central system controller;

3) a position/orientation sensor device mounted on the conveying vehicle at a known position and orientation relative to a center of the vehicle and a directional axis of the vehicle; and

4) one or more label readers each mounted on the conveying vehicle at a known position and orientation relative to a center of the vehicle and a directional axis of the vehicle;

the method comprising the steps of;

a) repeatedly determining the position of the center of the vehicle and the orientation of the directional axis of the vehicle with the position/orientation sensor device;

b) acquiring images with the label readers as the conveying vehicle moves about the facility;

c) processing each image to determine if that image contains a label having readable indicia;

d) if the indicia is not readable, repeating steps b) and c);

e) if the indicia is readable;

1) decoding the indicia to identify the asset;

2) determining the center of the indicia on the label; and

3) determining the location of the label center relative to the label reader based upon the position and the size of the indicia within the image;

f) determining the position of the label within the facility using the location of the indicia relative to the label reader and the position of the center of the vehicle and the orientation of the directional axis of the vehicle;

g) storing the identity and the position of each readable label in a database, called a Local Label Map, in the memory in the mobile computer;

h) transmitting the identity and the position of each label in the Local Label Map in each mobile computer to the central system controller over the wireless network; and

i) compiling the identity and the position of each label in all the Local Label Maps from each mobile computer in a database, called a Global Label Map, in the memory of the central computer.

12. The method of claim 11, further comprising:

j) if a label is already in the Global Label Map, averaging the newly determined position with the positions already in the Global Label Map and storing the averaged position in the Global Label Map.

13. The method of claim 11, further comprising:

j) transmitting the identity and the position of each label in the Global Label Map to each mobile computer over the wireless network to create a copy of the Global Label Map.

14. The method of claim 13, further comprising a method of transporting an asset in a facility from a first location to a second location upon receiving the identification of the asset to be transported and the destination for the asset, comprising the steps of:

k) determining the position of the asset from a copy of the Global Label Map in the memory of the mobile computer;

l) defining a Targeting Lane in front of the conveying vehicle and determining if any labels are in the Targeting Lane;

m) if no labels are in the Targeting Lane, repeating steps k) and l);

n) if any labels are in the Targeting Lane;

1) determining the closest label in the Targeting Lane;

2) projecting a Target Cube depth from the closest label position;

o) checking label positions in the copy of the Global Label Map to determine if any labels are within the Target Cube;

p) if any labels are in the Target Cube, verifying the identity of the label of the asset to be transported;

q) determining if the load detecting device has detected a “

Load ON”

event;

r) if a “

Load ON”

event has not occurred, repeating steps n)-q);

s) if a “

Load ON”

event has occurred, then determining the label(s) and the associated asset(s) to be part of a “

Load On Board”

;

t) acquiring the asset(s);

u) clearing the record(s) for the label(s) for the associated asset(s) in the Local Label Map;

v) clearing the record(s) for the asset(s) in a Local Load Map in the memory of the mobile computer;

w) transporting the asset(s) to a second destination location;

x) depositing the asset(s) at the second location;

y) determining the position and the orientation of the asset(s) within the facility when the load detecting device detects the asset(s) has been deposited at “

Load Off”

event by using the location of the asset(s) relative to the conveying vehicle and the position of the center of the vehicle and orientation of the directional axis of the vehicle; and

z) storing the identity, the position and the orientation of the deposited asset(s) in the Local Load Map in the memory in the mobile computer.

15. The method of claim 14, further comprising:

aa) transmitting the identity, the position and the orientation of the deposited asset to the system controller over the wireless network;

ab) storing the identity, the position and the orientation of the deposited asset in the Global Load Map in the memory of the central system controller;

ac) acquiring images with the one or more label readers as the conveying vehicle moves away from the deposited asset;

ad) processing each image to determine if that image comprises a label having readable indicia;

ae) if the indicia is not readable, repeating steps ac) and ad);

af) if the indicia is readable;

1) decoding the indicia to identify the asset;

2) determining a center of the indicia on the label; and

3) determining a location of the label center relative to the label reader based upon the position and the size of the indicia within the image;

ag) determining the position of the label within the facility using the location of the indicia relative to the label reader, the position and rotational orientation of the label reader relative to the center of the vehicle and the directional axis of the vehicle and the position of the center of the vehicle and directional axis of the vehicle within the facility;

ah) storing the identity and the position of each readable label within the facility in the Local Label Map in the memory of the mobile computer;

ai) transmitting the identity, the position and the orientation of the label of the deposited asset in the Local Label Map to the system controller over the wireless network; and

aj) storing the identity, the position and the orientation of the label of the deposited asset in the Global Label Map in the memory of the central system controller.

16. The method of claim 15, wherein the mobile computer on the vehicle updates the Local Load Map and the Local Label Map by downloading data from the Global Load Map and the Global Label Map in the central system controller via the wireless network.

17. The method of claim 13, further comprising a method of transporting a first asset in a facility from a first location to a second destination location already occupied by a second asset upon receiving the identification of the first asset to be transported and the destination for the first asset, and tracking the identity and location and rotational orientation of the second asset displaced during the deposition of the first asset, comprising the steps of:

k) determining the position of the first asset from a copy of the Global Label Map in the memory of the mobile computer;

l) defining a Targeting Lane in front of the conveying vehicle and determining if any labels are in the Targeting Lane;

m) if no labels are in the Targeting Lane, repeating steps k) and l);

n) if any labels are in the Targeting Lane;

1) determining the closest label in Targeting Lane;

2) projecting a Target Cube depth from the closest label position;

o) checking label positions in the copy of the Global Label Map to determine if any labels are within the Target Cube;

p) if any labels are in the Target Cube, verifying the identity of the label of the first asset to be transported;

q) determining if the load detecting device has detected a “

Load ON”

event;

r) if a “

Load ON”

event has not occurred, repeating steps n)-q);

s) if a “

Load ON”

event has occurred, then determining the label and the associated asset to be part of a “

Load On Board”

;

t) acquiring the asset;

u) clearing the record for the label for the associated asset in the Local Label Map;

v) clearing the record for the asset in a Local Load Map in the memory of the mobile computer;

w) receiving the destination location for the first asset from a central system controller and determining, from the copy of the Global Label Map in the memory of the mobile computer, that a second asset already occupies the destination location;

x) transporting the first asset to the second destination location;

y) contacting the second asset with the first asset and determining the location and direction of travel of the conveying vehicle at contact;

z) pushing the second asset to a displaced location when the first asset is being deposited in the second destination storage location;

aa) depositing the first asset at the second location;

ab) determining the position and the orientation of the assets within the facility when the load detecting device detects that the first asset has been deposited by using the location of the first asset relative to the conveying vehicle and the position of the center of the vehicle and orientation of the directional axis of the vehicle;

ac) upon deposition of the first asset, when a “

Load Off”

event occurs, updating the locations and rotational orientations of the second asset and the first asset in the Local Label Map and Local Load Map in the memory in the mobile computer, the distance of displacement of the second asset being determined by the difference in position of the conveying vehicle at the contact of the first asset with the second asset and the position of the conveying vehicle at occurrence of the “

Load Off”

event and the direction of the displacement being determined by the direction of travel of the conveying vehicle at contact;

ad) storing the identity, the position and the orientation of the deposited asset and the displaced asset in the Local Load Map in the memory in the mobile computer;

ae) transmitting the identity, the position and the orientation of the deposited first asset and the displaced second asset to the central system controller over the wireless network; and

af) storing the identity, the position and the orientation of the deposited first asset and the displaced second asset in the Global Load Map database.

18. The method of claim 17, wherein in step y), contact between the second asset and the first asset is determined from the copy of the Global Label Map and the Targeting Lane and Target Cube.

19. A method for identifying and locating assets with one or more optical label readers, each label reader being mounted on a corresponding conveying vehicle at a known position and orientation relative to a center and a directional axis of the vehicle, each conveying vehicle having an optical image acquisition system mounted thereon for sensing position and rotational orientation within a facility having an array of position markers, each asset having a machine-readable encoded indicia of a predetermined size in a predetermined position thereon, the method comprising the steps of:

a) identifying each asset in the facility;

b) creating a configuration table containing parameters necessary to identify the asset and to locate the asset relative to the conveying vehicle;

c) determining the current conveying vehicle position and orientation;

d) acquiring images with a label reader on the conveying vehicle;

e) determining if each image contains a readable label;

f) if the label is not readable, repeating steps d) and e);

g) if the label is readable, decoding the identity of the label and calculating the three-dimensional coordinates of the label relative to the conveying vehicle;

h) using the current conveying vehicle position and orientation, transforming the coordinates of the label to facility coordinates;

i) creating a Label Map database and adding the identity and position of each decoded label to the Label Map;

j) if the identified label is already in the Label Map, adding that label position to the Label Map and calculating an average position of that label based upon all stored positions of that label.