Systems and methods for monitoring and tracking movement and location of shipping containers and vehicles using a vision based system

• US 7,508,956 B2
• Filed: 06/04/2004
• Issued: 03/24/2009
• Est. Priority Date: 06/04/2003
• Status: Active Grant

• Current Assignee: ABB Incorporated (ABB Limited)
• Original Assignee: APS Technology Group Incorporated (ABB Limited)
• Inventors: Scheppmann, Leroy E.
• Primary Examiner(s): Johns; Andrew W
• Application Number:
  US10/861,678
• Publication Number:
  US 20050027435A1
• Time in Patent Office: 1,754 Days
• Field of Search: 382/100, 382/103, 382/104, 348/143, 348/148, 348/149, 340/933, 340/988, 414/139.9, 414/143.2
• US Class Current: 382/103
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

The invention provides for systems and methods for location-identification and monitoring of objects, such as shipping containers, at a transit terminal using an automated vision based tracking system. Such a system is ideally suited to provide minimal error rates, low cost implementation, real-time data and automatic merging of data with corresponding object identification systems and terminal operating systems. The systems and methods of the present invention provide for location-tracking of objects without the need to retrofit the objects with a tagging or marker device. The vision-based system incorporates the use of multiple imaging devices and a hand-off method for tracking the location of the objects over extended distances.

Claims

1. A system for monitoring the movement of objects in a transit terminal, the system comprising:

a plurality of imaging devices positioned about the transit terminal such that positioning of the imaging devices provides for a continuous field of view for an object that moves throughout any area of the transit terminal; and

one or more object monitoring and tracking hosts in communication with one or more of the plurality of imaging devices, at least one of the hosts having a processor configured to execute a vision-based scene analysis routine to monitor, in real-time, the movement of objects in the transit terminal.

2. The system of claim 1, wherein the transit terminal is one or more of an automotive transit terminal, a rail transit terminal, a marine transit terminal, and an intermodal transit terminal.

3. The system of claim 1, further comprising a central tracking server in communication with the one or more object monitoring and tracking hosts, wherein the server includes a processor configured to execute a correlation routine to track objects as they move from the field of view of one or more of the plurality of imaging devices controlled by a first host to the field of view of one or more of the plurality of imaging devices controlled by a second host.

4. The system of claim 3, further comprising an Optical Character Recognition (OCR) identifying system in communication with the central tracking server configured to provide object identifying information to the correlation routine.

5. The system of claim 4, further comprising a network interface configured to provide for communication between the OCR identifying system and the central tracking server.

6. The system of claim 5, wherein the network interface is configured to provide for communication between the central tracking server and a transit Terminal Operating System(TOS).

7. The system of claim 4, wherein at least one of the OCR identifying system and the central tracking server is configured to communicate with a Terminal Operating System (TOS) configured to manage the transit terminal based on the object identifying information provided by the OCR identifying system and object tracking information provided by the central tracking server.

8. The system of claim 3, further comprising a timing mechanism in communication with the central tracking server configured to provide clock functions to the correlation routine.

9. The system of claim 3, further comprising a memory unit configured to store one or more transit terminal maps that are used in conjunction with the correlation routine to determine object location and track movement.

10. The system of claim 1, further comprising an Optical Character Recognition (OCR) identifying system in communication with the one or more hosts, wherein the OCR identifying system is configured to identify the objects upon entrance to the transit terminal and is further configured to provide object identifying information to the scene analysis routine.

11. The system of claim 10, further comprising a network interface configured to provide for communication between the OCR identifying system and the one or more hosts.

12. The system of claim 11, wherein the network interface is configured to provide for communication between the one or more hosts and a transit Terminal Operating System (TOS).

13. The system of claim 1, wherein the objects are one or more of a shipping container, a chassis, and a vehicle.

14. The system of claim 1 further comprising a timing mechanism in communication with the one or more object monitoring and tracking hosts configured to provide clock functions to the scene analysis routine.

15. The system of claim 1, further comprising a memory unit configured to store one or more transit terminal maps that are used in conjunction with the scene analysis routine to determine object location and track movement.

16. A method for identifying and tracking the movement of objects at a transit terminal, the method comprising the steps of:

identifying a transit terminal-related movable object proximate an entry point of he transit terminal;

communicating object identification information to a Terminal Operating System (TOS);

providing object image data to a vision-based tracking system;

implementing a scene analysis routine within the vision-based tracking system; and

communicating scene analysis routine data to the TOS for managing the object at the transit terminal.

17. The method of claim 16, wherein the step of identifying an object proximate an entry point of the transit terminal further comprises using Optical Character Recognition (OCR) to identify the object.

18. The method of claim 16, wherein the step of identifying a terminal-related movable object proximate an entry point of the transit terminal further defines the object as one of a shipping container, a vehicle, and a chassis.

19. The method of claim 16, wherein the step of providing object image data to the vision-based tracking system further comprises capturing, in real-time, images of the object and communicating the images to the vision-based tracking system.

20. The method of claim 16, further comprising the step of identifying a form of the object upon providing the object image data to the vision-based tracking system.

21. The method of claim 20, wherein the step of identifying a form of the object upon providing the object movement data to the vision-based tracking system further defines the form of the object as chosen from the group consisting of a container, a chassis and a vehicle.

22. The method of claim 16, further comprising the step of correlating object image data with vision-based tracking system time.

23. The method of claim 22, wherein the step of correlating object image data with vision-based tracking system time is performed to provide an estimate of the object'"'"'s arrival time at a specified location.

24. The method of claim 16, further comprising the step of identifying an action of the object upon providing the object image data to the vision-based tracking system.

25. The method of claim 24, wherein the step of identifying an action of the object upon providing the object image data to the vision-based tracking system further defines the action of the object as chosen from the group consisting of hooking/unhooking a chassis from a vehicle, removing or adding a container from/to a chassis, and moving a container about the transit terminal.

26. A method for monitoring vehicles at a staging area in a transit terminal, the method comprising the steps of:

monitoring at a transit terminal, with one or more imaging devices, one or more vehicle pedestals and a queuing area associated with the pedestals;

providing image data from the pedestals and the queuing area to a vision-based monitoring system; and

managing the vehicle pedestals and the queuing area based on the image data provided to the vision-based monitoring system.

27. The method of claim 26, wherein the step of managing the vehicle pedestals and the queuing area based on the image data provided to the vision-based monitoring system further comprises redirecting vehicles in the queuing area based on the image data provided to the vision-based monitoring system.

28. The method of claim 26, wherein the step of managing the vehicle pedestals and the queuing area based on the image data provided to the vision-based monitoring system further comprises assessing the number of vehicles at the pedestals and in the queuing area based on the image data provided to the vision-based monitoring system.

29. The method of claim 26, wherein the step of managing the vehicle pedestals and the queuing area based on the image data provided to the vision-based monitoring system further comprises characterizing the vehicles at the pedestals and in the queuing area based on the image data provided to the vision-based monitoring system.

30. The method of claim 26, wherein the step of managing the vehicle pedestals and the queuing area based on the image data provided to the vision-based monitoring system further comprises measuring the speed of travel of the vehicles based on the image data provided to the vision-based monitoring system.

31. The method of claim 26, wherein the step of managing the vehicle pedestals and the queuing area based on the image data provided to the vision-based monitoring system further comprises detecting direction of movement of a vehicle based on the image data provided to the vision-based monitoring system.

32. A system for monitoring vehicles at a staging area in a transit terminal, the system comprising:

a one or more imaging devices positioned about the staging area in a transit terminal such that the imaging devices provide for a continuous field of view for an object that moves through a queuing area and a pedestal area associated with the staging area; and

one or more object monitoring and tracking hosts in communication with the one or more imaging devices having a processor that executes a vision-based scene analysis routine to monitor, in real-time, objects in the staging area.

33. The system of claim 32, wherein the transit terminal is one or more of a truck transit terminal, a rail transit terminal, a marine transit terminal, and an intermodal transit terminal.

34. The system of claim 33, further comprising a central tracking server in communication with the one or more object monitoring and tracking hosts, wherein the server includes a processor configured to execute a correlation routine to track objects as they move from the field of view of one or more of the plurality of imaging devices controlled by a first host to the field of view of one or more of the plurality of imaging devices controlled by a second host.

35. The system of claim 34, further comprising an Optical Character Recognition (OCR) indentifying system in communication with the central tracking server configured to provide object identifying information to the correlation routine.

36. The system of claim 32, further comprising an Optical Character Recognition (OCR) identifying system in communication with the one or more hosts configured to identify the vehicles upon entrance to an OCR portal.

37. The system of claim 32, further comprising a Terminal Operating System (TOS) in communication with the one or more object monitoring and tracking hosts, wherein data from the vision-based scene analysis routine is communicated to the TOS.

38. A method for real-time monitoring of storage of a transit container at a transit terminal, the method comprising the steps of:

providing for one or more imaging devices to monitor a transit container storage area;

providing real-time image data from the imaging devices to a vision-based monitoring system; and

managing the containers in the storage area based on the image data provided to the vision-based monitoring system.

39. The method of claim 38, wherein the step of providing for one or more imaging devices to monitor a transit container storage area further comprises providing for one or more imaging devices proximate a shipping vessel that monitor a storage area within the shipping vessel.

40. The method of claim 39, wherein the step of providing for one or more imaging devices proximate a shipping vessel that monitor a storage area within the shipping vessel further comprises providing for one or more imaging devices affixed to a quay crane that is positioned proximate a shipping vessel that monitor a storage area within the shipping vessel.

41. The method of claim 38, wherein the step of managing the containers in the storage area based on the image data provided to the vision-based monitoring system further comprises managing change in storage locations for the containers.

42. A system for real-time monitoring of the storage of a transit container at a transit terminal, the system comprising:

one or more imaging devices positioned proximate a container storage area, such that the imaging devices provide for a continuous field of view for one or more containers stored in the storage area; and

one or more object monitoring and tracking hosts in communication with the one or more imaging devices having a processor configured to execute a vision-based scene analysis routine to monitor, in real-time, containers in the storage area.

43. The system of claim 42, wherein the one or more imaging devices are located on a quay crane that is positioned proximate a container storage area located on a shipping vessel.

44. The system of claim 42, further comprising a central tracking server in communication with the one or more object monitoring and tracking hosts, wherein the server includes a processor configured to execute a correlation routine to track objects as they move from the field of view of one or more of the plurality of imaging devices controlled by a first host to the field of view of one or more of the plurality of imaging devices controlled by a second host.

45. The system of claim 44, further comprising a system for triggering optical character recognition imaging that is configured to communicate with the central tracking server, wherein the system for triggering optical character recognition imaging comprises:

one or more triggering imaging devices configured to capture images of objects proximate an Optical Character Recognition (OCR) station;

a triggering host in communication with the one or more triggering imaging devices, wherein the host includes a processor configured to execute a triggering routine that determines if the object is in a position for OCR;

an OCR system in communication with the triggering host, wherein the OCR system is configured to receive inputs from the triggering routine to commence the capture of images; and

one or more OCR cameras in communication with the OCR system, wherein the one or more OCR cameras are configured to capture images of the optical characters based on commands from the OCR system.

46. The system of claim 45, wherein the triggering routine is further configured to determine the direction of travel of the object.

47. The system of claim 45, wherein the triggering routine is further configured to determine the size of the object.

48. The system of claim 45, wherein the triggering routine is further configured to determine the form of the object.

49. The system of claim 42, further comprising a Terminal Operating System (TOS) in communication with the one or more object monitoring and tracking hosts, wherein data from the vision-based scene analysis routine is communicated to the TOS.

50. The system of claim 49, further comprising an Optical Character Recognition (OCR) identifying system in communication with the TOS configured to provide container identifying information to the TOS for identifying a location position of a container within the storage area.

51. A method for real-time monitoring of an area proximate a landing area associated with a quay crane the method comprising the step of:

providing for one or more imaging devices to monitor a landing area associated with a quay crane;

providing real-time image data from the imaging devices to a vision-based monitoring system; and

managing the landing area associated with the quay crane based on the image data provided to the vision-based monitoring system.

52. The method of claim 51, wherein the step of managing the landing area associated with the quay crane based on the image data provided to the vision-based monitoring system further comprises providing an alarm if hazardous conditions exist in the landing area.

53. A system for real-time monitoring of an area proximate a landing area associated with a quay crane, the system comprising:

one or more imaging devices positioned proximate a landing area associated with a quay crane, such that the imaging devices provide for a continuous field of view of the landing area; and

one or more object monitoring and tracking hosts in communication with the one or more imaging devices having a processor that configured to execute a vision-based scene analysis routine to monitor, in real-time, the landing area.

54. The system of claim 53, further comprising a Terminal Operating System (TOS) in communication with the one or more object monitoring and tracking hosts, wherein data from the vision-based scene analysis routine is communicated to the TOS.

55. The system of claim 53, further comprising an alarm configured to be triggered if hazardous conditions are determined to exist in the landing area based on image data provided to the vision-based scene analysis routine.