Vision system with deterministic low-latency communication

• US 7,760,238 B2
• Filed: 07/25/2007
• Issued: 07/20/2010
• Est. Priority Date: 07/25/2007
• Status: Active Grant

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Giesen, Robert J. B.
• Primary Examiner(s): Whipkey; Jason
• Application Number:
  US11/782,758
• Publication Number:
  US 20090027509A1
• Time in Patent Office: 1,091 Days
• Field of Search: 348/211.3, 348/211.11, 348/143
• US Class Current: 348/211.3
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Vision system and method. The system includes a vision system, including: a camera, a processor and memory, coupled to the camera, and an EtherCAT interface, coupled to the processor and memory, possibly contained in a chassis, and operable to couple to an EtherCAT network that provides for deterministic low-latency communication between the vision system and devices coupled to the EtherCAT network. The vision system may be configurable to operate as any of: an EtherCAT master device, an EtherCAT slave device, or a TCP/IP Ethernet device. One or more additional vision systems may couple to the EtherCAT network. The vision systems may be synchronized using an EtherCAT distributed clock technique, time-based triggered, and/or event triggered, via the EtherCAT network, e.g., in a multi-view vision, collaborative processing, control, I/O, or video servoing application, e.g., using an EtherCAT deterministic low latency closed loop feedback system, including processing, control, or I/O devices.

Claims

1. A system, comprising:

a vision system, comprising;

a camera;

a processor and memory, coupled to the camera;

a deterministic low-latency Ethernet-based network interface, coupled to the processor and memory, and operable to couple to a deterministic low-latency Ethernet-based network; and

at least one deterministic low-latency Ethernet-based motion control device, coupled to the vision system via the deterministic low-latency Ethernet-based network;

wherein the deterministic low-latency Ethernet-based network provides for deterministic low-latency communication between the vision system and devices coupled to the deterministic low-latency Ethernet-based network;

wherein the vision system is operable to;

acquire images;

determine motion control data based on information in the acquired images;

deterministically send the motion control data to the at least one deterministic low-latency Ethernet-based motion control device via the deterministic low-latency Ethernet-based network to control the at least one deterministic low-latency Ethernet-based motion control device to perform a portion of a motion control task; and

repeat said acquiring, said determining, and said sending in an iterative manner to perform successive portions of the motion control task based on the information, thereby performing a visual servoing control loop with a frequency of 1 KHz or higher.

2. The system of claim 1, wherein the vision system is a deterministic low-latency Ethernet-based slave device.

3. The system of claim 1, wherein the vision system is a deterministic low-latency Ethernet-based master device.

4. The system of claim 1, wherein the vision system is configurable to operate as any of:

a deterministic low-latency Ethernet-based master device;

a deterministic low-latency Ethernet-based slave device;

or

a TCP/IP Ethernet device.

5. The system of claim 1, further comprising:

one or more additional vision systems, coupled to the deterministic low-latency Ethernet-based network;

wherein the vision system and the one or more additional vision systems compose a plurality of vision systems, and wherein the plurality of vision systems are synchronized using a deterministic low-latency Ethernet-based distributed clock technique.

6. The system of claim 5, wherein the plurality of vision systems are operable to be time-based triggered via the deterministic low-latency Ethernet-based network.

7. The system of claim 6, further comprising:

wherein the vision system comprises a deterministic low-latency Ethernet-based master device, and each of the one or more additional vision systems comprises a respective deterministic low-latency Ethernet-based slave device, and wherein the plurality of vision systems implement a multi-view vision application with respect to a target object, wherein each of the one or more additional vision systems is operable to send vision data to the deterministic low-latency Ethernet-based master device via the deterministic low-latency Ethernet-based network.

8. The system of claim 6, further comprising:

wherein the vision system comprises a deterministic low-latency Ethernet-based master device, and each of the one or more additional vision systems comprises a respective deterministic low-latency Ethernet-based slave device, and wherein the plurality of vision systems are operable to perform collaborative processing with respect to a target object, wherein each of the one or more additional vision systems is operable to send vision data to the deterministic low-latency Ethernet-based master device via the deterministic low-latency Ethernet-based network.

9. The system of claim 1, further comprising:

one or more additional vision systems, coupled to the deterministic low-latency Ethernet-based network;

wherein the vision system and the one or more additional vision systems compose a plurality of vision systems, and wherein the plurality of vision systems are operable to be event triggered via the deterministic low-latency Ethernet-based network.

10. The system of claim 9, further comprising:

wherein the vision system comprises a deterministic low-latency Ethernet-based master device, and each of the one or more additional vision systems comprises a respective deterministic low-latency Ethernet-based slave device, and wherein the plurality of vision systems implement a multi-view vision application with respect to a target object, wherein each of the one or more additional vision systems is operable to send vision data to the deterministic low-latency Ethernet-based master device via the deterministic low-latency Ethernet-based network.

11. The system of claim 9, further comprising:

wherein the vision system comprises a deterministic low-latency Ethernet-based master device, and each of the one or more additional vision systems comprises a respective deterministic low-latency Ethernet-based slave device, and wherein the plurality of vision systems is operable to perform collaborative processing with respect to a target object, wherein each of the one or more additional vision systems is operable to send vision data to the deterministic low-latency Ethernet-based master device via the deterministic low-latency Ethernet-based network.

12. The system of claim 1, further comprising:

at least one deterministic low-latency Ethernet-based device, coupled to the vision system via the deterministic low-latency Ethernet-based network;

wherein the vision system is operable to send data on the deterministic low-latency Ethernet-based network for use by the at least one deterministic low-latency Ethernet-based device to perform a control function.

13. The system of claim 12, further comprising:

a deterministic low-latency Ethernet-based master device, coupled to the deterministic low-latency Ethernet-based network; and

one or more additional vision systems, coupled to the deterministic low-latency Ethernet-based network;

wherein the vision system and the one or more additional vision systems compose a plurality of vision systems, and wherein each of the plurality of vision systems is operable to send vision data to the deterministic low-latency Ethernet-based master device, wherein the deterministic low-latency Ethernet-based master device is operable to generate control data based on the received vision data, and send the control data to the at least one deterministic low-latency Ethernet-based device to perform a control function.

14. The system of claim 1, further comprising:

at least one deterministic low-latency Ethernet-based device, coupled to the vision system via the deterministic low-latency Ethernet-based network;

wherein the vision system is operable to send data on the deterministic low-latency Ethernet-based network for use by the at least one deterministic low-latency Ethernet-based device to perform an I/O function.

15. The system of claim 1,

wherein the a vision system comprises a smart camera, comprising:

the camera;

the processor and memory;

the deterministic low-latency Ethernet-based interface; and

a chassis, containing the camera, the processor and memory, and the deterministic low-latency Ethernet-based interface.

16. The system of claim 1, wherein the deterministic low-latency Ethernet-based network comprises an Ethernet-based fieldbus network.

17. A method, comprising:

providing one or more deterministic low-latency Ethernet-based motion control systems, each comprising a deterministic low-latency Ethernet-based interface;

providing one or more deterministic low-latency Ethernet-based vision systems, each comprising a camera, a processor and memory, and a deterministic low-latency Ethernet-based interface, wherein the one or more deterministic low-latency Ethernet-based motion control systems and the one or more deterministic low-latency Ethernet-based vision systems are coupled to a deterministic low-latency Ethernet-based network, composing a deterministic low latency closed loop feedback system, wherein the latency is less than or equal to 1 millisecond;

the one or more deterministic low-latency Ethernet-based vision systems acquiring images of a target object, and generating visual information based on the images;

the one or more deterministic low-latency Ethernet-based vision systems sending the visual information to a master deterministic low-latency Ethernet-based device coupled to the deterministic low-latency Ethernet-based network;

the master deterministic low-latency Ethernet-based device computing motion control information based on the visual information, and sending the motion control information to the one or more deterministic low-latency Ethernet-based motion control systems; and

the one or more deterministic low-latency Ethernet-based motion control systems operating according to the motion control information;

wherein said one or more deterministic low-latency Ethernet-based vision systems acquiring images, said one or more deterministic low-latency Ethernet-based vision systems sending the visual information, said master deterministic low-latency Ethernet-based device sending the motion control information, and said one or more deterministic low-latency Ethernet-based motion control systems operating implement a high frequency visual servoing control loop with an operating frequency greater than or equal to 1 Khz; and

repeating said one or more deterministic low-latency Ethernet-based vision systems acquiring images, said one or more deterministic low-latency Ethernet-based vision systems sending the visual information, said master deterministic low-latency Ethernet-based device sending the motion control information, and said one or more deterministic low-latency Ethernet-based motion control systems operating in an iterative manner, to perform a visual servoing task.

18. The method of claim 17, further comprising:

synchronizing the one or more deterministic low-latency Ethernet-based vision systems using a deterministic low-latency Ethernet-based distributed clock technique.

19. The method of claim 17, wherein the deterministic low-latency Ethernet-based network comprises an Ethernet-based fieldbus network.