Dual node composite image system architecture

• US 9,860,445 B2
• Filed: 06/15/2015
• Issued: 01/02/2018
• Est. Priority Date: 06/15/2015
• Status: Active Grant

• Current Assignee: Bendix Commercial Vehicle Systems LLC (Estate of Heinz Hermann Thiele)
• Original Assignee: Bendix Commercial Vehicle Systems LLC (Estate of Heinz Hermann Thiele)
• Inventors: Kuehnle, Andreas U., Gyori, Marton, Molin, Hans M., Jones, Karl H., Ramler, Travis G.
• Primary Examiner(s): Rahaman, Mohammed
• Assistant Examiner(s): Tarko, Asmamaw G
• Application Number:
  US14/739,302
• Publication Number:
  US 20160366336A1
• Time in Patent Office: 932 Days
• Field of Search: 348148
• CPC Class Codes: B60R 1/002 specially adapted for covering the peripheral part of the vehicle, e.g. for viewing tyres, bumpers or the like integrated in the windows B60R1/001; combined with rear-view mirrors B60R1/08
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

When employing a multi-node computing architecture for commercial articulated vehicles to generate a composite surround view of the vehicle, each processing node is associated with a vehicle segment. Each node has its own processor, memory and sensors. A master-slave relationship exists in the architecture. The segment processors collect, format, manage and package their local information. Communication between nodes is controlled by the master node, which can limit, adjust, subsample and alter the information and information flow from the slave nodes. The slave nodes can be freely recombined, using a pre-established communication standard or a format agreed upon at combination time.

Claims

1. A system that facilitates generating a composite image using a multi-node computer architecture for an articulated vehicle, comprising:

a first set of one or more cameras that capture images of an area surrounding a first portion of the articulated vehicle;

a first processing node that receives captured image data from the first set of one or more cameras;

a second set of one or more cameras that capture images of an area surrounding a second portion of the articulated vehicle;

a second processing node that receives captured image data from the second set of one or more cameras and transmits the captured image data to the first processing node;

wherein the first processing node is configured to generate a composite surround view image of the articulated vehicle from the captured image data received from the first and second sets of one or more cameras;

wherein the first processing node identifies one or more cameras in one or more of the first and second sets of one or more cameras is not functioning properly, and omits image data associated with the identified one or more cameras when generating the composite surround view image.

2. The system according to claim 1, wherein the first processing node is configured to generate a first surround view image by stitching together the image data from the first set of one or more cameras, and the second processing node is configured to generate a second surround view image by stitching together the image data from the second set of one or more cameras.

3. The system according to claim 2, further comprising a third processing node that is a portable computing device, which is configured to receive the first surround view image from the first processing node and the second surround view image from the second processing node, and to stitch together the first and second surround view images into a composite surround view image of the articulated vehicle and display the composite surround view image.

4. The system according to claim 3, wherein the second processing node is configured to receive the second surround view images and acts as a gateway device between the first and third processing nodes.

5. The system according to claim 2, wherein the first processing node is configured to receive the second surround view image from the second processing node and to stitch together the first and second surround view images into a composite surround view image of the articulated vehicle.

6. The system according to claim 1, wherein the first processing node is an electronic control unit (ECU) in a tractor portion of the articulated vehicle, and wherein the second processing node is an ECU in a trailer portion of the articulated vehicle.

7. The system according to claim 6, wherein the first processing node determines an articulation angle between the tractor portion and the trailer portion and accounts for the determined articulation angle when generating the composite surround view image.

8. The system according to claim 1, wherein the second processing node is further configured to receive and relay to the first processing node sensor information for one or more monitored parameters of the second portion of articulated vehicle.

9. The system according to claim 8, wherein the first processing node is further configured to detect that a data link between the first and second processing nodes is overloaded, and to instruct the second processing node to reduce data flow of at least one of the image data and the sensor information transmitted over the data link.

10. An electronic controller unit (ECU) that facilitates generating a composite image using a multi-node computer architecture for an articulated vehicle, comprising:

a processor configured to;

receive from a first set of one or more cameras captured video data of an area surrounding a first portion of the articulated vehicle;

receive from a secondary ECU video data captured by a second set of one or more cameras of an area surrounding a second portion of the articulated vehicle;

determine an angle of articulation between the first and second portions of the articulated vehicle; and

for each video frame, stitch together a composite surround view image of the articulated vehicle from the captured image data received from the first and second sets of one or more cameras while compensating for the determined articulation angle;

wherein the processor is further configured to determine that one or more cameras in one or more of the first and second sets of one or more cameras is not functioning properly, and to omit image data associated with the identified one or more cameras when generating the composite surround view image.

11. The ECU according to claim 10, wherein each frame of the video data captured by the second set of one or more cameras is stitched into a surround view of the second portion of the articulated vehicle prior to being received by the processor.

12. The ECU according to claim 11, wherein for each video frame, the processor is further configured to stitch together a surround view image of the first portion of the articulated vehicle and subsequently to stitch together the surround view images of the first and second portions of the articulated vehicle into a composite surround view image of the articulated vehicle.

13. The ECU according to claim 10, wherein the processor is further configured to receive from the secondary ECU current sensor information for one or more monitored parameters of the second portion of the articulated vehicle and to display the received sensor information to a driver with a composite surround view image.

14. The ECU according to claim 10, wherein the processor is further configured to detect that a data link between the ECU and the secondary ECU is overloaded, and to instruct the secondary ECU to reduce the data flow of at least one of the video data and the sensor information transmitted over the data link.

15. A method of generating a composite surround view image using a multi-node computer architecture for an articulated vehicle, comprising:

receiving at a first processing node captured image data of an area surrounding a first portion of the articulated vehicle from a first set of one or more cameras;

receiving at a second processing node captured image data of an area surrounding a second portion of the articulated vehicle from a second set of one or more cameras;

receiving at the first processing node the captured image data of the area surrounding the second portion of the articulated vehicle;

generating a composite image of the articulated vehicle from the captured image data received from the first and second sets of one or more cameras;

determining that one or more cameras in one or more of the first and second sets of one or more cameras is not functioning properly;

omitting image data associated with the identified one or more cameras when generating the composite surround view image.

16. The method according to claim 15, further comprising generating a first surround view image by stitching together the image data from the first set of one or more cameras, and generating a second surround view image by stitching together the image data from the second set of one or more cameras.

17. The method according to claim 16, further comprising receiving at a third processing node the first surround view image from the first processing node and the second surround view image from the second processing node, stitching together the first and second surround view images into a composite surround view image of the articulated vehicle, and displaying the composite surround view image on a display.

18. The method according to claim 17, wherein the second processing node is an gateway device and configured to receive the first and second surround view images and acts as a gateway device between the third processing node and the first processing nodes.

19. The method according to claim 16, wherein the first processing node is configured to receive the second surround view image from the second processing node and to stitch together the first and second surround view images into a composite surround view image of the articulated vehicle.

20. The method according to claim 15, wherein the first processing node is an electronic control unit (ECU) in a tractor portion of the articulated vehicle, and wherein the second processing node is an ECU in a trailer portion of the articulated vehicle.

21. The method according to claim 15 and wherein the first processing node determines an articulation angle between the first portion and the second portion and accounts for the determined articulation angle when generating the composite surround view image.

22. The method according to claim 15, wherein the second processing node is further configured to receive and relay to the first processing node sensor information for one or more monitored parameters of the second portion, and wherein the first processing node is further configured to detect that a data link between the first and second processing nodes is overloaded, and to instruct the second processing node to subsample at least one of the image data and the sensor information transmitted over the data link.

23. An apparatus that facilitates generating a composite image using a multi-node computer architecture for an articulated vehicle, comprising:

first receiving means for receiving at a first processing node captured image data of an area surrounding a first portion of the articulated vehicle from a first set of one or more cameras;

second receiving means for receiving at a second processing node captured image data of an area surrounding a second portion of the articulated vehicle from a second set of one or more cameras;

wherein the first receiving means is further configured to receive from the second processing node the captured image data of the area surrounding the second portion of the articulated vehicle; and

processing means for determining an articulation angle between the first and second portions of the articulated vehicle and compensating for the determined articulation angle when generating a composite image of the articulated vehicle from the captured image data received from the first and second sets of one or more cameras;

wherein the processing means is further configured to determine that one or more cameras in one or more of the first and second sets of one or more cameras is not functioning properly, and to omit image data associated with the identified one or more cameras when generating the composite surround view image.