Synchronization controller unit for sensor node

• US 10,531,075 B1
• Filed: 09/27/2018
• Issued: 01/07/2020
• Est. Priority Date: 09/27/2018
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Dariush, Behzad, Chen, Yi-Ting, Dedeoglu, Goksel
• Primary Examiner(s): Schnurr, John R
• Application Number:
  US16/143,702
• Time in Patent Office: 467 Days
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: H04L 12/40 Bus networks
• CPC Class Codes: H04L 12/40013 Details regarding a bus controller
• CPC Class Codes: H04L 2012/40273 the transportation system being a vehicle
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0096 Synchronisation or controlling aspects
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/662 by using master/slave camera arrangements for affecting the control of camera image capture, e.g. placing the camera in a desirable condition to capture a desired image
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A vehicular synchronization system includes an image sensor of a vehicle. The image sensor captures image data for an image sensor field of view. The vehicular synchronization system also includes a light sensor having a radial view. The light sensor captures light data from around the vehicle in a full rotation. The vehicular synchronization system includes a synchronization control unit having a data receiving module, a trigger time determination module, a timer module, and a sensor trigger module. The data receiving module receives a data packet from the light sensor. The trigger time determination module determines a trigger time when a desired light sensor orientation overlaps with the image sensor field of view. The timer module sets a timer to elapse at the trigger time. The sensor trigger module controls the image sensor to capture image data at the trigger time when the timer elapses.

Claims

1. A vehicular synchronization system, comprising:

an image sensor, of a vehicle, configured to capture image data for an image sensor field of view;

a light sensor having a radial view, wherein the light sensor is configured to capture light data from around the vehicle in a full rotation; and

a synchronization control unit having a processor, the processor configured to;

receive a data packet from the light sensor, wherein the data packet includes a light sensor time stamp from a reference clock, and wherein the data packet identifies a current orientation of the light sensor;

determine a trigger time when a desired light sensor orientation overlaps with the image sensor field of view based on the current orientation of the light sensor and the light sensor time stamp;

set a timer to elapse at the trigger time; and

control the image sensor to capture image data at the trigger time when the timer elapses.

2. The vehicular synchronization system of claim 1, wherein the processor is further configured to update the timer for an upcoming trigger time when a next desired light sensor orientation overlaps with the image sensor field of view.

3. The vehicular synchronization system of claim 1, wherein the reference clock is a component of a navigation system.

4. The vehicular synchronization system of claim 1, wherein the processor determines the trigger time based on the reference clock.

5. The vehicular synchronization system of claim 1, wherein the image sensor has an image sensor node associated with captured image data, wherein the synchronization control unit has a synchronization control node corresponding to the image sensor node, and wherein a time stamp node applies an image sensor time stamp to the captured image data based on the trigger time.

6. The vehicular synchronization system of claim 5, wherein the light sensor has a corresponding light sensor node associated with the captured light data and the data packet, and wherein the light data is synched with image data based on the image sensor time stamp and the data packet.

7. The vehicular synchronization system of claim 1, wherein the image sensor includes a first camera, a second camera, and a third camera, and wherein the first camera is associated with a first camera node, the second camera is associated with a second camera node, and the third camera is associated with a third camera node.

8. The vehicular synchronization system of claim 7, wherein the first camera is positioned left of the field of view, the second camera is centered on the field of view, and the third camera is positioned right of the field of view to create a stereoscopic view.

9. The vehicular synchronization system of claim 7, wherein a first time stamp node correlates the first camera node with a first synchronization control node, a second time stamp node correlates the second camera node with a second synchronization control node, and a third time stamp node correlates the third camera node with a third synchronization control node, and wherein the first time stamp node, the second time stamp node, and the third time stamp node collectively have image synched data.

10. The vehicular synchronization system of claim 9, wherein the light sensor is a light detection and ranging (LiDAR) sensor corresponding to, a LiDAR node associated with light data, and wherein the light data is synched with the image synched data to generate sensor synched data.

11. A computer-implemented method for synchronizing sensor nodes, comprising:

receiving a data packet from a light sensor having a radial view, wherein the light sensor is configured to capture light data from around a vehicle in a full rotation, and wherein the data packet identifies a current orientation of the light sensor and includes a light sensor time stamp;

determining a trigger time when a desired light sensor orientation overlaps with an image sensor field of view of an image sensor based on the current orientation of the light sensor and the light sensor time stamp;

setting a timer to elapse at the trigger time;

controlling the image sensor to capture image data at the trigger time; and

updating the timer for an upcoming trigger time when a next desired light sensor orientation overlaps with the image sensor field of view.

12. The computer-implemented method of claim 11, further comprising:

applying at least one image sensor times amp to image data from the image sensor based on a reference clock.

13. The computer-implemented method of claim 12, wherein the reference clock in integrated with a vehicle navigation system.

14. The computer-implemented method of claim 11, wherein captured image data is associated with the trigger time to create image synched data, and wherein the image synched data is correlated with the captured light data from the light sensor to generate sensor synched data.

15. A vehicular synchronization system, comprising:

an image sensor of a vehicle, wherein the image sensor is configured to capture image data in an image sensor field of view;

a light sensor having a radial view, wherein the light sensor is configured to capture light data from around the vehicle in a full rotation;

a synchronization control unit having a processor, the processor configured to;

receive a data packet from the light sensor, wherein the data packet identifies a current orientation of the light sensor and includes a light sensor time stamp;

determine a trigger time when a desired light sensor orientation overlaps with the image sensor field of view based on the current orientation of the light sensor and the light sensor time stamp;

set a timer to elapse at the trigger time; and

control the image sensor to capture image data at the trigger time when the timer elapses; and

a robot operating system having a memory and a light sensor node configured to receive the captured light data and at least one image sensor node configured to receive the captured image data and generate synchronized sensor data by combining the captured image data with the captured light data based on the trigger time.

16. The vehicular synchronization system of claim 15, wherein the processor is further configured to update the timer for an upcoming trigger time when a next desired light sensor orientation overlaps with the image sensor field of view.

17. The vehicular synchronization system of claim 15, wherein the robot operating system further comprises a time stamp node configured to apply an image sensor time stamp to the captured image data based on the trigger time.

18. The vehicular synchronization system of claim 15, wherein the image sensor includes a first camera, a second camera, and a third camera, and wherein the at least one image sensor node includes a first camera node, a second camera node, and a third camera node.

19. The vehicular synchronization system of claim 18, wherein a first time stamp node correlates the first camera node with a first synchronization control node, a second time stamp node correlates the second camera node with a second synchronization control node, and a third time stamp node correlates the third camera node with a third synchronization control node, and wherein the first time stamp node, the second time stamp node, and the third time stamp node collectively have image synched data.