Formatting sensor data for use in autonomous vehicle communications platform

• US 10,101,747 B2
• Filed: 10/05/2017
• Issued: 10/16/2018
• Est. Priority Date: 12/11/2015
• Status: Active Grant

• Current Assignee: Uatc, LLC (Uber Technologies, Inc.)
• Original Assignee: Uber Technologies, Inc.
• Inventors: Tascione, Daniel, Waters, Justin, Sibenac, Mark, Brindza, Jordan, Strother, Daniel Leland, Jones, Morgan
• Primary Examiner(s): Truong, Nguyen
• Application Number:
  US15/726,050
• Publication Number:
  US 20180088584A1
• Time in Patent Office: 376 Days
• Field of Search: 348148
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60Y 2400/3015 Optical cameras
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 19/13 Receivers
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0276 using signals provided by a source external to the vehicle involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04J 3/0688 Change of the master or reference, e.g. take-over or failure of the master
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof

Abstract

A sensor synchronization system for an autonomous vehicle is described. Upon initializing a master clock on a master processing node for a sensor apparatus of the autonomous vehicle, the system determines whether an external timing signal is available. If the signal is not available, the system sets the master clock using a local timing signal from a low-power clock on the autonomous vehicle. Based on a clock cycle of the master clock, the system propagates timestamp messages to the sensors of the sensor apparatus, receives sensor data, and formats the sensor data based on the timestamp messages.

Claims

1. A method of operating a sensor apparatus of an autonomous vehicle, the method comprising:

upon initializing a master clock on a master processing node for the sensor apparatus, determining whether an external timing signal is available;

upon determining that the external timing signal is not available, setting the master clock using a local timing signal from a low-power clock on the autonomous vehicle;

based on a clock cycle of the master clock, propagating a plurality of pulse timestamp messages from the master processing node to a plurality of sensors of the sensor apparatus;

receiving sensor data from the plurality of sensors; and

formatting the sensor data based at least in part on the plurality of pulse timestamp messages to enable the sensor data to be used for navigating the autonomous vehicle.

2. The method of claim 1, further comprising:

upon determining that the external timing signal is available, synchronizing the master clock with the external timing signal.

3. The method of claim 1, further comprising:

dividing the clock cycle of the master clock into a plurality of timing intervals; and

generating a plurality of sensor pulses, in relation to the pulse timestamp messages, based at least in part on framerates or resolutions of each of the plurality of sensors of the sensor apparatus, wherein each of the plurality of sensor pulses are aligned with a respective one of the timing intervals.

4. The method of claim 3, wherein the number of the plurality of timing intervals into which the clock cycle of the master clock is divided is based at least in part on a framerate or resolution of a second sensor apparatus.

5. The method of claim 3, wherein the sensor data is captured by the plurality of sensors in accordance with a sensor activation schedule based on the plurality of sensor pulses.

6. The method of claim 1, further comprising:

continuously updating a skew and offset on the low-power clock based on the clock cycle of the master clock.

7. The method of claim 1, wherein the master clock and the low-power clock are located on the same microcontroller.

8. The method of claim 7, wherein the low-power clock maintains time even when the microcontroller is powered down.

9. The method of claim 1, wherein the external timing signal is received from a global positioning system (GPS) receiver.

10. The method of claim 1, further comprising:

generating a three-dimensional sensor image for navigating the autonomous vehicle based at least in part on the sensor data.

11. A sensor synchronization system for an autonomous vehicle, comprising:

a master processing node for a sensor apparatus of the autonomous vehicle, the master processing node comprising;

circuitry for a master clock; and

circuitry for a low-power clock that maintains time even when a microcontroller for the master processing node is powered down;

a memory that stores instructions for operating the sensor apparatus; and

one or more processors that, upon executing the instructions, cause the sensor synchronization system to;

upon initializing the master clock, determine whether an external timing signal is available;

upon determining that the external timing signal is not available, set the master clock using a local timing signal from the low-power clock;

based on a clock cycle of the master clock, propagate a plurality of pulse timestamp messages from the master processing node to a plurality of sensors of the sensor apparatus;

receive sensor data from the plurality of sensors; and

format the sensor data based at least in part on the plurality of pulse timestamp messages to enable the sensor data to be used for navigating the autonomous vehicle.

12. The sensor synchronization system of claim 11, further comprising:

upon determining that the external timing signal is available, synchronizing the master clock with the external timing signal.

13. The sensor synchronization system of claim 11, further comprising:

dividing the clock cycle of the master clock into a plurality of timing intervals; and

generating a plurality of sensor pulses, in relation to the pulse timestamp messages, based at least in part on framerates or resolutions of each of the plurality of sensors of the sensor apparatus, wherein each of the plurality of sensor pulses are aligned with a respective one of the timing intervals.

14. The sensor synchronization system of claim 13, wherein the number of the plurality of timing intervals into which the clock cycle of the master clock is divided is based at least in part on a framerate or resolution of a second sensor apparatus.

15. The sensor synchronization system of claim 13, wherein the sensor data is captured by the plurality of sensors in accordance with a sensor activation schedule based on the plurality of sensor pulses.

16. The sensor synchronization system of claim 11, further comprising:

continuously updating a skew and offset on the low-power clock based on the clock cycle of the master clock.

17. The sensor synchronization system of claim 11, wherein the master clock and the low-power clock are located on the microcontroller for the master processing node.

18. The sensor synchronization system of claim 11, wherein the external timing signal is received from a global positioning system (GPS) receiver.

19. The sensor synchronization system of claim 11, further comprising:

generating a three-dimensional sensor image for navigating the autonomous vehicle based at least in part on the sensor data.

20. A non-transitory computer-readable storage medium containing instructions that, when executed by a processor of a sensor synchronization system for an autonomous vehicle, causes the sensor synchronization system to:

upon initializing a master clock on a master processing node for a sensor apparatus of the autonomous vehicle, determine whether an external timing signal is available;

upon determining that the external timing signal is not available, set the master clock using a local timing signal from a low-power clock on the autonomous vehicle;

based on a clock cycle of the master clock, propagate a plurality of pulse timestamp messages from the master processing node to a plurality of sensors of the sensor apparatus;

receive sensor data from the plurality of sensors; and

format the sensor data based at least in part on the plurality of pulse timestamp messages to enable the sensor data to be used for navigating the autonomous vehicle.