Real time machine vision and point-cloud analysis for remote sensing and vehicle control

• US 9,796,400 B2
• Filed: 01/20/2016
• Issued: 10/24/2017
• Est. Priority Date: 11/27/2013
• Status: Expired due to Fees

• Current Assignee: Condor Acquisition SUB II Incorporated (Luminar Technologies, Inc.)
• Original Assignee: Solfice Research Inc. (Luminar Technologies, Inc.)
• Inventors: Puttagunta, Shanmukha Sravan, Chraim, Fabien, Gupta, Anuj, Harvey, Scott, Creadore, Jason, Mills, Graham
• Primary Examiner(s): Dager, Jonathan M
• Assistant Examiner(s): Dunn, Alex C
• Application Number:
  US15/002,380
• Publication Number:
  US 20160221592A1
• Time in Patent Office: 643 Days
• CPC Class Codes: B61L 2205/04 Satellite based navigation systems, e.g. global positioning system [GPS]
• CPC Class Codes: B61L 23/041 Obstacle detection
• CPC Class Codes: B61L 23/34 for indicating the distance between vehicles or trains by the transmission of signals therebetween
• CPC Class Codes: B61L 25/025 Absolute localisation, e.g. providing geodetic coordinates
• CPC Class Codes: B61L 25/04 Indicating or recording train identities
• CPC Class Codes: B61L 27/04 Automatic systems, e.g. controlled by train; Change-over to manual control

Abstract

Methods and apparatus for real time machine vision and point-cloud data analysis are provided, for remote sensing and vehicle control. Point cloud data can be analyzed via scalable, centralized, cloud computing systems for extraction of asset information and generation of semantic maps. Machine learning components can optimize data analysis mechanisms to improve asset and feature extraction from sensor data. Optimized data analysis mechanisms can be downloaded to vehicles for use in on-board systems analyzing vehicle sensor data. Semantic map data can be used locally in vehicles, along with onboard sensors, to derive precise vehicle localization and provide input to vehicle to control systems.

Claims

1. A computer-implemented cloud computing system for mapping assets within point-cloud survey data, the system comprising:

a front end component accessible via a digital communications network for receiving a point-cloud dataset procured by local environment sensors installed within a plurality of remote vehicles;

a data storage component, the data storage component storing the point-cloud dataset and subdividing the point-cloud dataset into a plurality of data chunks, each data chunk comprising (a) data associated with a region-of-interest within the point-cloud dataset at least partially bounded by regions-of-interest associated with other data chunks, and (b) a padding region about region-of-interest;

a processing unit comprising a compute cluster with a plurality of computation nodes, the processing unit receiving streamed data chunks from the data storage component and distributing the data chunks amongst the computation nodes for parallel application of one or more analysis mechanisms to each data chunk to extract asset information; and

a map generator combining asset information extracted from the data analysis mechanisms into an output map comprising a collection of vectors in a coordinate frame representing an area under observation, for use by other remote vehicles.

2. The computer-implemented cloud computing system of claim 1, in which each data chunk comprises one or more tiles of point-cloud data.

3. The computer-implemented cloud computing system of claim 2, in which each tile comprises a subset of point-cloud data within a rectangular column extending lengthwise along the Earth'"'"'s gravity vector.

4. The computer-implemented cloud computing system of claim 3, in which each data chunk contains a number of contiguous tiles optimized to achieve a target data chunk size.

5. A computer-implemented cloud computing system for mapping assets within point-cloud survey data, the system comprising:

a front end component accessible via a digital communications network for receiving a point-cloud dataset procured by local environmental sensors installed within one or more remote vehicles;

a data storage component, the data storage component storing the point-cloud dataset and subdividing the point-cloud dataset into a plurality of data chunks;

a processing unit comprising a compute cluster, the processing unit receiving streamed data chunks from the data storage component and applying one or more analysis mechanisms to each data chunk to extract asset information; and

a map generator combining asset information extracted from the data analysis mechanisms into an output map for use by other remote vehicles;

in which the map generator further comprises an annotation integrity verifier comparing asset information in an output map with asset information in one or more prior output maps corresponding to a common local environment, to generate a notification when discrepancies are detected.

6. A computer-imlemented cloud computing system for mapping assets within point-cloud survey data, the system comprising:

a front end component accessible via a digital communications network for receiving a point-cloud dataset procured by local environment sensors installed within one or more remote vehicles;

a data storage component, the data storage component storing the point-cloud dataset and subdividing the point-cloud dataset into a plurality of data chunks;

a processing unit comprising a compute cluster, the processing unit receiving streamed data chunks from the data storage component and applying one or more analysis mechanisms to each data chunk to extract asset information;

a map generator combining asset information extracted from the data analysis mechanisms into an output map for use by other remote vehicles;

a compression mechanism operating to compress the point-cloud data prior to storage within the data storage component; and

a decompression mechanism operating to decompress the point-cloud data prior to application of the analysis mechanisms by the processing unit;

whereby the compression mechanism modulates its compression ratio to balance a data retrieval rate from the data storage component, with a data processing rate achievable by the processing unit.