Collaborative data processing

• US 10,324,189 B2
• Filed: 10/24/2017
• Issued: 06/18/2019
• Est. Priority Date: 10/24/2017
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Luo, Zheng, Lv, Liyun, Zhang, Qingshan
• Primary Examiner(s): Aziz, Adnan
• Application Number:
  US15/792,517
• Publication Number:
  US 20190120964A1
• Time in Patent Office: 602 Days
• CPC Class Codes: B60W 2556/65 Data transmitted between vehicles
• CPC Class Codes: G01C 21/3819 Road shape data, e.g. outline of a route
• CPC Class Codes: G01C 21/3885 Transmission of map data to client devices; Reception of map data by client devices
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/51 Display arrangements
• CPC Class Codes: G08G 1/0112 from the vehicle, e.g. floating car data [FCD]
• CPC Class Codes: G08G 1/096791 where the origin of the information is another vehicle
• CPC Class Codes: G08G 1/20 Monitoring the location of vehicles belonging to a group, e.g. fleet of vehicles, countable or determined number of vehicles

Abstract

Embodiments are disclosed for collaboratively scanning and processing sensor data for building a map of an environment around a group of vehicles. An example an in-vehicle computing system of a vehicle includes a sensor subsystem in communication with an optical sensor, a processor, and memory storing instructions executable by the processor to instruct the optical sensor to scan an assigned region around the vehicle, receive locally scanned data corresponding to the assigned region, process the scanned data to build a first portion of a three-dimensional map, transmit the processed scanned data to at least one other vehicle, receive additional map data from the at least one other vehicle, and build a second, different portion of the three-dimensional map using the received additional map data. In a first example of the in-vehicle computing system, the optical sensor may additionally or alternatively include a Light Detection and Ranging (LiDAR) sensor system.

Claims

1. An in-vehicle computing system of a vehicle, the in-vehicle computing system comprising:

a display device;

a sensor subsystem in communication with an optical sensor;

a processor; and

memory storing instructions executable by the processor to;

form a group with at least one other vehicle;

instruct the optical sensor to scan an assigned region around the vehicle, the assigned region being one of a plurality of assigned regions in a collaborative scanning area around the group, each vehicle of the group being assigned, from a master vehicle of the group or a remote device, a different respective region of the plurality of assigned regions in the collaborative scanning area around the group, two or more of the plurality of assigned regions being at least partially overlapping with one another;

receive, from the optical sensor, locally scanned data corresponding to the assigned region;

process the locally scanned data to build a first portion of a three-dimensional map of an environment of the vehicle;

transmit the processed scanned data to the at least one other vehicle;

receive additional map data from the at least one other vehicle;

build a second, different portion of the three-dimensional map using the received additional map data; and

display the three-dimensional map via the display device, the three-dimensional map corresponding to the collaborative scanning area around the group.

2. The in-vehicle computing system of claim 1, wherein the optical sensor includes a Light Detection and Ranging (LiDAR) sensor system, and wherein the locally scanned data includes point-cloud data corresponding to the assigned region.

3. The in-vehicle computing system of claim 1, wherein forming the group comprises forming the group with the at least one other vehicle based on a proximity of the at least one other vehicle and/or a predicted route of the at least one other vehicle.

4. The in-vehicle computing system of claim 3, wherein the instructions are further executable to receive an assignment to scan the assigned region from another vehicle of the group and/or from a remote, cloud-based computing system in communication with the vehicle.

5. The in-vehicle computing system of claim 3, wherein the group is formed on an ad hoc basis to include only a selected number of neighboring vehicles, each of the vehicles in the group being separated from one another by less than a threshold distance.

6. The in-vehicle computing system of claim 3, wherein the processed scanned data is transmitted to the at least one other vehicle along with a group identification associated with the group wherein the additional map data includes scanned data in a portion of the assigned region that partially overlaps with a respective assigned region that is assigned to the at least one other vehicle, and wherein the scanned data in the portion of the assigned region is used to align the received additional map data with the locally scanned data.

7. The in-vehicle computing system of claim 6, wherein the received additional map data is received from the at least one other vehicle along with the group identification associated with the group, and wherein the group is further formed based on a speed of the at least one other vehicle and traffic information in the environment of the vehicle.

8. The in-vehicle computing system of claim 1, wherein the instructions are further executable to process the locally scanned data by converting the locally scanned data to a range image, obtaining a previously generated three-dimensional road map associated with the environment of the vehicle, subtracting static portions of the range image determined to be located on a surface of the three-dimensional road map, and processing only non-subtracted portions of the range image to build the first portion of the three-dimensional map.

9. The in-vehicle computing system of claim 8, wherein subtracting the static portions of the range image includes partitioning the range image using a mesh, registering the range image to the three-dimensional road map, and determining, for each partition of the range image, whether all vertices of the partition are included on the surface of the three-dimensional road map.

10. The in-vehicle computing system of claim 1, wherein the assigned region includes an assigned range in degrees around the vehicle and wherein the vehicle communicates with the at least one other vehicle according to a vehicle-to-vehicle communication protocol.

11. The in-vehicle computing system of claim 10, wherein the vehicle-to-vehicle communication protocol defines an encryption protocol to be used for transmissions between the vehicle and the at least one other vehicle, and wherein the instructions are further executable to encrypt the processed scanned data according to the determined encryption protocol.

12. The in-vehicle computing system of claim 11, wherein the received additional map data is encrypted according to the determined encryption protocol.

13. A method for performing a collaborative scanning operation at a first vehicle, the method comprising:

forming a group with at least one other vehicle;

receiving, from a selected vehicle of the group, an assignment of a first assigned region of a collaborative scanning area around the group, the first assigned region being one of a plurality of assigned regions in the collaborative scanning area, each vehicle of the group being assigned a different respective region of the plurality of assigned regions in the collaborative scanning area, two or more of the plurality of assigned regions being non-overlapping with one another; and

building a three-dimensional map of the collaborative scanning area using both locally scanned and processed data and additional map data, the additional map data corresponding to at least one of the plurality of scanned regions other than the first assigned region, the additional map data being scanned and processed by the at least one other vehicle and received from the at least one other vehicle via vehicle-to-vehicle communication, and the locally scanned and processed data corresponding to the first assigned region.

14. The method of claim 13, wherein the additional map data is not based on data that is scanned at the first vehicle.

15. The method of claim 13, wherein the locally scanned and processed data includes point-cloud data that is processed by converting the point-cloud data to a range image, registering the range image to a previously-generated road map, and subtracting data from the range image that is located on a surface of the road map.

16. The method of claim 13, wherein the at least one other vehicle is grouped with the first vehicle based on a proximity of the at least one other vehicle to the first vehicle.

17. An in-vehicle computing system of a first vehicle, the in-vehicle computing system comprising:

a sensor subsystem in communication with an optical sensor;

a processor; and

memory storing instructions executable by the processor to;

discover a second vehicle and a third vehicle within a threshold distance of the first vehicle;

form a group with the second vehicle and the third vehicle;

assign each of the first vehicle, the second vehicle, and the third vehicle a first, second, and third scan region, respectively, for scanning an environment of the group, each scan region including a portion of a collaborative scanning area around the group, at least two of the scan regions being at least partially overlapping with one another;

send a respective assignment to each of the second vehicle and the third vehicle instructing the second vehicle and the third vehicle to scan the second scan region and the third scan region, respectively;

scan, via the optical sensor, the first scan region and locally process resulting scanned data;

send the locally processed scanned data to the second vehicle and the third vehicle;

receive, from the second vehicle and the third vehicle, additional map data associated with the second scan region and third scan region, respectively; and

build a three-dimensional map of the collaborative scanning area using the locally processed scanned data and the additional map data from the second and third vehicles.

18. The in-vehicle computing system of claim 17, wherein locally processing the resulting scanned data is performed by locally processing only portions of the scanned data that are not located on a surface of a previously-generated road map of the first scan region.

19. The in-vehicle computing system of claim 18, wherein the optical sensor includes a Light Detection and Ranging (LiDAR) sensor system, and wherein the locally scanned data includes point-cloud data corresponding to the assigned region.

20. The in-vehicle computing system of claim 17, wherein the first, second, and third scan regions are determined to be assigned to the respective first, second, and third vehicles based on locations of the first, second, and third vehicles relative to one another.