Point cloud visualization using bi-modal color schemes based on 4D lidar datasets

• US 9,165,383 B1
• Filed: 11/21/2011
• Issued: 10/20/2015
• Est. Priority Date: 11/21/2011
• Status: Active Grant

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Exelis Incorporated (L3Harris Technologies, Inc.)
• Inventors: Rodgers, Christopher T., Mendez-Rodriguez, Javier
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): CRADDOCK, ROBERT J
• Application Number:
  US13/301,072
• Time in Patent Office: 1,429 Days
• Field of Search: 701/9, 701/514
• US Class Current: 1/1
• CPC Class Codes: G01C 23/00 Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/51 Display arrangements
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes

Abstract

A system is provided for alerting a crew in an airborne platform. The system includes a module for receiving point cloud data from a LIDAR system, including range data between the LIDAR system and multiple points in the point cloud. The system also includes a module for placing the multiple points into first and second zones, wherein the first zone has range data of points in the point cloud located within a first distance from the airborne platform, and the second zone has range data of points located further than the first distance. The first distance is predetermined by an operational environment of the airborne platform. The system further includes a color module for coloring the points in the first zone with a first color composition and coloring the points in the second zone with a second color composition. A color display is provided for displaying the colored points in the first and second zones.

Claims

1. A method of real-time point cloud visualization on a color display of a piloted aircraft to provide situational awareness to pilots flying under obscured condition, comprising the steps of:

receiving point cloud data from a LIDAR, including range data of vertical obstructions between a piloted aircraft and multiple points in the point cloud indicating vertical obstructions with respect to the piloted aircraft;

dividing the multiple points into a plurality of zones, each zone including points indicating positions of the vertical obstructions within a respective interval of range away from the piloted aircraft;

determining an index vector for each zone of the plurality of zones using an altitude value for the piloted aircraft;

coloring the multiple points in each zone using a plurality of colors defined by the index vector;

indicating a range of the vertical obstructions from the piloted aircraft by displaying, on the color display device of the piloted aircraft, the multiple points indicating positions of the vertical obstructions in each zone with the plurality of colors with reference to the piloted aircraft;

calculating a derivative of the index vector as a function of range; and

changing the color between points, if the derivative is not continuous between the points.

2. The method of claim 1 wherein receiving the point cloud data includes receiving scanning angle data to determine orientation of each of the multiple points within the field of view (FOV) of the range sensor.

3. The method of claim 1 wherein dividing the multiple points includes dividing the multiple points into first and second zones, wherein

the first zone includes points located closely to the range sensor, closely defined as a range interval less than a predetermined threshold value, and

the second zone includes points located remotely from the range sensor, remotely defined as a range interval greater than the predetermined threshold value.

4. The method of claim 3 wherein the predetermined threshold is determined by an operational environment of an airborne platform.

5. The method of claim 1 wherein receiving the point cloud data includes receiving the point cloud data from the range sensor disposed on an airborne platform.

6. The method of claim 1 wherein coloring the multiple points includes the steps of:

receiving a color composition (CC) for each of the zones, and

assigning color index values to each of the zones,

wherein each color index value includes a value between zero and a maximum number in the respective zone, and

the color index value is based on a distance between the range sensor and a respective point in the respective zone.

7. The method of claim 6 wherein receiving the color composition for each of the zones includes receiving a 3×

N matrix in which each column corresponds to one of an RGB color space, or one of an HSI color space.

8. The method of claim 6 further including the color index value is based on a minimum distance between an airborne platform housing the range sensor and a ground terrain below the airborne platform.

9. The method of claim 8 wherein the distance and minimum distance are height of each point above the ground terrain and height of the airborne platform above the ground terrain, respectively.

10. The method of claim 1 wherein coloring the multiple points includes the steps of:

determining a first range value between the range sensor and a first point in the point cloud;

determining a second range value between the range sensor and a second point in the point cloud;

determining a change in value between the first range value and the second range value; and

coloring the first point differently from the second point, if the change in value exceeds a predetermined value.

11. A computer program product which, when loaded in a computer system, causes the computer system to carry out steps for generating a real-time point cloud visualization on a display of a piloted aircraft to provide situational awareness to the pilot flying under obscured conditions, the steps comprising:

receiving point cloud data from a LIDAR system disposed in the piloted aircraft, including range data between the LIDAR system and multiple points indicating positions of the vertical obstructions in the point cloud data,

placing the multiple points into first and second zones, wherein the first zone includes range data of points in the point cloud located within a first distance from the piloted aircraft, and the second zone includes range data of points in the point cloud located further than the first distance from the piloted aircraft,

determining an index vector for the first zone and the second zone using an altitude value for the piloted aircraft;

coloring the points in the first zone with a first color composition and coloring the points in the second zone with a second color composition, and

displaying on a color display of the piloted aircraft colored points in the first zone and colored points in the second zone, wherein the respective color compositions indicate a ranges of a vertical obstruction to the piloted aircraft;

assigning color index values to each of the zones, wherein;

each color index value includes a value between zero and a maximum number in the respective zone, and

the color index value depends on a distance between the LIDAR system disposed in the piloted aircraft and a respective point in the respective zone defining an indexing function;

calculating a derivative of the indexing function containing two points in a respective zone, and

changing, on the color display, the colors between the two points in the respective zone, if the derivative of the indexing function is not continuous between the first and second points.

12. The computer program product of claim 11, wherein when loaded in a computer system causes the computer system to carry out steps, further comprising steps of:

setting a predetermined threshold

of the piloted aircraft, and

the first distance extends from the piloted aircraft to the predetermined threshold.

13. The computer program product of claim 11 wherein

the color index value further depends on a minimum distance between the piloted aircraft and a ground terrain below the piloted aircraft.

14. The computer program product of claim 13 wherein

the distance and minimum distance are height of each point above the ground terrain and height of the piloted aircraft above the ground terrain, respectively.

15. A computer program product which, when loaded in a computer system, causes the computer system to carry out steps for generating a display of vertical obstructions for alerting a crew in an airborne platform, the steps comprising:

receiving point cloud data from a LIDAR system disposed in an airborne platform, including range data between the LIDAR system and multiple points in the point cloud indicating vertical obstructions with respect to the airborne platform,

placing the multiple points into first and second zones, wherein the first zone includes range data of points in the point cloud representing vertical obstructions located within a first distance from the airborne platform, and the second zone includes range data of points in the point cloud representing objects located further than the first distance from the airborne platform, wherein the first distance is predetermined by an operational environment of the airborne platform,

coloring the points in the first zone with a first color composition and coloring the points in the second zone with a second color composition,

displaying the colored points in the first zone and the colored points in the second zone, wherein the color composition in the first zone indicates a vertical obstruction to the airborne platform,

assigning color index values to each of the zones,

wherein each color index value includes a value between zero and a maximum number in the respective zone, and

the color index value depends on a distance between the LIDAR system and a respective point in the respective zone, and

the color index value is based on the following equations;