SYSTEM AND METHOD FOR PROCESSING CAPTURED IMAGES

• US 20180365890A1
• Filed: 06/17/2015
• Published: 12/20/2018
• Est. Priority Date: 06/17/2015
• Status: Active Grant

• Current Assignee: Rosemount Aerospace, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Original Assignee: Rosemount Aerospace, Inc. (Raytheon Technologies Corporation d/b/a RTX)
• Inventors: Ryde, Julian C., Ding, Xuchu
• Granted Patent: US 10,489,971 B2
• Time in Patent Office: Days
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G06T 15/405 using Z-buffer
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2219/2021 Shape modification
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light

Abstract

An image processing system configured to process perceived images of an environment includes a central processing unit (CPU) including a memory storage device having stored thereon a computer model of the environment, at least one sensor configured and disposed to capture a perceived environment including at least one of visual images of the environment and range data to objects in the environment, and a rendering unit (RU) configured and disposed to render the computer model of the environment forming a rendered model of the environment. The image processing system compares the rendered model of the environment to the perceived environment to update the computer model of the environment.

Claims

1. An image processing system configured to process perceived images of an environment comprising:

a central processing unit (CPU) including a memory storage device having stored thereon a computer model of the environment;

at least one sensor configured and disposed to capture a perceived environment including at least one of visual images of the environment and a range data to objects in the environment; and

a rendering unit (RU) configured and disposed to render the computer model of the environment forming a rendered model of the environment, wherein the image processing system compares the rendered model of the environment to the perceived environment to update the computer model of the environment.

2. The image processing system of claim 1, wherein the at least one sensor comprises a red, green, blue, and distance (RGBD) device.

3. The image processing system of claim 1, wherein the at least one sensor comprises a first sensor configured to capture visual images of the environment, and a second sensor configured to determine range to objects in the environment.

4. The image processing system according to claim 3, wherein the first sensor comprises a camera.

5. The image processing system according to claim 3, wherein the second sensor comprises a LiDAR sensor.

6. The image processing system according to claim 1, wherein the RU comprises a graphics processing unit (GPU).

7. A system for processing images captured by a moving platform, the system comprising:

a central processing unit (CPU), said CPU being interconnected functionally via a system bus to;

at least one sensor configured and disposed to capture a perceived environment including at least one of visual images of the environment and range data to objects in the environment;

a rendering unit (RU); and

at least one memory device thereupon stored a computer model of the environment and a set of instructions which, when executed by said CPU, causes the image processing system to;

capture a perceived environment through the at least one sensor, the perceived environment including visual images and range data;

render the computer model of the environment in the RU forming a rendered model of the environment;

compare the rendered model of the environment with the perceived environment; and

selectively update the computer model of the environment based on the perceived environment.

8. The system according to claim 7, wherein the set of instructions which, when executed by said CPU, causes the image processing system to:

employ an iterative process to compute a position and orientation of the moving platform by performing a Z-buffer optimization of the perceived environment against the computer model of the environment.

9. The method of claim 8, wherein the set of instructions which, when executed by said CPU, causes the image processing system to:

perform a direct numerical optimization on a per-point error metric of the Z-buffer optimization.

10. The system of claim 7, wherein the set of instructions which, when executed by said CPU, causes the image processing system to:

employ an iterative process to compute a position and orientation of the moving platform by performing an iterative closest point (ICP) optimization against the computer model of the environment.

11. The system of claim 10, wherein the ICP optimization includes a point-to-point optimization of the perceived environment.

12. The system of claim 10, wherein the ICP optimization includes a point-to-plane optimization of the perceived environment.

13. The system of claim 7, wherein the set of instructions which, when executed by said CPU, causes the image processing system to:

form at least one of a plurality of points, voxels, and textured polygon meshes that embody the perceived environment.

14. The system of claim 13, wherein each of the at least one of the plurality of points, voxels, and textured polygon meshes includes geographical positioning system (GPS) coordinates.