Methods and apparatuses for reconstructing a 3D sonar image

• US 10,061,025 B2
• Filed: 03/03/2016
• Issued: 08/28/2018
• Est. Priority Date: 03/05/2015
• Status: Active Grant

• Current Assignee: Navico Incorporated (Brunswick Corporation)
• Original Assignee: Navico Holding A.S. (Brunswick Corporation)
• Inventors: Kirmani, Shahzad F.
• Primary Examiner(s): Baghdasaryan, Hovhannes
• Application Number:
  US15/060,032
• Publication Number:
  US 20160259052A1
• Time in Patent Office: 908 Days
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 15/8902 Side-looking sonar
• CPC Class Codes: G01S 7/527 Extracting wanted echo signals Doppler systems G01S15/50
• CPC Class Codes: G01S 7/6245 Stereoscopic displays; Three-dimensional displays; Pseudo-three dimensional displays

Abstract

Methods, apparatuses, and computer program products are provided for producing a 3D image of an underwater environment. 2D sonar return data that defines an angle and distance value may be taken from a theoretical 2D slice of the underwater environment. Multiple sets of 2D sonar return data are taken from corresponding 2D slices of the underwater environment during travel. The multiple sets of 2D sonar return data can be inputted into a 3D point cloud/matrix based on the distance value and angle and the corresponding 2D slice associated with each sonar return. The 3D point cloud/matrix can then be processed to determine a cluster of sonar returns that correspond to the bottom surface. Additional filtering may be performed and the bottom surface can be reconstructed to form a 3D image for display to a user.

Claims

1. A method for imaging an underwater environment, wherein the underwater environment defines a bottom surface, the method comprising:

receiving a set of 2D sonar return data received by at least two transducer elements traversing a body of water;

generating, by a sonar signal processor, a 3D matrix based on a plurality of sets of 2D sonar return data received as the at least one transducer element traverses the body of water, wherein the 3D matrix comprises a plurality of sonar returns defined by each of the received plurality of sets of 2D sonar return data, wherein each of the plurality of sonar returns defines a strength value, a distance value, and an angle;

determining, based on at least the angle of each of the plurality of sonar returns and at least one of the strength value or the distance value of each of the plurality of sonar returns, a group of some of the plurality of sonar returns in the 3D matrix that correspond to the bottom surface of the underwater environment, wherein determining the group of some of the plurality of sonar returns in the 3D matrix comprises;

generating, based on the distance value and the angle of each of the plurality of sonar returns, a 3D representation of all of the plurality of sonar returns in the 3D matrix such that each of the plurality of sonar returns defines an x coordinate, a y coordinate, and a z coordinate;

determining a plurality of clusters based on at least some of the plurality of sonar returns in the 3D representation, wherein each of the plurality of clusters comprises sonar returns that are within a threshold distance of each other within the 3D representation; and

selecting a cluster with the largest number of sonar returns from among the plurality of clusters such that the sonar returns within the selected cluster form the determined group of some of the plurality of sonar returns in the 3D matrix that correspond to the bottom surface of the underwater environment; and

generating a 3D representation of the bottom surface of the underwater environment based on the group of some of the plurality of sonar returns in the 3D matrix.

2. The method of claim 1 further comprising displaying the 3D representation of the bottom surface of the underwater environment.

3. The method of claim 1 further comprising filtering the group of some of the plurality of sonar returns in the 3D matrix that corresponds to the bottom surface to ensure that sonar returns with matching x coordinates and y coordinate do not have different z coordinates.

4. The method of claim 1, wherein generating the 3D representation of the bottom surface of the underwater environment comprises:

generating a 2D surface of triangles with points, wherein each point of each triangle defines the x coordinate and the y coordinate of each sonar return from the group of some of the plurality of sonar returns that correspond to the bottom surface of the underwater environment; and

updating each point of the triangles of the 2D surface with the z coordinate of the corresponding sonar return from the group of some of the plurality of sonar returns that correspond to the bottom surface of the underwater environment to generate the 3D representation.

5. The method of claim 4, wherein generating the 3D representation of the bottom surface of the underwater environment further comprises scaling the 3D representation to real-world coordinates.

6. The method of claim 1, wherein generating the 3D representation of the bottom surface of the underwater environment further comprises defining a transparency value of each of the group of some of the plurality of sonar returns that correspond to the bottom surface of the underwater environment based on the strength value of each sonar return.

7. The method of claim 1 further comprising filtering each set of 2D sonar return data based on the angle and the strength value for each of the plurality of sonar returns in the 3D matrix corresponding to each set of 2D sonar return data.

8. The method of claim 7, wherein filtering each set of 2D sonar return data comprises removing any sonar return from each set of 2D sonar return data that does not have the highest strength value among sonar returns within a range of angles.

9. The method of claim 1 further comprising filtering the plurality of sonar returns in the 3D matrix by removing sonar returns determined to be outside at least one of a threshold distance from another sonar return or threshold strength value.

10. The method of claim 1, wherein the plurality of sets of 2D sonar data define a first plurality of sets of 2D sonar data and the 3D representation of the bottom surface of the underwater environment defines a first 3D representation of the bottom surface of the underwater environment, wherein the method further comprises:

generating a second 3D matrix based on a second plurality of sets of 2D sonar return data received as the at least one transducer element traverses the body of water, wherein the second 3D matrix comprises a second plurality of sonar returns defined by each of the received second plurality of sets of 2D sonar return data, wherein each of the second plurality of sonar returns defines a strength value, a distance value, and an angle, wherein the second plurality of sets of 2D sonar return data are subsequently received after the first plurality of sets of 2D sonar return data;

determining, based on at least the angle of each of the second plurality of sonar returns and at least one of the strength value or the distance value of each of the second plurality of sonar returns, a group of some of the second plurality of sonar returns in the second 3D matrix that correspond to the bottom surface of the underwater environment; and

generating a second 3D representation of the bottom surface of the underwater environment based on the group of some of the second plurality of sonar returns in the second 3D matrix.

11. The method of claim 10 further comprising updating at least a portion of the first 3D representation of the bottom surface of the underwater environment near an end of the first 3D representation based on at least a portion of the second 3D representation of the bottom surface of the underwater environment near a beginning of the second 3D representation.

12. The method of claim 11, wherein updating at least the portion of the first 3D representation comprises updating a z coordinate of points within a threshold distance of the end of the first 3D representation to correspond to an average of z coordinates from both points within the threshold distance of the end of the first 3D representation and points within a second threshold distance of the beginning of the second 3D representation.

13. An apparatus comprising a sonar signal processor and a memory including computer program code, the memory and the computer program code configured to, with the sonar signal processor, cause the apparatus to:

receive a set of 2D sonar return data received by at least two transducer elements traversing a body of water;

generate a 3D matrix based on a plurality of sets of 2D sonar return data received as the at least one transducer element traverses the body of water, wherein the 3D matrix comprises a plurality of sonar returns defined by each of the received plurality of sets of 2D sonar return data, wherein each of the plurality of sonar returns defines a strength value, a distance value, and an angle;

determine, based on at least the angle of each of the plurality of sonar returns and at least one of the strength value or the distance value of each of the plurality of sonar returns, a group of some of the plurality of sonar returns in the 3D matrix that correspond to the bottom surface of the underwater environment, wherein determining the group of some of the plurality of sonar returns in the 3D matrix comprises;

generating, based on the distance value and the angle of each of the plurality of sonar returns, a 3D representation of all of the plurality of sonar returns in the 3D matrix such that each of the plurality of sonar returns defines an x coordinate, a y coordinate, and a z coordinate;

determining a plurality of clusters based on at least some of the plurality of sonar returns in the 3D representation, wherein each of the plurality of clusters comprises sonar returns that are within a threshold distance of each other within the 3D representation; and

selecting a cluster with the largest number of sonar returns from among the plurality of clusters such that the sonar returns within the selected cluster form the determined group of some of the plurality of sonar returns in the 3D matrix that correspond to the bottom surface of the underwater environment; and

generate a 3D representation of the bottom surface of the underwater environment based on the group of some of the plurality of sonar returns in the 3D matrix.

14. The apparatus of claim 13 further comprising the memory and the computer program code configured to, with the sonar signal processor, cause the apparatus to display the 3D representation of the bottom surface of the underwater environment.

15. The apparatus of claim 13, wherein generating the 3D representation of the bottom surface of the underwater environment comprises:

generating a 2D surface of triangles with points, wherein each point of each triangle defines the x coordinate and the y coordinate of each sonar return from the group of some of the plurality of sonar returns that correspond to the bottom surface of the underwater environment; and

updating each point of the triangles of the 2D surface with the z coordinate of the corresponding sonar return from the group of some of the plurality of sonar returns that correspond to the bottom surface of the underwater environment to generate the 3D representation.

16. The apparatus of claim 13 further comprising the memory and the computer program code configured to, with the sonar signal processor, cause the apparatus to filter each set of 2D sonar return data based on the angle and the strength value for each of the plurality of sonar returns in the 3D matrix corresponding to each set of 2D sonar return data, wherein the filtering comprises removing any sonar return from each set of 2D sonar return data that does not have the highest strength value among sonar returns within a range of angles.

17. Computer program product comprising a non-transitory computer readable medium having program code portions stored thereon, the program code portions configured, when said program product is run with a sonar signal processor, to:

receive a set of 2D sonar return data received by at least two transducer elements traversing a body of water;

generate a 3D matrix based on a plurality of sets of 2D sonar return data received as the at least one transducer element traverses the body of water, wherein the 3D matrix comprises a plurality of sonar returns defined by each of the received plurality of sets of 2D sonar return data, wherein each of the plurality of sonar returns defines a strength value, a distance value, and an angle;

determine, based on at least the angle of each of the plurality of sonar returns and at least one of the strength value or the distance value of each of the plurality of sonar returns, a group of some of the plurality of sonar returns in the 3D matrix that correspond to the bottom surface of the underwater environment, wherein determining the group of some of the plurality of sonar returns in the 3D matrix comprises;

generating, based on the distance value and the angle of each of the plurality of sonar returns, a 3D representation of all of the plurality of sonar returns in the 3D matrix such that each of the plurality of sonar returns defines an x coordinate, a y coordinate, and a z coordinate;

determining a plurality of clusters based on at least some of the plurality of sonar returns in the 3D representation, wherein each of the plurality of clusters comprises sonar returns that are within a threshold distance of each other within the 3D representation; and

selecting a cluster with the largest number of sonar returns from among the plurality of clusters such that the sonar returns within the selected cluster form the determined group of some of the plurality of sonar returns in the 3D matrix that correspond to the bottom surface of the underwater environment; and

generate a 3D representation of the bottom surface of the underwater environment based on the group of some of the plurality of sonar returns in the 3D matrix.

18. The computer program product of claim 17 further comprising the program code portions configured, when said program product is run with a sonar signal processor, to display the 3D representation of the bottom surface of the underwater environment.