Methods and apparatuses for constructing a 3D sonar image of objects in an underwater environment

• US 10,281,577 B2
• Filed: 04/14/2016
• Issued: 05/07/2019
• Est. Priority Date: 04/20/2015
• Status: Active Grant

• Current Assignee: Navico Incorporated (Brunswick Corporation)
• Original Assignee: Navico Holding A.S. (Brunswick Corporation)
• Inventors: Hunt, Matthew, Kirmani, Shahzad F.
• Primary Examiner(s): Alsomiri, Isam A
• Assistant Examiner(s): Ndure, Amie M
• Application Number:
  US15/098,603
• Publication Number:
  US 20160306040A1
• Time in Patent Office: 1,118 Days
• CPC Class Codes: G01S 15/87 Combinations of sonar systems
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 15/8902 Side-looking sonar
• CPC Class Codes: G01S 15/96 for locating fish
• CPC Class Codes: G01S 7/539 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/6245 Stereoscopic displays; Three-dimensional displays; Pseudo-three dimensional displays
• CPC Class Codes: G01S 7/6272 producing cursor lines and indicia by electronic means

Abstract

Methods, apparatuses, and computer program products are therefore provided for producing a 3D image of an underwater environment. An example method for providing an image of an underwater environment includes analyzing sonar returns to identify and display objects, such as fish or debris, in a 3D view. Such an image allows for differentiation between the sea floor and objects in a 3D sonar view. Some example methods provide for real-time tracking of fish. Further, the fish or other objects may be displayed over a continuous surface geometry based on sonar returns from a lake, sea, or river “floor.”

Claims

1. A method for imaging an underwater environment, the method comprising:

generating, by a sonar signal processor, a 3D matrix based on sonar return data received from the underwater environment, wherein the 3D matrix comprises a plurality of sonar returns each defined by 3D positional values;

determining, based on at least the 3D positional values of each of the plurality of sonar returns, one or more groups of sonar returns among the plurality of sonar returns in the 3D matrix associated with one or more objects in the underwater environment, wherein the one or more objects are different than a bottom surface of the underwater environment, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment comprises;

generating, based on the 3D positional values of each of the plurality of sonar returns, a 3D representation of all of the plurality of sonar returns in the 3D matrix such that each of the plurality of sonar returns defines an x coordinate, a y coordinate, and a z coordinate;

determining one or more clusters among the plurality of sonar returns in the 3D representation, wherein the one or more clusters among the plurality of sonar returns in the 3D representation each comprise sonar returns that are within a threshold distance of each other within the 3D representation; and

determining a material of an object corresponding to each cluster based on power and gain of the at least one transducer element, density of the sonar returns within the cluster, and the strength value of at least one of the sonar returns within the cluster; and

generating a 3D image of the one or more groups of sonar returns associated with the one or more objects in the underwater environment.

2. The method of claim 1 further comprising displaying the 3D image of the one or more groups of sonar returns associated with the one or more objects in the underwater environment.

3. The method of claim 1, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment further comprises:

determining which corresponds to the bottom surface of the underwater environment by determining the cluster with the largest number of sonar returns; and

removing the cluster that corresponds to the bottom surface of the underwater environment from the 3D representation.

4. The method of claim 3, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment further comprises:

removing any sonar returns from the 3D representation that include a z coordinate below a corresponding z coordinate of the cluster that corresponds to the bottom surface of the underwater environment.

5. The method of claim 3, further comprising:

generating a 3D image of the cluster that corresponds to the bottom surface of the underwater environment; and

generating a 3D image of the underwater environment based on the 3D image of the cluster that corresponds to the bottom surface of the underwater environment and the 3D image of the one or more groups of sonar returns associated with the one or more objects.

6. The method of claim 5, further comprising:

generating a second 3D matrix based on second sonar return data received from the underwater environment;

determining one or more second groups of sonar returns among a plurality of sonar returns in the second 3D matrix associated with one or more objects in the underwater environment; and

updating the 3D image of the underwater environment by updating the 3D image of one or more groups of sonar returns associated with the one or more objects in the underwater environment without updating the 3D image of the cluster that corresponds to the bottom surface.

7. The method of claim 1, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment further comprises:

determining at least one of a shape, a size, and a direction of movement of an object associated with each cluster.

8. The method of claim 1, wherein determining the one or more groups of sonar returns associated with the one or more objects in the underwater environment further comprises:

generating a first 2D representation of each sonar return in at least one cluster, wherein the first 2D representation includes the x coordinate and the y coordinate of each sonar return in the at least one cluster;

generating a second 2D representation of each sonar return in at least one cluster, wherein the second 2D representation includes the x coordinate and the z coordinate of each sonar return in the at least one cluster;

generating a third 2D representation of each sonar return in at least one cluster, wherein the third 2D representation includes the y coordinate and the z coordinate of each sonar return in the at least one cluster; and

analyzing the first 2D representation, the second 2D representation, and the third 2D representation to determine one or more characteristics of the at least one cluster, wherein the one or more characteristics of the at least one cluster include at least one of a shape, a size, and a direction of movement of an object associated with each cluster.

9. The method of claim 1, further comprising:

analyzing the one or more groups of sonar returns associated with the one or more objects to determine movement of the one or more objects; and

generating the 3D image of the one or more sonar returns associated with the one or more objects by generating a trail that corresponds to the movement of the one or more objects.

10. The method of claim 1, wherein generating the 3D image of the one or more groups of sonar returns associated with the one or more objects comprises generating one or more mesh images characterizing the respective one or more objects based on the sonar returns associated with the corresponding object.

11. The method of claim 1, wherein generating the 3D image of the one or more groups of sonar returns associated with the one or more objects comprises positioning one or more predefined icons for the respective one or more objects, wherein each predefined icon is positioned within the underwater environment at a location corresponding to the sonar returns associated with the respective object.

12. The method of claim 1, wherein generating the 3D image of the one or more groups of sonar returns associated with the one or more objects comprises generating raw sonar return data associated with the sonar returns associated with the one or more objects.

13. The method of claim 1, wherein the 3D positional values of each sonar return comprise a distance value, an angle, and a position of capture of the sonar return.

14. An apparatus comprising a sonar signal processor and a memory including computer program code, the memory and the computer program code configured to, with the sonar signal processor, cause the apparatus to:

generate a 3D matrix based on sonar return data received from an underwater environment, wherein the 3D matrix comprises a plurality of sonar returns defined by 3D positional values;

determine, based on the 3D positional values of each of the sonar returns, one or more groups of sonar returns among the plurality of sonar returns in the 3D matrix associated with one or more objects in the underwater environment, wherein the one or more objects are different than a bottom surface of the underwater environment, wherein the determination of the one or more groups of sonar returns associated with the one or more objects in the underwater environment comprises;

generating, based on the 3D positional values of each of the plurality of sonar returns, a 3D representation of all of the plurality of sonar returns in the 3D matrix such that each of the plurality of sonar returns defines an x coordinate, a y coordinate, and a z coordinate;

determining one or more clusters among the plurality of sonar returns in the 3D representation, wherein the one or more clusters among the plurality of sonar returns in the 3D representation each comprise sonar returns that are within a threshold distance of each other within the 3D representation; and

determining a material of an object corresponding to each cluster based on power and gain of the at least one transducer element, density of the sonar returns within the cluster, and the strength value of at least one of the sonar returns within the cluster; and

generate a 3D image of the one or more groups of sonar returns associated with the one or more objects in the underwater environment.

15. The apparatus of claim 14, wherein the memory and the computer program code are further configured to, with the sonar signal processor, cause the apparatus to determine the one or more groups of sonar returns associated with the one or more objects in the underwater environment by:

determining which corresponds to the bottom surface of the underwater environment by determining the cluster with the largest number of sonar returns; and

removing the cluster that corresponds to the bottom surface of the underwater environment from the 3D representation.

16. A computer program product comprising a non-transitory computer readable medium having program code portions stored thereon, the program code portions configured, when said program product is run with a sonar signal processor, to:

generate a 3D matrix based on a sonar return data received from an underwater environment, wherein the 3D matrix comprises a plurality of sonar returns defined by 3D positional values;

determine, based on the 3D positional values of each of the plurality of sonar returns, one or more groups of sonar returns among the plurality of sonar returns in the 3D matrix associated with one or more objects in the underwater environment, wherein the one or more objects are different than a bottom surface of the underwater environment, by;

generating, based on the 3D positional values of each of the plurality of sonar returns, a 3D representation of all of the plurality of sonar returns in the 3D matrix such that each of the plurality of sonar returns defines an x coordinate, a y coordinate, and a z coordinate;

determining one or more clusters among the plurality of sonar returns in the 3D representation, wherein the one or more clusters among the plurality of sonar returns in the 3D representation each comprise sonar returns that are within a threshold distance of each other within the 3D representation; and

determining a material of an object corresponding to each cluster based on power and gain of the at least one transducer element, density of the sonar returns within the cluster, and the strength value of at least one of the sonar returns within the cluster; and

generate a 3D image of the one or more groups of sonar returns associated with the one or more objects in the underwater environment.