Person detection in a marine environment

• US 10,372,976 B2
• Filed: 05/05/2016
• Issued: 08/06/2019
• Est. Priority Date: 05/05/2016
• Status: Active Grant

• Current Assignee: Brunswick Corporation
• Original Assignee: Brunswick Corporation
• Inventors: Kollmann, Troy J., Nowick, Jon M.
• Primary Examiner(s): Rahman, Mohammad J
• Application Number:
  US15/147,181
• Publication Number:
  US 20170323154A1
• Time in Patent Office: 1,188 Days
• Field of Search: None
• CPC Class Codes: B63B 79/10 using sensors, e.g. pressure sensors, strain gauges or accelerometers
• CPC Class Codes: B63B 79/40 for controlling the operation of vessels, e.g. monitoring their speed, routing or maintenance schedules
• CPC Class Codes: B63J 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

An object detection system for a marine vessel having at least one marine drive includes at least one image sensor positioned on the marine vessel and configured to capture an image of a marine environment on or around the marine vessel, and a processor. The object detection system further includes an image scanning module executable on the processor that receives the image as input. The image scanning module includes an artificial neural network trained to detect patterns within the image of the marine environment associated with one or more predefined objects, and to output detection information regarding a presence or absence of the one or more predefined objects within the image of the marine environment.

Claims

1. An object detection system for a marine vessel having at least one marine drive, the object detection system comprising:

at least one image sensor positioned on the marine vessel and configured to capture an image of a first marine environment around the marine vessel, wherein the first marine environment is a water surface behind the marine vessel and the at least one image sensor is a visible light camera positioned on the marine vessel at an angle and distance from the water surface;

a processor; and

an image scanning module executable on the processor and configured to receive the image as input and output information regarding a presence or absence of a person located in the first marine environment, the image scanning module comprising an artificial neural network trained to detect patterns within the image of the first marine environment associated with a person located behind the marine vessel, and to output detection information regarding the presence or absence of the person within the image of the first marine environment;

wherein the artificial neural network of the image scanning module is trained using positive and negative training images of objects on a water surface having a range of view angles and fields of view that approximate the angle and distance from the water surface of the image sensor positioned on the marine vessel.

2. The object detection system of claim 1, wherein the detection information indicates a probability value of the presence of the person within the image.

3. The object detection system of claim 1, wherein the artificial neural network of the image scanning module is trained using positive training images of persons on a water surface having the range of view angles and/or fields of view.

4. The object detection system of claim 1, further comprising

a second image sensor positioned on the marine vessel and configured to capture a second image of a second marine environment being a second water surface around the marine vessel, wherein the second marine environment is at a different position with respect to the marine vessel than the first marine environment and the second image sensor is positioned on the marine vessel at an angle and distance from the second water surface;

a second processor; and

a second image scanning module executable on the second processor, the second image scanning module including a second artificial neural network trained to receive the image of the second marine environment as input and output second detection information regarding the presence or absence of one or more predefined objects within the second image, wherein the second artificial neural network of the second image scanning module is trained using positive and negative training images of objects on a water surface having a range of view angles and fields of view that approximate the angle and distance from the second water surface of the second image sensor positioned on the marine vessel.

5. The system of claim 4, further comprising a powertrain control unit configured to receive the detection information from the first image scanning module and the second image scanning module and to generate a control instruction to the marine drive based on the detection information.

6. The system of claim 4, wherein the second image sensor is a visible light camera positioned on the marine vessel and the second marine environment is a water surface of an area in front of the marine vessel, and wherein the image scanning module is further configured to output the detection information regarding the presence of at least one of the person, a predefined marine animal, or a floating debris object of a predefined size range located in the area in front of the marine vessel.

11. The object detection system of claim 3, wherein the artificial neural network of the image scanning module is trained using negative training images of objects other than persons on a water surface having the range of view angles and/or fields of view.

14. The system of claim 1, wherein the range of view angles and fields of view of the training images approximate a view angle range or distance range of the image sensor from the water surface caused by motion of the marine vessel with respect to the first marine environment.

7. A method of detecting a presence of one or more objects in a marine environment of a marine vessel, the method comprising:

determining a range of view angles and fields of view that approximate an angle and distance from a water surface of an image sensor positioned on the marine vessel;

training an artificial neural network to detect patterns associated with a predefined object in images of a marine environment imaged by the image sensor, wherein the training includes;

providing positive training images of the predefined object on a water surface from view angles or fields of view that are within the range of view angles and fields of view of the image sensor;

providing negative training images of objects other than the predefined object on a water surface from view angles or fields of view that are within the range of view angles and/or fields of view of the image sensor;

recording an image with the image sensor positioned on the marine vessel;

providing the image as input to the artificial neural network; and

outputting detection information regarding a presence or absence of the object within the image.

8. The method of claim 7, wherein the detection information includes a probability of the presence of the predefined object in the image.

9. The method of claim 7, further comprising sending the detection information via a CAN bus to a powertrain control unit, and controlling at least one marine drive with the powertrain control unit based on the detection information.

10. The method of claim 7, further comprising generating an alert on the marine vessel based on the detection information.

15. The method of claim 7, wherein the range of view angles and fields of view of the training images approximate a view angle range or distance range of the image sensor from the water surface caused by motion of the marine vessel with respect to the first marine environment.

12. A driver detection system for a marine vessel having at least one marine drive, the driver detection system comprising:

at least one image sensor positioned on the marine vessel and configured to capture an image of a helm area on the marine vessel, wherein the at least one image sensor is a visible light camera positioned to image the helm area;

a processor; and

an image scanning module executable on the processor and configured to receive the image as input and output detection information regarding a presence or absence of a person located in the helm area, the image scanning module including an artificial neural network trained to detect patterns within the image associated with a person in the helm area, and to output the detection information regarding the presence or absence of the person located in the helm area.

13. A driver detection system of claim 12, wherein the artificial neural network of the image scanning module is trained for person detection in a particular helm area using training images of a same vessel model as the helm area or a same arrangement of vessel control input devices as the helm area.