SYSTEMS AND METHODS FOR CONTROLLING VEHICLE POSITION AND ORIENTATION

• US 20150378361A1
• Filed: 06/30/2015
• Published: 12/31/2015
• Est. Priority Date: 06/30/2014
• Status: Active Grant

• Current Assignee: Collin Walker, Gary Walker
• Original Assignee: Collin Walker, Gary Walker
• Inventors: Walker, Collin, Walker, Gary
• Granted Patent: US 9,734,583 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01C 21/203 Specially adapted for sailing ships
• CPC Class Codes: G05D 1/0206 specially adapted to water vehicles
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/75 involving models

Abstract

Systems and related methods for controlling vehicle position and movement are disclosed. A system includes one or more computers configured to receive data corresponding to video image frames of at least a portion of a course, and analyze the video image frame to estimate at least one of position and orientation of the vehicle within the course. A method includes identifying groups of pixels from the video image frames that are determined to correspond to non-background objects in the course. The method also includes correlating at least a portion of the identified groups of pixels with known objects in the course, and analyzing positions of the groups of pixels within the video image frames and known positions of at least one of an image capture device and known objects relative to the course to estimate at least one of the position and orientation of the vehicle within the course.

Claims

1. A method of controlling water vehicle position and movement, comprising:

identifying, with a computer, groups of pixels from video frame data corresponding to video image frames taken of at least a portion of a water course, the identified groups of pixels determined to correspond to objects in the water course;

correlating at least a portion of the identified groups of pixels with known objects in the water course;

estimating at least one of position and orientation of a water vehicle within the water course by analyzing positions of the groups of pixels within video image frames corresponding to the video frame data and known spatial positions of at least one of an image capture device that captured the video frame data and the known objects within the water course;

estimating at least one of steering parameters and acceleration parameters that are determined to navigate the water vehicle on a desired path through the water course; and

transmitting, with the computer, navigation data including the at least one of the steering parameters and the acceleration parameters to at least one of;

a user interface configured to provide the at least one of the steering parameters and the acceleration parameters to a driver of the water vehicle; and

an automatic operating device configured to receive the navigation data and automatically adjust at least one of steering and acceleration of the water vehicle according to the at least one of the steering parameters and the acceleration parameters.

2. The method of claim 1, wherein identifying groups of pixels from video frame data comprises identifying the groups of pixels from video frame data provided by one or more cameras secured to the water vehicle.

3. The method of claim 1, wherein identifying groups of pixels from video frame data comprises identifying the groups of pixels from video frame data provided by one or more cameras located on-shore.

4. The method of claim 1, wherein identifying groups of pixels determined to correspond to objects in a water course comprises using a background subtraction process to identify the groups of pixels determined to correspond to the non-background objects in the water course.

5. The method of claim 4, wherein using a background subtraction process comprises using a visual background extractor (ViBe) process.

6. The method of claim 1, wherein estimating at least one of position and orientation of a water vehicle further comprises analyzing sensor data from one or more sensors to determine the at least one of the position and the orientation of the water vehicle.

7. The method of claim 1, wherein correlating at least a portion of the identified groups of pixels with known objects in the water course comprises:

assigning a score to each group of pixels of at least a portion of the identified groups of pixels for its fitness for correlation with each known object of at least a portion of the known objects; and

correlating a different group of pixels with each known object of the portion of the known objects, the group of pixels correlated to each of the known objects having a best score for the correlated known object assigned thereto.

8. The method of claim 1, wherein correlating at least a portion of the identified groups of pixels with known objects in the water course comprises assigning a different pair of groups of pixels to each of a plurality of known pairs of buoys in the water course.

9. The method of claim 1, wherein correlating at least a portion of the identified groups of pixels with known objects in the water course comprises eliminating at least one identified group of pixels from consideration for correlation with the known objects if the at least one identified group of pixels has properties that are inconsistent with properties of the known objects.

10. The method of claim 1, wherein correlating at least a portion of the identified groups of pixels with known objects in the water course comprises correlating the at least a portion of the identified groups of pixels with lights mounted to the water vehicle.

11. A system, comprising:

one or more image capture devices configured to provide data corresponding to video image frames of a course that a vehicle is to navigate through to one or more computers;

the one or more computers, wherein the one or more computers are configured to receive the data corresponding to the video image frames from the one or more image capture devices, and execute computer-readable instructions including;

a segmentation software module configured to identify groups of pixels from the video image frames that correspond to determined objects in the course;

a model fitting software module configured to correlate known objects in the course with the groups of pixels identified by the segmentation software module; and

a pose estimation software module configured to estimate a position and orientation of the vehicle by analyzing image locations of the groups of pixels within the video image frames and known locations of at least one of;

the one or more image capture devices; and

the known objects correlated with the groups of pixels by the model fitting software module; and

one or more interface devices operably coupled to the one or more computers and configured to at least partially automate navigation of the vehicle through the course.

12. The system of claim 11, wherein the one or more image capture devices include at least one image capture device located remote from the vehicle.

13. The system of claim 11, further comprising the vehicle, wherein the vehicle comprises a water vehicle.

14. The system of claim 11, further comprising the vehicle, wherein the vehicle includes the one or more interface devices and at least one of:

the one or more computers; and

the one or more image capture devices.

15. The system of claim 11, wherein the one or more interface devices configured to at least partially automate navigation of the vehicle through the course include a user interface configured to provide perceptible feedback to a driver of the vehicle, the perceptible feedback configured to provide information instructing the driver of the vehicle to perform at least one manual control of the vehicle selected from the group consisting of steering the vehicle, accelerating the vehicle, and decelerating the vehicle.

16. The system of claim 15, wherein the user interface is configured to provide at least one of visual feedback, audible feedback, and haptic feedback to the driver of the vehicle.

17. The system of claim 11, wherein the one or more interface devices configured to at least partially automate navigation of the vehicle through the course include one or more automatic operating devices operably coupled to the one or more computers, and wherein the one or more computers include a control module configured to control the one or more automatic operating devices.

18. The system of claim 17, wherein the one or more automatic operating devices include a steering element of the vehicle operably coupled to an electrically controlled motor configured to adjust the steering element of the vehicle responsive to control signals received from the control module.

19. The system of claim 11, further comprising at least one of the known objects in the course, wherein the at least one of the known objects includes one or more lights, wherein the one or more lights are configured to at least one of flash on and off according to a predetermined pattern the one or more computers are configured to recognize, and change a color of light emitted by one or more lights.

20. A system comprising:

a water vehicle comprising;

an image capture device secured to the water vehicle, the image capture device configured to capture video image frames of at least a portion of a water course;

a computer operably coupled to the image capture device, the computer comprising at least one processing element operably coupled to at least one data storage device including computer-readable instructions stored thereon, the processing element configured to execute the computer-readable instructions, the computer-readable instructions configured to instruct the processing element to;

identify groups of pixels from the video image frames that correspond to determined non-background objects in the water course;

correlate at least a portion of the identified groups of pixels with known objects in the water course; and

estimate at least one of position and orientation of the water vehicle within the water course by comparing locations, within the video image frames, of the identified groups of pixels that correlate with the known objects in the water course to known locations of the known objects in the water course; and

a user interface configured to provide human-perceptible feedback to a driver of the water vehicle to indicate corrections to be made to at least one of a position, a direction, and a speed of the water vehicle.