Detecting vehicle movement through wheel movement

• US 10,019,805 B1
• Filed: 09/29/2015
• Issued: 07/10/2018
• Est. Priority Date: 09/29/2015
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Robinson, Craig Lewin
• Primary Examiner(s): Rush, Eric
• Application Number:
  US14/869,015
• Time in Patent Office: 1,015 Days
• Field of Search: 382100, 382103, 382104, 382107, 382287, 382291, 356 27- 29, 356138, 35613909, 356614, 356615, 356620, 356622, 356955, 356958, 348148, 348149, 348154, 348155, 348169, 340435, 340436, 340438, 340441, 340463-467, 340901, 340903, 340936-938, 701 19, 701 20, 701 70, 701 71, 701 74, 701 93, 701 96, 701119, 701300, 701301, 702142, 702145-149, 702151, 180197
• CPC Class Codes: B60K 2031/0025 Detecting position of target vehicle, e.g. vehicle driving ahead from host vehicle
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G07C 5/0866 the electronic data carrier being a digital video recorder in combination with video camera
• CPC Class Codes: G08G 1/01 Detecting movement of traffic to be counted or controlled G08G1/07 - G08G1/14 take precedence
• CPC Class Codes: G08G 1/0112 from the vehicle, e.g. floating car data [FCD]
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Aspects of the disclosure relate to detecting vehicle movement. For example, one or more computing devices may receive first image data representative of a vehicle'"'"'s wheel and second image data representative of the wheel captured subsequent to the capture of the first image data. The one or more computing devices may determine a first location of a first portion of the wheel based on the first image data, and a second location of the first portion of the wheel based on the second image data. The one or more computing devices may calculate a value based on the angular distance between the first location and the second location of the first portion, and based on the value, determine whether the vehicle is in motion. Upon determining the vehicle is in motion the one or more computing devices may provide a signal that the vehicle is in motion.

Claims

1. A computer implemented method for detecting vehicle movement, the method comprising:

receiving, with one or more computing devices, first image data representative of a first vehicle'"'"'s wheel;

determining, with the one or more computing devices and based on the first image data, a first location of a first portion of the wheel;

receiving, with the one or more computing devices, second image data representative of the wheel captured subsequent to the capture of the first image data;

determining, with the one or more computing devices and based on the second image data, a second location of the first portion of the wheel;

generating, with the one or more computing devices, an initial determination whether the first vehicle is in motion;

assigning, with the one or more computing devices, a confidence level to the initial determination;

determining the confidence level satisfies a threshold value; and

upon determining the confidence level satisfies the threshold value;

calculating, with the one or more computing devices, a value based on an angular distance between the first location and the second location of the first portion;

determining, with the one or more computing devices and based on the value, whether the first vehicle is in motion; and

providing a signal that the first vehicle is in motion.

2. The method of claim 1, wherein receiving first image data representative of the wheel further comprises:

receiving, from a sensor located on a second vehicle, a series of images of an environment surrounding the second vehicle;

analyzing the series of images to identify a first image having image data that includes visual characteristics of the first vehicle and the wheel of the first vehicle; and

selecting, as the first image data, the image data associated with said visual characteristics of the wheel.

3. The method of claim 1, wherein receiving second image data representative of the wheel further comprises:

receiving, from a sensor located on a second vehicle, a series of images of an environment surrounding the second vehicle;

analyzing the series of images to identify a second image having image data that includes visual characteristics of the first vehicle and the wheel of the first vehicle; and

selecting, as the second image data, the image data associated with said visual characteristics of the wheel.

4. The method of claim 3, wherein the second image data is updated to image data from an image captured subsequently to the second image upon a determination that the first vehicle is not in motion.

5. The method of claim 1, wherein determining a first location of a first portion of the wheel further comprises:

determining a trackable feature within the first image data representative of the wheel;

determining a location of the trackable feature relative to the center of the wheel; and

selecting the location of the trackable feature as the first location of the first portion of the wheel.

6. The method of claim 1, wherein determining a second location of a first portion of the wheel further comprises:

determining a trackable feature within the second image data representative of the wheel;

determining a location of the trackable feature relative to the center of the wheel; and

selecting the location of the trackable feature as the second location of the first portion of the wheel.

7. The method of claim 1, wherein determining whether the first vehicle is in motion further comprises:

comparing the value to a first threshold value; and

determining the first vehicle is in motion if the value is greater than the first threshold value.

8. The method of claim 1, wherein generating the initial determination whether the first vehicle is in motion comprises:

determining, with the one or more computing devices and based on the first image data, a first linear distance between the first vehicle and an external object;

determining, with the one or more computing devices and based on the second image data, a second linear distance between the first vehicle and the external object;

calculating, with the one or more computing devices, a difference value between the first linear distance and the second linear distance; and

based on the difference value, generating the initial determination whether the first vehicle is in motion.

9. The method of claim 1, wherein determining the confidence level satisfies a threshold value includes determining the confidence level is less than the threshold value.

10. A system for detecting vehicle movement comprising:

one or more sensors;

one or more computing devices; and

memory storing instructions, the instructions executable by the one or more computing devices;

wherein the instructions comprise;

determining, based on first image data representative of a first vehicle'"'"'s wheel received from the one or more sensors, a first location of a first portion of the wheel;

determining, based on second image data representative of the wheel captured subsequent to the capture of the first image data received from the one or more sensors, a second location of the first portion of the wheel;

generating an initial determination whether the first vehicle is in motion;

assigning a confidence level to the initial determination;

determining the confidence level satisfies a threshold value; and

upon determining the confidence level satisfies the threshold value;

calculating a value based on an angular distance between the first location and the second location of the first portion;

determining, based on the value, whether the first vehicle is in motion; and

providing a signal that the first vehicle is in motion.

11. The system of claim 10, wherein receiving first image data representative of the wheel further comprises:

receiving, from the one or more sensors, a series of images of an environment surrounding a second vehicle;

analyzing the series of images to identify a first image having image data that includes visual characteristics of the first vehicle and the wheel of the first vehicle; and

selecting, as the first image data, the image data associated with said visual characteristics of the wheel.

12. The system of claim 10, wherein receiving second image data representative of the wheel further comprises:

receiving, from the one or more sensors, a series of images of an environment surrounding a second vehicle;

analyzing the series of images to identify a second image having image data that includes visual characteristics of the first vehicle and the wheel of the first vehicle; and

selecting, as the second image data, the image data associated with said visual characteristics of the wheel.

13. The system of claim 12, wherein the second image data is updated to image data from an image captured subsequently to the second image upon a determination that the first vehicle is not in motion.

14. The system of claim 10, wherein determining a first location of a first portion of the wheel further comprises:

determining a trackable feature within the first image data representative of the wheel;

determining a location of the trackable feature relative to the center of the wheel; and

selecting the location of the trackable feature as the first location of the first portion of the wheel.

15. The system of claim 10, wherein determining a second location of a first portion of the wheel further comprises:

determining a trackable feature within the second image data representative of the wheel;

determining a location of the trackable feature relative to the center of the wheel; and

selecting the location of the trackable feature as the second location of the first portion of the wheel.

16. The system of claim 10, wherein determining whether the first vehicle is in motion further comprises:

comparing the value to a first threshold value; and

determining the first vehicle is in motion if the value is greater than the first threshold value.

17. The system of claim 10, wherein generating the initial determination whether the first vehicle is in motion comprises:

determining, based on the first image data, a first linear distance between the first vehicle and an external object;

determining, based on the second image data, a second linear distance between the first vehicle and the external object;

calculating a difference value between the first linear distance and the second linear distance; and

based on the difference value, generating the initial determination whether the first vehicle is in motion.

18. The system of claim 10, wherein determining the confidence level satisfies a threshold value includes determining the confidence level is less than the threshold value.

19. A non-transitory computer-readable storage medium on which computer-readable instructions of a program are stored, the instructions, when executed by a processor, cause the processor to perform a method, the method comprising:

receiving, from one or more computing devices, first image data representative of a first vehicle'"'"'s wheel;

determining, based on the first image data, a first location of a first portion of the wheel;

receiving, from the one or more computing devices, second image data representative of the wheel captured subsequent to the capture of the first image data;

determining, based on the second image data, a second location of the first portion of the wheel;

generating an initial determination whether the first vehicle is in motion;

assigning a confidence level to the initial determination;

determining the confidence level satisfies a threshold value; and

upon determining the confidence level satisfies the threshold value;

calculating a value based on an angular distance between the first location and the second location of the first portion;

determining, based on the value, whether the first vehicle is in motion; and

providing a signal that the first vehicle is in motion.

20. The non-transitory computer-readable storage medium of claim 19, wherein receiving first image data representative of the wheel further comprises:

receiving, from a sensor located on a second vehicle, a series of images of an environment surrounding the second vehicle;

analyzing the series of images to identify a first image having image data that includes visual characteristics of the first vehicle and the wheel of the first vehicle; and

selecting, as the first image data, the image data associated with said visual characteristics of the wheel.