Method of processing sensor data for navigating a vehicle

• US 9,135,510 B2
• Filed: 08/24/2011
• Issued: 09/15/2015
• Est. Priority Date: 08/25/2010
• Status: Expired due to Fees

• Current Assignee: University of Sydney
• Original Assignee: University of Sydney
• Inventors: Peynot, Thierry, Brunner, Christopher
• Primary Examiner(s): TISSOT, ADAM D
• Application Number:
  US13/818,915
• Publication Number:
  US 20130211659A1
• Time in Patent Office: 1,483 Days
• Field of Search: 701/23, 701/28, 701/29.2, 701/41, 701/33.7, 340/435, 340/438, 340/903
• US Class Current: 1/1
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Apparatus for and method of processing sensor data for the purpose of navigating a vehicle (for example an autonomous or semi-autonomous vehicle), the sensor data being from a sensor (for example a camera) mounted on the vehicle. The method can include can include: for a number of time-steps, measuring a value of a parameter of a scene using the sensor to produce a sequence of images of the scene; determining a value of one or more image quality metrics (for example, spatial entropy and spatial information) in each image in the sequence; and identifying as either valid or invalid, a sub-sequence of images in the sequence based on the determined metric values.

Claims

1. A method of processing sensor data for navigating a vehicle, the sensor data being from a sensor mounted on the vehicle, the method comprising:

for a number of time-steps, measuring a value of a parameter of a scene using the sensor to produce a sequence of images of the scene;

using a computing device to determine a value of one or more image quality metrics in each image in the sequence, wherein computing said values of said image quality metrics does not include determining position values of the vehicle from the images; and

using the computing device to identify as either valid or invalid, a sub-sequence of images in the sequence based on an evolution of the determined image quality metric values within that sub-sequence.

2. A method according to claim 1, wherein the vehicle is an autonomous vehicle, or a semi-autonomous vehicle.

3. A method according to claim 2 further comprising:

performing a perception process for the autonomous vehicle in which image subsequences identified as valid are treated differently to image sub-sequences identified as invalid.

4. A method according to claim 1, wherein the one or more image quality metric values comprise more than one different image quality metric.

5. A method according to claim 4, wherein the identifying as either valid or invalid, images in the sequence based on the determined metric values, comprises:

for a sub-sequence, comparing the values of different metrics determined in that sub-sequence.

6. A method according to claim 1, wherein the one or more image quality metrics comprise one or more of the following metrics:

a metric indicative of brightness of an image, a metric indicative of contrast within an image, a metric indicative blur within an image, a metric indicative of image sharpness, a metric indicative of a level of spatial information within an image, and a metric indicative of spatial entropy of an image.

7. A method according to claim 1, wherein the one or more image quality metrics comprise the following metrics:

a metric indicative of brightness of an image, a metric indicative of contrast within an image, and a metric indicative of a level of spatial information within an image.

8. A method according to claim 1, wherein the one or more image quality metrics comprises the following metrics:

a metric indicative of a level of spatial information within an image, and a metric indicative of spatial entropy of an image.

9. A method according to claim 1, comprising:

for each of the time-steps, measuring a value of a further parameter of the scene fusing a further sensor to produce a further sequence of images of the scene; and

determining a metric value of the one or more image quality metrics in each image in the further sequence;

wherein

identifying as either valid or invalid, a sub-sequence of images in the sequence based on the determined metric values, comprises comparing the values of a metric determined in that sub-sequence to the values of that metric determined in the further subsequence of images; and

wherein;

the further sub-sequence is a sub-sequence of images in the further sequence;

the further sub-sequence corresponds to the same time-steps as the sub-sequence;

the sensor and the further sensor are different types of sensors; and

the parameter and the further parameter are different parameters.

10. A method of navigating a vehicle, the method comprising processing sensor data according to claim 1.

13. A computer program or plurality of computer programs arranged such that when executed by a computer system will cause the computer system to operate in accordance with the method of claim 1.

14. A machine readable non-transitory storage medium storing a computer program or at least one of a plurality of computer programs for executing the method according to claim 1.

15. A method according to claim 1, wherein each of the one or more image quality metrics determined in each image in the sequence is one of:

a metric indicative of brightness of an image;

a metric indicative of contrast within an image;

a metric indicative of a level of spatial information within an image; and

a metric indicative of spatial entropy of an image.

11. Apparatus for processing sensor data for the purpose of navigating a vehicle, the apparatus comprising:

a sensor for mounting on the vehicle, the sensor being configured to, for a number of time-steps, measure a value of a parameter of a scene to produce a sequence of images of the scene; and

one or more processors arranged to;

determine a metric value of one or more image quality metrics in each image in the sequence, wherein computing said values of said image quality metrics does not include determining position values of the vehicle from the images; and

identify as either valid or invalid, a sub-sequence of images in the sequence based on an evolution of the determined image quality metric values within that subsequence.

12. Apparatus of claim 11, wherein the one or more processors are configured to perform a perception process for an autonomous vehicle using the image sub-sequences identified as valid.

16. Apparatus of claim 11, wherein each of the one or more image quality metrics determined in each image in the sequence is one of:

a metric indicative of brightness of an image;

a metric indicative of contrast within an image;

a metric indicative of a level of spatial information within an image; and

a metric indicative of spatial entropy of an image.