Camera Fields of View for Object Detection

• US 20180288320A1
• Filed: 04/03/2017
• Published: 10/04/2018
• Est. Priority Date: 04/03/2017
• Status: Abandoned Application

• Current Assignee: Uatc, LLC (Uber Technologies, Inc.)
• Original Assignee: Uber Technologies, Inc.
• Inventors: Melick, Peter, Kuvelker, Jay, Zajac, Brian Thomas, Wellington, Carl
• Application Number:
  US15/477,638
• Publication Number:
  US 20180288320A1
• Time in Patent Office: Days
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The present disclosure provides systems and methods for detecting, localizing, and classifying objects that are proximate to an autonomous vehicle. A sensor system can include one or more ranging systems and a plurality of cameras. The plurality of cameras can be positioned such that a field of view for each camera of the plurality of cameras overlaps a field of view of at least one adjacent camera. The one or more ranging systems can be configured to transmit ranging data to a perception system for detecting objects of interest and the plurality of cameras can be configured to transmit image data to the perception system for classifying the objects of interest.

Claims

1. A sensor system comprising:

one or more ranging systems; and

a plurality of cameras,

the plurality of cameras positioned such that a field of view for each camera of the plurality of cameras overlaps a field of view of at least one adjacent camera,

the plurality of cameras further positioned about at least one of the one or more ranging systems such that a combined field of view of the plurality of cameras comprises an approximately 360-degree field of view;

wherein the one or more ranging systems are configured to transmit ranging data to a perception system for detecting objects of interest; and

wherein the plurality of cameras are configured to transmit image data to the perception system for classifying the objects of interest.

2. The sensor system of claim 1, wherein the plurality of cameras are positioned such that an object partially viewed by a first camera on a boundary of a first camera field of view will be fully viewed by a second camera adjacent to the first camera within a second adjacent camera field of view, thereby providing a full view of the object to replace any split view of the object captured by the first camera.

3. The sensor system of claim 1, wherein the plurality of cameras are positioned such that an object at least partially viewed within a first image area on a boundary of a first camera field of view will also be viewed within a second image area in an adjacent camera field of view, the second image area overlapping the first image area and providing a greater view of the object than provided in the first image area.

4. The sensor system of claim 1, wherein each camera field of view comprises a horizontal field of view of about 90 degrees.

5. The sensor system of claim 1, wherein the one of the one or more ranging systems comprises a light detection and ranging (LIDAR) device.

6. The sensor system of claim 5, wherein the LIDAR device is centrally mounted relative to the plurality of cameras.

7. The sensor system of claim 5, wherein one or more of the plurality of cameras are positioned and oriented relative to the LIDAR device to reduce parallax effects relative to objects detected within the ranging data from the LIDAR device and within the image data from the plurality of cameras.

8. The sensor system of claim 1, wherein a field of view overlap among a first set of the plurality of cameras is greater than a field of view overlap among a second set of the plurality of cameras.

9. The sensor system of claim 1, wherein the plurality of cameras comprise:

a forward-facing camera;

two forward side cameras; and

two rear side cameras.

10. The sensor system of claim 9, wherein the plurality of cameras further comprise a rear-facing camera.

11. An autonomous vehicle comprising:

a vehicle computing system comprising;

one or more processors; and

one or more memories including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising;

detecting objects of interest; and

classifying the detected objects of interest;

a sensor system comprising;

a plurality of cameras, the plurality of cameras positioned such that a field of view for each camera of the plurality of cameras overlaps a field of view of at least one adjacent camera; and

wherein the plurality of cameras are configured to transmit image data to the vehicle computing system for classifying objects of interest.

12. The autonomous vehicle of claim 11, further comprising:

one or more ranging systems;

wherein the one or more ranging systems are configured to transmit ranging data to the vehicle computing system for detecting where the objects of interest are located proximate to the autonomous vehicle.

13. The autonomous vehicle of claim 11, wherein the vehicle computing system classifies the objects of interest within image data from the plurality of cameras as one or more of a vehicle, a bicycle, or a pedestrian.

14. The autonomous vehicle of claim 11, wherein one or more of the plurality of cameras are configured to provide a front bias in a field of view overlap.

15. The autonomous vehicle of claim 11, wherein a field of view overlap among two or more forward facing cameras is greater than a field of view overlap of two or more rearward facing cameras.

16. The autonomous vehicle of claim 11, wherein the sensor system is mounted on a roof of the autonomous vehicle.

17. The autonomous vehicle of claim 16, wherein the plurality of cameras includes one or more forward facing cameras mounted on the roof of the autonomous vehicle in a position closer to a front of the autonomous vehicle than a position of a driver seat within the autonomous vehicle.

18. The autonomous vehicle of claim 16, wherein the plurality of cameras comprises:

at least one left rear side camera mounted on a left side of the roof of the autonomous vehicle to provide a view of a left lane adjacent to the autonomous vehicle; and

at least one right rear side camera mounted on a right side of the roof of the autonomous vehicle to provide a view of a right lane adjacent to the autonomous vehicle.

19. The autonomous vehicle of claim 18, wherein the left rear side camera is mounted near the left edge of the roof of the autonomous vehicle and wherein the right rear side camera is mounted near the right edge of the roof of the autonomous vehicle.

20. A computer-implemented method of detecting objects of interest comprising:

receiving, by one or more computing devices, ranging data from one or more ranging systems configured to transmit ranging signals relative to an autonomous vehicle;

receiving, by the one or more computing devices, image data from a plurality of cameras configured to capture images relative to the autonomous vehicle, the plurality of cameras positioned such that a field of view for each camera of the plurality of cameras overlaps a field of view of at least one adjacent camera;

detecting, by the one or more computing devices, an object of interest proximate to the autonomous vehicle within the ranging data;

determining, by the one or more computing devices, a first image area within image data captured by a first camera within the plurality of cameras containing the object of interest;

determining, by the one or more computing devices, a second image area within image data captured by a second camera within the plurality of cameras containing the object of interest, the second image area overlapping the first image area and providing a greater view of the object than provided in the first image area; and

classifying, by the one or more computing devices, the object of interest based at least in part on the second image area.