Stereoscopic camera with automatic obstruction removal

• US 8,456,518 B2
• Filed: 03/31/2010
• Issued: 06/04/2013
• Est. Priority Date: 03/31/2010
• Status: Active Grant

• Current Assignee: James Cameron
• Original Assignee: James Cameron
• Inventors: Pace, Vincent, Campbell, Patrick
• Primary Examiner(s): AN, SHAWN S
• Application Number:
  US12/752,100
• Publication Number:
  US 20110242286A1
• Time in Patent Office: 1,161 Days
• Field of Search: 348/47, 348/46, 348/43, 348/51, 348/42
• US Class Current: 348/47
• CPC Class Codes: H04N 13/133 Equalising the characteristics of different image components, e.g. their average brightness or colour balance
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof

Abstract

There is disclosed a stereographic camera system, a computing device and a method of operating a stereographic camera system. The stereoscopic camera system may a left camera and a right camera and an obstruction detector to determine whether or not a field of view of one of the left camera and the right camera is at least partially obstructed. A controller may cause the camera system to automatically output only non-obstructed image data, in response to a determination by the obstruction detector that a field of view of one of the left camera and the right camera is at least partially obstructed.

Claims

1. A stereoscopic camera system, comprising:

a left camera and a right camera

an obstruction detector to determine whether or not a field of view of one of the left camera and the right camera is at least partially obstructed

a controller that causes the camera system to automatically switch to a non-stereoscopic mode, outputting only image data from a non-obstructed one of the left camera and the right camera, in response to the obstruction detector determining that a field of view of one of the left camera and the right camera is at least partially obstructed.

2. The stereoscopic camera system of claim 1, wherein the obstruction detector comprises:

a left obstruction sensor and a right obstruction sensor to sense obstructions within the fields of view of the left camera and the right camera, respectively.

3. The stereoscopic camera system of claim 2, wherein

each of the left and right obstruction sensors senses a distance to a nearest object within the field of view of the respective camera

the field of view of each of the left and right camera is determined to be obstructed if the distance to the nearest object within the field of view is less than a predetermined minimum distance.

4. The stereoscopic camera system of claim 1, the obstruction detector further comprising:

an image processor to detect obstructions by analyzing at least some image data from the left camera and the right camera.

5. The stereoscopic camera system of claim 4, wherein the image processor performs actions comprising:

determining whether or not the left camera is obstructed by comparing image data from the left side of an image captured by the left camera and image data from the left side of an image captured by the right camera

determining whether or not the right camera is obstructed by comparing image data from the right side of the image captured by the left camera and image data from the right side of the image captured by the right camera.

6. A stereoscopic camera system, comprising:

a left camera and a right camera

an image processor to determine whether or not a field of view of one of the left camera and the right camera is at least partially obstructed, the image processor performing actions comprising;

computing first image metrics across a first analysis window near to a left side of a field of view of the left camera

computing second image metrics across a second processing window near to a right side of the field of view of the left camera

computing third image metrics across a third processing window near to a left side of a field of view of the right camera

computing fourth image metrics across a fourth processing window near to a right side of the field of view of the right camera

determining the left camera is obstructed if the first image metrics are substantially different from the third image metrics

determining the right camera is obstructed if the fourth image metrics are substantially different from the second image metrics

a controller that causes the camera system to automatically output only non-obstructed image data, in response to the image processor determining that a field of view of one of the left camera and the right camera is at least partially obstructed.

7. The stereoscopic camera system of claim 6, wherein the first, second, third, and fourth image metrics each include at least one of, a mean, a standard deviation, a variance, and a high spatial frequency content of at least one of, a color component, a luminance component, and a chrominance component of the image data within the respective processing window.

8. A method of operating a stereoscopic camera including a left camera and a right camera, the method comprising:

determining whether or not a field of view of one of the left camera and the right camera is at least partially obstructed

automatically switching the stereoscopic camera to a non-stereoscopic mode, outputting only image data from a non-obstructed one of the left camera and the right camera, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed.

9. The method of claim 8, wherein determining whether or not a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

sensing obstructions within the fields of view of the left camera and the right camera with a left obstruction sensor and a right obstruction sensor, respectively.

10. The method of claim 9, wherein

each of the left and right obstruction sensors senses a distance to a nearest object within the field of view of the respective camera

the field of view of each of the left and right camera is determined to be obstructed if the distance to the nearest object within the respective field of view is less than a predetermined minimum distance.

11. The method of claim 8, wherein determining whether or not a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

analyzing at least some image data from the left camera and the right camera.

12. The method of claim 11, wherein analyzing at least some image data further comprises:

determining whether or not the left camera is obstructed by comparing image data from the left side of an image captured by the left camera and image data from the left side of an image captured by the right camera

determining whether or not the right camera is obstructed by comparing image data from the right side of the image captured by the left camera and image data from the right side of the image captured by the right camera.

13. A method of operating a stereoscopic camera including a left camera and a right camera, the method comprising:

computing first image metrics across a first analysis window near to a left side of a field of view of the left camera

computing second image metrics across a second processing window near to a right side of the field of view of the left camera

computing third image metrics across a third processing window near to a left side of a field of view of the right camera

computing fourth image metrics across a fourth processing window near to a right side of the field of view of the right camera

determining the left camera is obstructed if the first image metrics are substantially different from the third image metrics

determining the right camera is obstructed if the fourth image metrics are substantially different from the second image metrics

automatically outputting only non-obstructed image data in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed.

14. The method of claim 13, wherein the first, second, third, and fourth image metrics each include at least one of, a mean, a standard deviation, a variance, and a high spatial frequency content of at least one of, a one color component, a luminance component, and a chrominance component of the image data within the respective processing window.

15. A computing device, comprising:

a processor

memory coupled to the processor

a camera interface for connection to a stereoscopic camera

a storage device coupled to the processor, the storage device storing instructions which, when executed by the processor, cause the computing device to perform actions including;

receiving, via the camera interface, left image data and right image data

determining, based on the left image data and the right image data, whether or not a field of view of one of the left camera and the right camera is at least partially obstructed

automatically switching the stereoscopic camera to a non-stereoscopic mode, outputting only image data from a non-obstructed one of the left camera and the right camera, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed.

16. The computing device of claim 15, wherein determining whether or not a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

determining whether or not the left camera is obstructed by comparing image data from the left side of an image captured by the left camera and image data from the left side of an image captured by the right camera

determining whether or not the right camera is obstructed by comparing image data from the right side of the image captured by the left camera and image data from the right side of the image captured by the right camera.

17. A computing device, comprising:

a processor

memory coupled to the processor

a camera interface for connection to a stereoscopic camera

a storage device coupled to the processor, the storage device storing instructions which, when executed by the processor, cause the computing device to perform actions including

receiving, from the stereoscopic camera via the camera interface, left image data and right image data

computing first image metrics across a first analysis window near to a left side of a field of view of the left camera

computing second image metrics across a second processing window near to a right side of the field of view of the left camera

computing third image metrics across a third processing window near to a left side of a field of view of the right camera

computing fourth image metrics across a fourth processing window near to a right side of the field of view of the right camera

determining the left camera is obstructed if the first image metrics are substantially different from the third image metrics

determining the right camera is obstructed if the fourth image metrics are substantially different from the second image metrics

automatically outputting only non-obstructed, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed.

18. The computing device of claim 17, wherein the first, second, third, and fourth image metrics each include at least one of, a mean, a standard deviation, a variance, and a high spatial frequency content of at least one of, a color component, a luminance component, and a chrominance component of the image data within the respective processing window.