Stereoscopic Camera With Automatic Obstruction Removal

• US 20110242286A1
• Filed: 03/31/2010
• Published: 10/06/2011
• Est. Priority Date: 03/31/2010
• Status: Active Grant

• Current Assignee: James Cameron
• Original Assignee: James Cameron
• Inventors: Pace, Vincent, Campbell, Patrick
• Granted Patent: US 8,456,518 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: H04N 13/133 Equalising the characteristics of different image components, e.g. their average brightness or colour balance
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof

Abstract

There is disclosed a stereographic camera system, a computing device and a method of operating a stereographic camera system. The stereoscopic camera system may a left camera and a right camera and an obstruction detector to determine whether or not a field of view of one of the left camera and the right camera is at least partially obstructed. A controller may cause the camera system to automatically output only non-obstructed image data, in response to a determination by the obstruction detector that a field of view of one of the left camera and the right camera is at least partially obstructed.

Claims

1. A stereoscopic camera system, comprising:

a left camera and a right camera

an obstruction detector to determine whether or not a field of view of one of the left camera and the right camera is at least partially obstructed

a controller that causes the camera system to automatically output only non-obstructed image data, in response to the obstruction detector determining that a field of view of one of the left camera and the right camera is at least partially obstructed.

2. The stereoscopic camera system of claim 1, wherein

the controller causes the camera system to automatically switch to a non-stereoscopic mode, outputting only image data from a non-obstructed one of the left camera and the right camera in response to the obstruction detector determining that the field of view of one of the left camera and the right camera is at least partially obstructed.

3. The stereoscopic camera system of claim 1, wherein

the controller automatically replaces obstructed portions of image data of an obstructed one of the left camera and the right camera with synthesized non-obstructed image data in response to the obstruction detector determining that a field of view of one of the left camera and the right camera is at least partially obstructed.

4. The stereoscopic camera system of claim 1, wherein

the controller causes the camera system to automatically output only non-obstructed image data, in response to the obstruction detector determining that a field of view of one of the left camera and the right camera is at least partially obstructed by selecting at least one of switching to a non-stereoscopic mode and replacing obstructed portions of image data, in response to a selection parameter.

5. The stereoscopic camera system of claim 1, wherein the obstruction detector comprises:

a left obstruction sensor and a right obstruction sensor to sense obstructions within the fields of view of the left camera and the right camera, respectively.

6. The stereoscopic camera system of claim 5, wherein

each of the left and right obstruction sensors senses a distance to a nearest object within the field of view of the respective camera

the field of view of each of the left and right camera is determined to be obstructed if the distance to the nearest object within the field of view is less than a predetermined minimum distance.

7. The stereoscopic camera system of claim 1, the obstruction detector further comprising:

an image processor to detect obstructions by analyzing at least some image data from the left camera and the right camera.

8. The stereoscopic camera system of claim 7, wherein the image processor performs actions comprising:

determining whether or not the left camera is obstructed by comparing image data from the left side of an image captured by the left camera and image data from the left side of an image captured by the right camera

determining whether or not the right camera is obstructed by comparing image data from the right side of the image captured by the left camera and image data from the right side of the image captured by the right camera.

9. The stereoscopic camera system of claim 8, wherein the image processor performs actions comprising:

computing first image metrics across a first analysis window near to a left side of a field of view of the left camera

computing second image metrics across a second processing window near to a right side of the field of view of the left camera

computing third image metrics across a third processing window near to a left side of a field of view of the right camera

computing fourth image metrics across a fourth processing window near to a right side of the field of view of the right camera

determining the left camera is obstructed if the first image metrics are substantially different from the third image metrics

determining the right camera is obstructed if the fourth image metrics are substantially different from the second image metrics.

10. The stereoscopic camera system of claim 9, wherein the first, second, third, and fourth image metrics each include at least one of, a mean, a standard deviation, a variance, and a high spatial frequency content of at least one of, a color component, a luminance component, and a chrominance component of the image data within the respective processing window.

11. A method of operating a stereoscopic camera including a left camera and a right camera, the method comprising:

determining whether or not a field of view of one of the left camera and the right camera is at least partially obstructed

automatically outputting only non-obstructed image data, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed.

12. The method of claim 11, wherein automatically outputting only non-obstructed image data, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

automatically switching the stereoscopic camera to a non-stereoscopic mode, outputting only image data from a non-obstructed one of the left camera and the right camera.

13. The method of claim 11, wherein automatically outputting only non-obstructed image data, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

automatically replacing obstructed portions of image data of an obstructed one of the left camera and the right camera with synthesized non-obstructed image data.

14. The method of claim 11, wherein automatically outputting only non-obstructed image data, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

selecting at least one of automatically switching the stereoscopic camera to a non-stereoscopic mode, outputting only image data from a non-obstructed one of the left camera and the right camera and automatically replacing obstructed portions of image data of an obstructed one of the left camera and the right camera with synthesized non-obstructed image data, in response to a selection parameter.

15. The method of claim 11, wherein determining whether or not a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

sensing obstructions within the fields of view of the left camera and the right camera with a left obstruction sensor and a right obstruction sensor, respectively.

16. The method of claim 15, wherein

each of the left and right obstruction sensors senses a distance to a nearest object within the field of view of the respective camera

the field of view of each of the left and right camera is determined to be obstructed if the distance to the nearest object within the respective field of view is less than a predetermined minimum distance.

17. The method of claim 11, wherein determining whether or not a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

analyzing at least some image data from the left camera and the right camera.

18. The method of claim 17, wherein analyzing at least some image data further comprises:

determining whether or not the left camera is obstructed by comparing image data from the left side of an image captured by the left camera and image data from the left side of an image captured by the right camera

determining whether or not the right camera is obstructed by comparing image data from the right side of the image captured by the left camera and image data from the right side of the image captured by the right camera.

19. The method of claim 17, wherein analyzing at least some image data further comprises:

computing first image metrics across a first analysis window near to a left side of a field of view of the left camera

computing second image metrics across a second processing window near to a right side of the field of view of the left camera

computing third image metrics across a third processing window near to a left side of a field of view of the right camera

computing fourth image metrics across a fourth processing window near to a right side of the field of view of the right camera

determining the left camera is obstructed if the first image metrics are substantially different from the third image metrics

determining the right camera is obstructed if the fourth image metrics are substantially different from the second image metrics.

20. The method of claim 19, wherein the first, second, third, and fourth image metrics each include at least one of, a mean, a standard deviation, a variance, and a high spatial frequency content of at least one of, a one color component, a luminance component, and a chrominance component of the image data within the respective processing window.

21. A computing device, comprising:

a processor

memory coupled to the processor

a camera interface for connection to a stereoscopic camera

a storage device coupled to the processor, the storage device storing instructions which, when executed by the processor, cause the computing device to perform actions including;

receiving, via the camera interface, left image data and right image data

determining, based on the left image data and the right image data, whether or not a field of view of one of the left camera and the right camera is at least partially obstructed

automatically outputting only non-obstructed image data, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed.

22. The computing device of claim 21, wherein determining whether or not a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

determining whether or not the left camera is obstructed by comparing image data from the left side of an image captured by the left camera and image data from the left side of an image captured by the right camera

determining whether or not the right camera is obstructed by comparing image data from the right side of the image captured by the left camera and image data from the right side of the image captured by the right camera.

23. The computing device of claim 21, wherein determining whether or not a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

computing first image metrics across a first analysis window near to a left side of a field of view of the left camera

computing second image metrics across a second processing window near to a right side of the field of view of the left camera

computing third image metrics across a third processing window near to a left side of a field of view of the right camera

computing fourth image metrics across a fourth processing window near to a right side of the field of view of the right camera

determining the left camera is obstructed if the first image metrics are substantially different from the third image metrics

determining the right camera is obstructed if the fourth image metrics are substantially different from the second image metrics.

24. The computing device of claim 23, wherein the first, second, third, and fourth image metrics each include at least one of, a mean, a standard deviation, a variance, and a high spatial frequency content of at least one of, a color component, a luminance component, and a chrominance component of the image data within the respective processing window.

25. The computing device of claim 21, wherein automatically outputting only non-obstructed image data, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

automatically switching the stereoscopic camera to a non-stereoscopic mode, outputting only image data from a non-obstructed one of the left camera and the right camera.

26. The computing device of claim 21, wherein automatically outputting only non-obstructed image data, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

automatically replacing obstructed portions of image data of an obstructed one of the left camera and the right camera with synthesized non-obstructed image data.

27. The computing device of claim 21, wherein automatically outputting only non-obstructed image data, in response to a determination that a field of view of one of the left camera and the right camera is at least partially obstructed further comprises:

selecting one of automatically switching the stereoscopic camera to a non-stereoscopic mode, outputting only image data from a non-obstructed one of the left camera and the right camera and automatically replacing obstructed portions of image data of an obstructed one of the left camera and the right camera with synthesized non-obstructed image data, in response to a selection parameter.