System and method for correlating camera views

• US 7,629,995 B2
• Filed: 09/24/2004
• Issued: 12/08/2009
• Est. Priority Date: 08/06/2004
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Palmer, Robert Gerald Jr., Salivar, William Matthew, Rozmus, John Michael
• Primary Examiner(s): Philippe; Gims S
• Application Number:
  US10/949,701
• Publication Number:
  US 20060028548A1
• Time in Patent Office: 1,901 Days
• Field of Search: 348/143, 348/36, 348/159, 348/335, 348/584, 351/246, 356/139.03
• US Class Current: 348/143
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method for correlating views of two or more video camera systems includes obtaining a plurality of data point coordinate sets to represent relative positioning between the camera systems and data point objects in the environment. The views may be correlated through Interpolation or extrapolation using the obtained data point coordinate sets. Devices such as lasers may also be used to correlate views of two or more video camera systems.

Claims

1. A method for correlating views of two or more video camera systems in an environment, the method comprising:

(a) capturing a image of a portion of the environment with a pan-tilt-zoom camera having a field of view;

(b) activating a laser beam to create a detectable reference on an object in the environment, the object being within the field of view of the PTZ camera;

(c) capturing an immersive image of the environment with an immersive camera system;

(d) identifying the detectable reference in the environment in the immersive image by virtue of the laser beam;

(e) determining the direction of the detectable reference relative the immersive camera; and

(f) correlating the pan-tilt-zoom and immersive cameras such that the object is viewable in the image and the immersive image.

2. The method of claim 1, wherein the laser is a laser range finder.

3. The method of claim 1, wherein the act of correlating comprises orienting the pan-tilt-zoom camera in response to a real-time feedback algorithm.

4. The method of claim 1, wherein the act of correlating comprises collecting a plurality of data point coordinate sets for the pan-tilt-zoom and immersive cameras.

5. The method of claim 4, further comprising the act of storing the data point coordinate sets as correlation data.

6. The method of claim 4, wherein each data point coordinate set represents a direction of an object relative the respective camera.

7. The method of claim 6, wherein each data point coordinate set further represents a distance of an object relative the respective camera.

8. The method of claim 1, wherein the act of identifying comprising filtering the light in the immersive image.

9. A method for correlating views of two or more video camera systems, the method comprising:

(a) capturing an immersive video image of an environment with a first video camera system, the first video camera system having a wide-angle field of view;

(b) capturing a second video image of the environment with a second video camera system, the second video camera system having a field of view that is narrower than the wide-angle field of view, the second video camera system comprising a pan-tilt-zoom camera;

(c) activating a laser, the laser being oriented substantially parallel with the line of sight of the pan-tilt-zoom camera, the laser providing a detectable reference in the environment;

(d) detecting an object of interest in the immersive video image;

(e) detecting the detectable reference in the immersive video image; and

(f) orienting the pan-tilt-zoom camera to capture the object of interest within the field of view of the pan-tilt-zoom camera based at least in part on the location of the detectable reference in the immersive video image.

10. The method of claim 9, wherein the step of detecting the detectable reference comprises using a filter.

11. The method of claim 9, wherein the act of orienting the pan-tilt-zoom camera is performed automatically using a feedback loop.