System and method for correlating camera views

• US 8,692,881 B2
• Filed: 06/25/2009
• Issued: 04/08/2014
• Est. Priority Date: 08/06/2004
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Salivar, William Matthew, Rozmus, John Michael, Palmer, Robert Gerald Jr.
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US12/491,836
• Publication Number:
  US 20090262196A1
• Time in Patent Office: 1,748 Days
• Field of Search: 348/143, 348/36, 348/335, 348/584, 348/647, 348/207.99, 348/159, 382/286, 345/647, 396/85, 356/139.03, 351/246
• US Class Current: 348/143
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method for correlating views of two or more video camera systems includes obtaining a plurality of data point coordinate sets to represent relative positioning between the camera systems and data point objects in the environment. The views may be correlated through Interpolation or extrapolation using the obtained data point coordinate sets. Devices such as lasers may also be used to correlate views of two or more video camera systems.

Claims

1. A method for correlating views of two or more video camera systems in an environment, the method comprising:

obtaining a first data point coordinate set corresponding to a first video camera system, the first data point coordinate set representing the location of a data point object in the environment relative to the first camera system, the first camera system providing an immersive view of the environment;

obtaining a second data point coordinate set corresponding to a second video camera system, the second data point coordinate set representing the location of the data point object in the environment relative to the second camera system, wherein the second camera system comprises a pan-tilt-zoom video camera having a view of the environment that is narrower than the immersive view;

storing the first and second data point sets as correlation data;

detecting a detectable reference provided by a laser on a data point object in the immersive view;

indicating a point of interest in the immersive view; and

correlating the views of the first and second camera systems by referencing the correlation data and adjusting the view of the second video camera system based on the correlation data and the detectable reference such that the detectable reference corresponds to the point of interest.

2. The method of claim 1, wherein a plurality of first and second data point coordinate sets are stored as correlation data.

3. The method of claim 2, wherein the act of correlating comprises interpolation or extrapolation using the data point coordinate sets.

4. The method of claim 1, wherein the act of adjusting comprises orienting only the second camera system.

5. The method of claim 1, wherein the act of obtaining a second data point comprises orienting the line of sight for the second camera system such that the data point object is displayed in the video image from the second camera system.

6. The method of claim 5, wherein the act of obtaining a first data point comprises selecting the data point object in the environment from the video image of the first video camera system.

7. The method of claim 1, wherein the data point object is identified using object recognition algorithms.

8. A video image capture system, comprising:

a first video camera system configured to capture an immersive video image, the first video camera system providing a wide field of view of an environment;

a second video camera system comprising a pan-tilt-zoom video camera, the pan-tilt-zoom video camera providing a narrow field of view within the wide field of view of the immersive video image;

a laser used for correlating the wide and narrow fields of view by providing an detectable reference on a data point object; and

an interface used for indicating a point of interest in the immersive video image;

wherein the wide and narrow fields of view are correlated by detecting the detectable reference in the wide field of view, and orienting the second video camera system such that the detectable reference corresponds to the point the interest.

9. The video image capture system of claim 8, wherein a plurality of first and second data point coordinate sets corresponding to the first and second video camera systems are stored as correlation data, the first and second data point coordinate sets representing a location of a data point object relative to the first and second video camera systems.

10. The video image capture system of claim 9, wherein the act of correlating comprises interpolation or extrapolation using the data point coordinate sets.

11. The video image capture system of claim 8, wherein the act of orienting comprises orienting only the second camera system.

12. The video image capture system of claim 9, wherein the second data point coordinate set is obtained by orienting the line of sight for the second camera system such that the data point object is displayed in the video image from the second camera system.

13. The video image capture system of claim 12, wherein the first data point coordinate set is obtained by selecting the data point object in the environment from the video image of the first video camera system.

14. The video image capture system of claim 8, wherein a data point object in the environment is identified using object recognition algorithms.