System and method for correlating camera views

• US 9,749,525 B2
• Filed: 11/04/2013
• Issued: 08/29/2017
• Est. Priority Date: 08/06/2004
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Inventors: Salivar, William Matthew, Rozmus, John Michael, Palmer, Robert Gerald
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US14/071,188
• Publication Number:
  US 20140055620A1
• Time in Patent Office: 1,394 Days
• Field of Search: 348143, 348 E7085, 348 E7086, 348155, 3482221, 348169, 7071041
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method for correlating views of two or more video camera systems includes obtaining a plurality of data point coordinate sets to represent relative positioning between the camera systems and data point objects in the environment. The views may be correlated through Interpolation or extrapolation using the obtained data point coordinate sets. Devices such as lasers may also be used to correlate views of two or more video camera systems.

Claims

1. A system comprising:

a first camera that provides a first image signal, wherein the first camera is mounted to a terrestrial vehicle;

a second camera that provides a second image signal, wherein the second camera is mounted to the terrestrial vehicle;

a third camera that provides a third image signal, wherein the third camera is mounted to the terrestrial vehicle;

a display; and

a processor coupled to the first, second, and third cameras and that;

generates a perspectively corrected image signal around the terrestrial vehicle by performing a perspective correction technique on the first image signal and the second image signal,

receives user input, and

based on the received user input, determines to selectively output to the display a first output image or a second output image

wherein,

the first output image having a first display configuration includes a first image around the terrestrial vehicle based on the perspectively corrected image signal and a second image based on the third image signal, wherein the first image and the second image are concurrently displayed adjacent to each other on the display, and

the second output image having a second display configuration includes a third image based on the third image signal in full screen on the display.

2. The system according to claim 1, wherein at least one of the cameras generates PTZ images or narrower images.

3. The system according to claim 1, wherein at least one of the cameras generates immersive images or wider images.

4. The system according to claim 1, wherein the cameras are physically separated.

5. The system according to claim 1, wherein a subset of the cameras is located within a housing.

6. The system according to claim 1, wherein the cameras receive commands from the processor.

7. The system according to claim 6, wherein at least one of the commands is user input.

8. The system according to claim 6, wherein at least one of the commands orients or controls a level of zoom.

9. The system according to claim 6, wherein at least one of the commands is vocal information.

10. The system according to claim 1, wherein correlation data is referenced to account for parallax when a user selects an object of interest to be viewed by at least one of the first and second cameras.

11. The system according to claim 1, wherein correlation data correlates the first image signal provided by the first camera and the second image signal provided by the second camera, the correlation data being accessible by the processor for such correlation.

12. The system according to claim 11, wherein the correlation data describes a map or model.

13. The system according to claim 1, wherein the processor correlates an image signal of one of the first and second cameras to an image signal of another of the cameras.

14. The system according to claim 1, further comprising one or more motion detectors.

15. The system according to claim 14, wherein the one or more motion detectors modify a frame rate of at least one of the first and second cameras.

16. The system according to claim 1, further comprising one or more interfaces connected to the processor.

17. The system according to claim 1, wherein at least one of the first and second cameras includes a non-wide angle lens.

18. The system according to claim 1, wherein the generated perspectively corrected image signal spans approximately 360 degrees in at least one direction and surrounds the terrestrial vehicle.

19. A system comprising:

a first camera that provides a first image signal, wherein the first camera is mounted to a terrestrial vehicle;

a second camera that provides a second image signal, wherein the second camera is mounted to the terrestrial vehicle;

a third camera that provides a third image signal, wherein the third camera is mounted to the terrestrial vehicle, wherein the first camera, the second camera, and the third camera include a CMOS image sensor;

a display; and

a processor coupled to the first, second, and third cameras and that;

generates a perspectively corrected image signal around the terrestrial vehicle to which the first camera, second camera, and third camera are mounted, by performing a perspective correction technique on the first image signal and the second image signal, and

based on user input, determines to selectively output to the display a first output image or a second output image

wherein,

the first output image having a first display configuration includes a first image around the terrestrial vehicle based on the perspectively corrected image signal and a second image based on the third image signal, wherein the first image and the second image are concurrently displayed adjacent to each other on the display, and

the second output image having a second display configuration includes a third image based on the third image signal in full screen on the display.