System and method for correlating camera views

• US 9,992,415 B2
• Filed: 07/28/2017
• Issued: 06/05/2018
• Est. Priority Date: 08/06/2004
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Inventors: Salivar, William Matthew, Rozmus, John Michael, Palmer, Robert Gerald
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US15/662,615
• Publication Number:
  US 20180054567A1
• Time in Patent Office: 312 Days
• Field of Search: 8405, 715757, 716616
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method for correlating views of two or more video camera systems includes obtaining a plurality of data point coordinate sets to represent relative positioning between the camera systems and data point objects in the environment. The views may be correlated through Interpolation or extrapolation using the obtained data point coordinate sets. Devices such as lasers may also be used to correlate views of two or more video camera systems.

Claims

1. An imaging system comprising:

a plurality of cameras coupled to a terrestrial vehicle, the plurality of cameras including a first camera configured to output a first video signal, a second camera configured to output a second video signal, and a third camera configured to output a third video signal, wherein a first video image associated with the first video signal includes a field of view of an environment of the first camera, a second video image associated with the second video signal includes a field of view of an environment of the second camera, and a third video image associated with the third video signal includes a field of view of an environment of the third camera;

at least one processor; and

a computer-readable medium encoded with instructions, which when executed by the at least one processor, configures the imaging system to;

receive user input,

based on a received first user input, process the first video signal and the second video signal to generate a first display configuration including a first perspectively corrected video image so as to display a first region of the first video image and a second region of the second video image concurrently on a display of a display system, and

based on a received second user input, process the third video signal to generate a second display configuration including a second perspectively corrected video image so as to display a third region of the third video image in full screen on the display.

2. The system according to claim 1, wherein the instructions of the computer-readable medium configure the imaging system to perform at least one of a perspective correction technique and a dewarping technique on at least one of the first and second video signals, so as to generate the first perspectively corrected image.

3. The system according to claim 1, wherein an edge of the first region of the first video image and an edge of the second region of the second video image are displayed adjacent to each other on the display of the display system.

4. The system according to claim 3, wherein the edge of the first region of the first video image and the edge of the second region of the second video image are arranged in a side-by-side configuration.

5. The system according to claim 1, further comprising one or more sensors coupled to the at least one processor and configured to detect an object in the environment, wherein the instructions of the computer-readable medium configure the imaging system to generate an alarm signal based on the detection of the object by the one or more sensors.

6. The system according to claim 1, wherein the field of view of the first video image overlaps the field of view of the second video image.

7. The system according to claim 6, wherein images of at least one of the first and second cameras have a field of view of 120 degrees or more.

8. The system according to claim 1, wherein the instructions of the computer-readable medium configure the imaging system to output to the display of the display system the first perspectively corrected video image.

9. An imaging system comprising:

a plurality of cameras coupled to a terrestrial vehicle, the plurality of cameras including a first camera configured to output a first video signal, a second camera configured to output a second video signal, and a third camera configured to output a third video signal, wherein a first video image associated with the first video signal includes a field of view of an environment of the first camera, a second video image associated with the second video signal includes a field of view of an environment of the second camera, and a third video image associated with the third video signal includes a field of view of an environment of the third camera; and

at least one processor configuring the imaging system to;

process, based on a received first user input, the first video signal and second video signal to generate a first display configuration including a first perspectively corrected video image so as to display a first region of the first video image and a second region of the second video image concurrently on a display of a display system, and

process, based on a received second user input, the third video signal to generate a second display configuration including a second perspectively corrected video image so as to display a third region of the third video image in full screen on the display.

10. The system according to claim 9, wherein the imaging system is configured to perform at least one of a perspective correction technique and a dewarping technique on at least one video image associated with the first and second video signals, so as to generate the first perspectively corrected image.

11. The system according to claim 9, wherein an edge of the first region of the first video image and an edge of the second region of the second video image are displayed adjacent to each other on the display of the display system.

12. The system according to claim 11, wherein the edge of the first region of the first video image and the edge of the second region of the second video image are arranged in a side-by-side configuration.

13. The system according to claim 9, further comprising one or more sensors coupled to the at least one processor and configured to detect an object in the environment, wherein the imaging system is configured to generate an alarm signal based on the detection of the object by the one or more sensors.

14. The system according to claim 9, wherein the field of view of the first video image overlaps the field of view of the second video image.

15. The system according to claim 14, wherein images of at least one of the first and second cameras have a field of view of 120 degrees or more.

16. The system according to claim 9, wherein the imaging system is configured to output to the display of the display system the first perspectively corrected video image.

17. An imaging system comprising:

a plurality of cameras coupled to a terrestrial vehicle, the plurality of cameras including a first camera configured to output a first video signal, a second camera configured to output a second video signal, and a third camera configured to output a third video signal, wherein a first video image associated with the first video signal includes a field of view of an environment of the first camera, a second video image associated with the second video signal includes a field of view of an environment of the second camera, and a third video image associated with the third video signal includes a field of view of an environment of the third camera;

at least one processor; and

a computer-readable medium encoded with instructions, which when executed by the at least one processor, configures the imaging system to;

based on a received first user input, process the first video signal and the second video signal to generate a first display configuration including a first perspectively corrected video image so as to display a first region of the first video image and a second region of the second video image concurrently on a display of a display system, and

based on a received second user input, process the third video signal to generate a second display configuration including a second perspectively corrected video image so as to display a third region of the third video image, wherein a size of the third region of the third video image is greater than at least one of the first region of the first video image and the second region of the second video image.

18. The system according to claim 17, wherein the computer-readable medium encoded with instructions configures the imaging system to perform at least one of a perspective correction technique and a dewarping technique on at least one of the first and second video signals, so as to generate the first perspectively corrected image.

19. The system according to claim 17, wherein an edge of the first region of the first video image and an edge of the second region of the second video image are displayed adjacent adjacent to each other on the display of the display system.

20. The system according to claim 19, wherein the edge of the first region of the first video image and the edge of the second region of the second video image are arranged in a side-by-side configuration.

21. The system according to claim 17, wherein the field of view of the first video image overlaps the field of view of the second video image.

22. The system according to claim 17, wherein images of at least one of the first and second cameras have a field of view of 120 degrees or more.

23. The system according to claim 17, wherein the instructions of the computer-readable medium configure the imaging system to output to the display of the display system the first perspectively corrected video image.

24. An imaging system comprising:

a plurality of cameras coupled to a terrestrial vehicle, the plurality of cameras including a first camera configured to output a first video signal, a second camera configured to output a second video signal, and a third camera configured to output a third video signal, wherein a first video image associated with the first video signal includes a field of view of an environment of the first camera, a second video image associated with the second video signal includes a field of view of an environment of the second camera, and a third video image associated with the third video signal includes a field of view of an environment of the third camera; and

at least one processor configuring the imaging system to;

process, based on a received first user input, the first video signal and the second video signal to generate a first display configuration including a first perspectively corrected video image so as to display a first region of the first video image and a second region of the second video image concurrently on a display of a display system, and process, based on a received second user input, the third video signal to generate a second display configuration including a second perspectively corrected video image so as to display a third region of the third video image, wherein a size of the third region of the third video image is greater than at least one of the first region of the first video image and the second region of the second video image.

25. The system according to claim 24, wherein the imaging system is configured to perform at least one of a perspective correction technique and a dewarping technique on at least one video image associated with the first and second video signals, so as to generate the first perspectively corrected image.

26. The system according to claim 24, wherein an edge of the first region of the first video image and an edge of the second region of the second video image are displayed adjacent to each other on the display of the display system.

27. The system according to claim 26, wherein the edge of the first region of the first video image and the edge of the second region of the second video image are arranged in a side-by-side configuration.

28. The system according to claim 24, wherein the field of view of the first video image overlaps the field of view of the second video image.

29. The system according to claim 28, wherein images of at least one of the first and second cameras have a field of view of 120 degrees or more.

30. The system according to claim 24, wherein the imaging system is configured to output to the display of the display system the first perspectively corrected video image.