Imaging system, angle-of-view adjustment method, and angle-of-view adjustment program

• US 10,356,301 B2
• Filed: 03/27/2018
• Issued: 07/16/2019
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Oshima, Hiroyuki
• Primary Examiner(s): Haskins, Twyler L
• Assistant Examiner(s): Garces-Rivera, Angel L
• Application Number:
  US15/936,836
• Publication Number:
  US 20180295270A1
• Time in Patent Office: 476 Days
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/662 by using master/slave camera arrangements for affecting the control of camera image capture, e.g. placing the camera in a desirable condition to capture a desired image
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An embodiment of the invention provides an imaging system (10) in which captured data obtained by an imaging unit (22) of a pan/tilt camera (14) provided in a moving body (12) is transmitted and a captured image is displayed on a display screen (200). The imaging system (10) includes a coordinate designation unit (46) that designates designation coordinates and movement destination coordinates as a command to adjust an angle of view, an angle-of-view adjustment period calculation unit (50) that calculates pan/tilt operation parameters, using the designation coordinates and the movement destination coordinates, a movement state calculation unit (52) that calculates a movement state of the imaging unit from the captured data, and a pan/tilt operation parameter correction unit (54) that corrects the pan/tilt operation parameter on the basis of the movement state of the imaging unit.

Claims

1. An imaging system comprising:

a moving body;

an imaging unit that is provided in the moving body;

a captured data transmission unit that transmits captured data obtained by an imaging operation of the imaging unit;

a captured data receiving unit that receives the captured data transmitted from the captured data transmission unit;

a pan/tilt operation unit that instructs the imaging unit to perform at least one of a pan operation or a tilt operation;

an imaging operation unit that is connected to the imaging unit so as to communicate with the imaging unit and transmits an imaging control signal for operating the imaging unit to the imaging unit;

a display unit that displays a captured image indicated by the captured data received by the captured data receiving unit on a display screen;

a coordinate designation unit that designates designation coordinates which are coordinates of a movement target and movement destination coordinates which are coordinates of a movement destination of the movement target in the image displayed on the display screen of the display unit in a case in which an angle of view of the imaging unit is adjusted;

an angle-of-view adjustment operation unit that calculates pan/tilt operation parameters applied to the pan/tilt operation unit, using the designation coordinates and the movement destination coordinates;

an angle-of-view adjustment period calculation unit that calculates an angle-of-view adjustment period which is a period from the designation of the designation coordinates to the adjustment of the angle of view of the imaging unit;

a movement state calculation unit that calculates motion parameters indicating a moving direction and a movement distance in a case in which the imaging unit is moved by the movement of the moving body for the angle-of-view adjustment period from the captured data obtained by the imaging operation of the imaging unit;

a pan/tilt operation parameter correction unit that corrects the pan/tilt operation parameters calculated by the angle-of-view adjustment operation unit on the basis of the motion parameters calculated by the movement state calculation unit and the angle-of-view adjustment period calculated by the angle-of-view adjustment period calculation unit in a case in which the angle of view of the imaging unit is adjusted; and

a pan/tilt operation command unit that transmits a signal corresponding to the pan/tilt operation parameters to the pan/tilt operation unit, using communication, and transmits a signal corresponding to the pan/tilt operation parameters corrected by the pan/tilt operation parameter correction unit to the pan/tilt operation unit.

2. The imaging system according to claim 1,

wherein the angle-of-view adjustment operation unit calculates a first motion vector as the pan/tilt operation parameters from the designation coordinates and the movement destination coordinates,

the movement state calculation unit calculates a correction vector as the motion parameters of the imaging unit during the angle-of-view adjustment period, and

the pan/tilt operation parameter correction unit corrects the pan/tilt operation parameters indicated by the first motion vector, using the correction vector.

3. The imaging system according to claim 2,

wherein the movement state calculation unit calculates, as the correction vector, a velocity vector indicating a velocity of the imaging unit in a case in which the imaging unit is moved by the movement of the moving body for the angle-of-view adjustment period and an acceleration vector indicating acceleration of the imaging unit in a case in which the imaging unit is moved by the movement of the moving body for the angle-of-view adjustment period.

4. The imaging system according to claim 3,

wherein the movement state calculation unit acquires the first motion vector indicating a movement distance and a moving direction of the imaging unit for a first period between a first time and a second time before the first time, from first captured data which is acquired by the imaging operation of the imaging unit at the first time and second captured data which is acquired by the imaging operation of the imaging unit at the second time, and

the movement state calculation unit calculates a first velocity vector that has a magnitude obtained by dividing a magnitude of the first motion vector by the first period and has a direction parallel to a direction indicated by the first motion vector.

5. The imaging system according to claim 4,

wherein the movement state calculation unit uses feature point detection to acquire the first motion vector.

6. The imaging system according to claim 4,

wherein the movement state calculation unit acquires a second motion vector indicating a movement distance and a moving direction of the imaging unit for a second period between the second time and a third time before the second time, from the second captured data and third captured data which is acquired by the imaging operation of the imaging unit at the third time,

the movement state calculation unit calculates a second velocity vector that has a magnitude obtained by dividing a magnitude of the second motion vector by the second period and has a direction parallel to a direction indicated by the second motion vector, and

the movement state calculation unit calculates an acceleration vector that has a magnitude obtained by dividing a magnitude of a difference vector, which is obtained by subtracting the second velocity vector from the first velocity vector, by the first period and has a direction parallel to the difference vector.

7. The imaging system according to claim 4,

wherein the movement state calculation unit uses the latest captured data acquired by the imaging operation of the imaging unit as the first captured data.

8. The imaging system according to claim 1,

wherein the angle-of-view adjustment operation unit calculates, as the pan/tilt operation parameters, an operating speed of the pan/tilt operation unit from a predetermined pan/tilt operation period from the start of a pan/tilt operation to the completion of the pan/tilt operation, the designation coordinates, and the movement destination coordinates.

9. The imaging system according to claim 8, further comprising:

an angle-of-view information acquisition unit that acquires information about the angle of view of a lens included in the imaging unit,

wherein the angle-of-view adjustment operation unit calculates the pan/tilt operation period, using a maximum operating speed of the pan/tilt operation unit and the angle of view of the lens included in the imaging unit.

10. The imaging system according to claim 9, further comprising:

a coordinate conversion processing unit that performs a polar coordinate conversion process of converting the designation coordinates and the movement destination coordinates represented by a two-dimensional coordinate system into three-dimensional coordinates, using the information about the angle of view of the imaging unit acquired by the angle-of-view information acquisition unit,

wherein the movement state calculation unit calculates, as the operation parameters of the pan/tilt operation unit, displacement angles of the designation coordinates and the movement destination coordinates in the three-dimensional coordinate system converted by the coordinate conversion processing unit.

11. The imaging system according to claim 1, further comprising:

a captured data communication period acquisition unit that acquires a captured data communication period which is a communication period of the captured data acquired by the imaging operation of the imaging unit;

an operation command communication period acquisition unit that acquires an operation command communication period which is a communication period of the operation command transmitted from the pan/tilt operation command unit; and

a pan/tilt operation period acquisition unit that acquires a pan/tilt operation period from the acquisition of the operation command by the pan/tilt operation unit to the completion of the operation of the pan/tilt operation unit,

wherein the angle-of-view adjustment period calculation unit adds the captured data communication period acquired by the captured data communication period acquisition unit, the operation command communication period acquired by the operation command communication period acquisition unit, and the pan/tilt operation period acquired by the pan/tilt operation period acquisition unit to calculate the angle-of-view adjustment period.

12. The imaging system according to claim 11, further comprising:

a communication period measurement unit that measures a period from a time when a captured data transmission request is transmitted to the imaging unit to a time when the captured data transmitted from the imaging unit is received,

wherein the captured data communication period acquisition unit acquires the period measured by the communication period measurement unit as the captured data communication period.

13. The imaging system according to claim 12,

wherein the captured data communication period acquisition unit acquires an average of a plurality of measurement results of the communication period measurement unit as the captured data communication period.

14. The imaging system according to claim 1, further comprising:

a moving body remote control unit that remotely controls the moving body and is provided independently of the imaging operation unit.

15. The imaging system according to claim 1,

wherein the angle-of-view adjustment operation unit uses coordinates of a central position of the display screen as the movement destination coordinates.

16. A method for adjusting an angle of view of an imaging unit that is provided in a moving body and is capable of performing at least one of a pan operation or a tilt operation, the method comprising:

a captured data transmission step of transmitting captured data obtained by an imaging operation of the imaging unit;

a captured data receiving step of receiving the captured data transmitted in the captured data transmission step;

an imaging operation step of transmitting an imaging control signal for operating the imaging unit to the imaging unit, using communication;

a display step of displaying a captured image indicated by the captured data received in the captured data receiving step on a display screen;

a coordinate designation step of designating designation coordinates which are coordinates of a movement target and movement destination coordinates which are coordinates of a movement destination of the movement target in the image displayed on the display screen in a case in which the angle of view of the imaging unit is adjusted;

an angle-of-view adjustment operation step of calculating pan/tilt operation parameters, using the designation coordinates and the movement destination coordinates;

an angle-of-view adjustment period calculation step of calculating an angle-of-view adjustment period which is a period from the designation of the designation coordinates to the adjustment of the angle of view of the imaging unit;

a movement state calculation step of calculating motion parameters indicating a moving direction and a movement distance in a case in which the imaging unit is moved by the movement of the moving body for the angle-of-view adjustment period from the captured data obtained by the imaging operation of the imaging unit;

a pan/tilt operation parameter correction step of correcting the pan/tilt operation parameters calculated in the angle-of-view adjustment operation step on the basis of the motion parameters calculated in the movement state calculation step and the angle-of-view adjustment period calculated in the angle-of-view adjustment period calculation step in a case in which the angle of view of the imaging unit is adjusted; and

a pan/tilt operation command step of instructing the imaging unit to perform at least one of the pan operation or the tilt operation on the basis of the pan/tilt operation parameters corrected in the pan/tilt operation parameter correction step.

17. A computer-readable non-transitory tangible medium storing an angle-of-view adjustment program that causes a computer to function as:

imaging means provided in a moving body;

captured data transmission means for transmitting captured data obtained by an imaging operation of the imaging means;

captured data receiving means for receiving the captured data transmitted from the captured data transmission means;

pan/tilt operation means for instructing the imaging means to perform at least one of a pan operation or a tilt operation;

imaging operation means that is connected to the imaging means so as to communicate with the imaging means and transmits an imaging control signal for operating the imaging means to the imaging means;

display means for displaying a captured image indicated by the captured data received by the captured data receiving means on a display screen;

coordinate designation means for designating designation coordinates which are coordinates of a movement target and movement destination coordinates which are coordinates of a movement destination of the movement target in the image displayed on the display screen of the display means in a case in which an angle of view of the imaging means is adjusted;

angle-of-view adjustment operation means for calculating pan/tilt operation parameters applied to the pan/tilt operation means, using the designation coordinates and the movement destination coordinates;

angle-of-view adjustment period calculation means for calculating an angle-of-view adjustment period which is a period from the designation of the designation coordinates to the adjustment of the angle of view of the imaging means;

movement state calculation means for calculating motion parameters indicating a moving direction and a movement distance in a case in which the imaging means is moved by the movement of the moving body for the angle-of-view adjustment period from the captured data obtained by the imaging operation of the imaging means;

pan/tilt operation parameter correction means for correcting the pan/tilt operation parameters calculated by the angle-of-view adjustment operation means on the basis of the motion parameters calculated by the movement state calculation means and the angle-of-view adjustment period calculated by the angle-of-view adjustment period calculation means in a case in which the angle of view of the imaging means is adjusted; and

pan/tilt operation command means for transmitting a signal corresponding to the pan/tilt operation parameters to the pan/tilt operation means, using communication, and transmitting a signal corresponding to the pan/tilt operation parameters corrected by the pan/tilt operation parameter correction means to the pan/tilt operation means.