Tracking system, terminal device, camera device, tracking imaging method, and program

• US 10,192,318 B2
• Filed: 08/01/2017
• Issued: 01/29/2019
• Est. Priority Date: 02/02/2015
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Hayashi, Daisuke
• Primary Examiner(s): Williams, Jeffery A
• Application Number:
  US15/666,019
• Publication Number:
  US 20170330335A1
• Time in Patent Office: 546 Days
• Field of Search: 348169
• CPC Class Codes: G03B 17/561 Support related camera accessories
• CPC Class Codes: G03B 2206/00 Systems for exchange of information between different pieces of apparatus, e.g. for exchanging trimming information, for photo finishing
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

In a preferred aspect of the present invention, at least one of a camera-side controller or a terminal-side controller performs a tracking image generation process (P1) of generating tracking image data from captured image data. Further, at least one of the camera-side controller or the terminal-side controller performs a tracking calculation process (P2) of acquiring target information on the basis of the tracking image data. The tracking image data includes a high-resolution region with relatively higher resolution and a low-resolution region with relatively lower resolution. In the tracking image generation process (P1), basic tracking information including information on at least one of a size or a speed of the tracking target is acquired, and a high-resolution region and a low-resolution region are determined on the basis of the basic tracking information. The tracking image generation process (P1) and the tracking calculation process (P2) are performed in parallel.

Claims

1. A tracking system, comprising:

a camera device including an imaging unit that acquires captured image data, an imaging direction adjustment unit that adjusts an imaging direction of the imaging unit, a camera-side controller that controls the imaging direction adjustment unit on the basis of target information indicating a position of a tracking target, and a camera-side communication unit that is connected to the camera-side controller; and

a terminal device including a terminal-side communication unit that is able to communicate with the camera-side communication unit, a terminal-side controller that is connected to the terminal-side communication unit, and a user interface that is connected to the terminal-side controller,

wherein at least one of the camera-side controller or the terminal-side controller performs a tracking image generation process of generating tracking image data from the captured image data,

at least one of the camera-side controller or the terminal-side controller performs a tracking calculation process of acquiring the target information on the basis of the tracking image data,

the tracking image data is data of an image including a high-resolution region with relatively higher resolution and a low-resolution region with relatively lower resolution,

in the tracking image generation process, basic tracking information including information on at least one of a size or a speed of the tracking target is acquired, the high-resolution region and the low-resolution region are determined on the basis of the basic tracking information, and the tracking image data is generated, and

the tracking image generation process and the tracking calculation process are performed in parallel.

2. The tracking system according to claim 1,

wherein at least a portion of the tracking calculation process of acquiring the target information indicating the position of the tracking target at a first timing on the basis of the tracking image data generated from the captured image data acquired at the first timing is performed simultaneously with at least a portion of the tracking image generation process of generating the tracking image data that is used to acquire the target information indicating the position of the tracking target at a second timing after the first timing, from the captured image data acquired at the second timing.

3. The tracking system according to claim 1,

wherein the tracking image generation process and the tracking calculation process are performed in one of the camera-side controller and the terminal-side controller.

4. The tracking system according to claim 1,

wherein one of the tracking image generation process and the tracking calculation process is performed in one of the camera-side controller and the terminal-side controller, and

the other of the tracking image generation process and the tracking calculation process is performed in the other of the camera-side controller and the terminal-side controller.

5. The tracking system according to claim 1,

wherein the tracking calculation process is performed by both of the camera-side controller and the terminal-side controller, and

a controller having a relatively higher processing capability between the camera-side controller and the terminal-side controller performs the tracking calculation process of the high-resolution region, and a controller having a relatively lower processing capability between the camera-side controller and the terminal-side controller performs the tracking calculation process of the low-resolution region.

6. The tracking system according to claim 1,

wherein at least one of the camera-side controller or the terminal-side controller performs a pre-search process of acquiring a spatial frequency of the captured image data of at least a portion of the imaging range of the camera device, and

in the tracking image generation process, resolution of the tracking image data is determined on the basis of the spatial frequency acquired through the pre-search process.

7. The tracking system according to claim 6,

wherein in the tracking image generation process, in a case where the spatial frequency acquired through the pre-search process is relatively low, resolution of the tracking image data is set to be lower than in a case where the spatial frequency is relatively high.

8. A terminal device that is connectable to a camera device including an imaging unit that acquires captured image data, an imaging direction adjustment unit that adjusts an imaging direction of the imaging unit, a camera-side controller that controls the imaging direction adjustment unit on the basis of target information indicating a position of a tracking target, and a camera-side communication unit that is connected to the camera-side controller, the terminal device comprising:

a terminal-side communication unit that is able to communicate with the camera-side communication unit;

a terminal-side controller that is connected to the terminal-side communication unit; and

a user interface connected to the terminal-side controller,

wherein at least one of the camera-side controller or the terminal-side controller performs a tracking image generation process of generating tracking image data from the captured image data,

the terminal-side controller performs a tracking calculation process of acquiring the target information on the basis of the tracking image data, and transmits the target information to the camera-side controller via the terminal-side communication unit and the camera-side communication unit,

the tracking image data is data of an image including a high-resolution region with relatively higher resolution and a low-resolution region with relatively lower resolution,

in the tracking image generation process, basic tracking information including information on at least one of a size or a speed of the tracking target is acquired, the high-resolution region and the low-resolution region are determined on the basis of the basic tracking information, and the tracking image data is generated, and

the tracking image generation process and the tracking calculation process are performed in parallel.

9. A camera device connectable to a terminal device including a terminal-side communication unit, a terminal-side controller that is connected to the terminal-side communication unit, and a user interface that is connected to the terminal-side controller, the camera device comprising:

an imaging unit that acquires the captured image data;

an imaging direction adjustment unit that adjusts an imaging direction of the imaging unit; and

a camera-side controller that controls the imaging direction adjustment unit on the basis of target information indicating a position of the tracking target, the camera-side controller performing a tracking image generation process of generating tracking image data from the captured image data and a tracking calculation process of acquiring the target information on the basis of the tracking image data in parallel,

wherein the tracking image data is data of an image including a high-resolution region with relatively higher resolution and a low-resolution region with relatively lower resolution, and

in the tracking image generation process, basic tracking information including information on at least one of a size or a speed of the tracking target is acquired, the high-resolution region and the low-resolution region are determined on the basis of the basic tracking information, and the tracking image data is generated.

10. A tracking imaging method of a tracking system comprising a camera device including an imaging unit that acquires captured image data, an imaging direction adjustment unit that adjusts an imaging direction of the imaging unit, a camera-side controller that controls the imaging direction adjustment unit on the basis of target information indicating a position of a tracking target, and a camera-side communication unit that is connected to the camera-side controller, and a terminal device including a terminal-side communication unit that is able to communicate with the camera-side communication unit, a terminal-side controller that is connected to the terminal-side communication unit, and a user interface that is connected to the terminal-side controller, the tracking imaging method comprising:

a step of performing a tracking image generation process of generating tracking image data from the captured image data by at least one of the camera-side controller or the terminal-side controller; and

a step of performing a tracking calculation process of acquiring the target information on the basis of the tracking image data by at least one of the camera-side controller or the terminal-side controller,

wherein the tracking image data is data of an image including a high-resolution region with relatively higher resolution and a low-resolution region with relatively lower resolution,

in the tracking image generation process, basic tracking information including information on at least one of a size or a speed of the tracking target is acquired, the high-resolution region and the low-resolution region are determined on the basis of the basic tracking information, and the tracking image data is generated, and

the tracking image generation process and the tracking calculation process are performed in parallel.

11. A computer-readable non-transitory tangible medium having a program recorded thereon, the program for controlling a tracking system comprising a camera device including an imaging unit that acquires captured image data, an imaging direction adjustment unit that adjusts an imaging direction of the imaging unit, a camera-side controller that controls the imaging direction adjustment unit on the basis of target information indicating a position of a tracking target, and a camera-side communication unit that is connected to the camera-side controller, and a terminal device including a terminal-side communication unit that is able to communicate with the camera-side communication unit, a terminal-side controller that is connected to the terminal-side communication unit, and a user interface that is connected to the terminal-side controller, the program causing a computer to execute:

a step of performing a tracking image generation process of generating tracking image data from the captured image data by at least one of the camera-side controller or the terminal-side controller; and

a step of performing a tracking calculation process of acquiring the target information on the basis of the tracking image data by at least one of the camera-side controller or the terminal-side controller,

wherein the tracking image data is data of an image including a high-resolution region with relatively higher resolution and a low-resolution region with relatively lower resolution,

in the tracking image generation process, basic tracking information including information on at least one of a size or a speed of the tracking target is acquired, the high-resolution region and the low-resolution region are determined on the basis of the basic tracking information, and the tracking image data is generated, and the tracking image generation process and the tracking calculation process are performed in parallel.