Surveillance information generation apparatus, imaging direction estimation apparatus, surveillance information generation method, imaging direction estimation method, and program

• US 10,579,881 B2
• Filed: 05/08/2019
• Issued: 03/03/2020
• Est. Priority Date: 09/01/2015
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Oami, Ryoma
• Primary Examiner(s): Gilliard, Delomia L
• Application Number:
  US16/406,705
• Publication Number:
  US 20190318172A1
• Time in Patent Office: 300 Days
• Field of Search: None
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A surveillance information generation apparatus (2000) includes a first surveillance image acquisition unit (2020), a second surveillance image acquisition unit (2040), and a generation unit (2060). The first surveillance image acquisition unit (2020) acquires a first surveillance image (12) generated by a fixed camera (10). The second surveillance image acquisition unit (2040) acquires a second surveillance image (22) generated by a moving camera (20). The generation unit (2060) generates surveillance information (30) relating to object surveillance, using the first surveillance image (12) and first surveillance information (14).

Claims

1. An imaging direction estimation system comprising:

at least one memory storing instructions; and

at least one processor coupled to the at least one memory, the at least one processor being configured to execute the instructions to;

estimate a real observed moving direction of an object in a captured image acquired by a camera whose position is not fixed;

estimate a real position and a real moving direction of the object in a real world in a timing when the captured image was acquired, based on information other than the captured image;

select, from among a plurality of candidate directions which are candidates for an imaging direction of the camera, at least two candidate directions which are regarded as the imaging direction in the timing when the captured image was acquired, based on at least one of information acquired by an electronic compass and a background included in the captured image;

estimate, for each candidate direction in the at least two candidate directions, a candidate moving direction which is a direction regarded as an observed moving direction of the object under presumption that the object moving in the real moving direction is observed by the camera facing the candidate direction, based on a position of the camera and the estimated real moving direction; and

determine one candidate direction, among the at least two candidate directions, to be the imaging direction of the camera through comparing each estimated candidate moving direction and the real observed moving direction.

2. The imaging direction estimation system according to claim 1, wherein the at least one processor is further configured to:

exclude a predetermined number of directions based on difference between the plurality of directions and a direction indicated by the electronic compass; and

select directions which have not been excluded, as the at least two candidate directions.

3. The imaging direction estimation system according to claim 1, wherein the at least one processor is further configured to:

exclude a predetermined number of directions based on position relation between the position of the camera and the background; and

select directions which have not been excluded, as the at least two candidate directions.

4. The imaging direction estimation system according to claim 1, wherein the at least one processor is further configured to:

determine the imaging direction a plurality of times by using a plurality of captured images acquired by the camera and acquire imaging directions;

specify one imaging direction based on the acquired imaging directions; and

output the specified one imaging direction as a comprehensive estimation result.

5. The imaging direction estimation system according to claim 4, wherein the at least one processor is further configured to:

divide the acquired imaging directions into a plurality of groups; and

specify the one imaging direction by using only imaging directions included in one or more groups in which variance of the imaging directions in the one or more groups is equal to or less than a predetermined value.

6. The imaging direction estimation system according to claim 1, wherein the at least one processor is further configured to:

determine a first plurality of candidate directions by using a captured image of the object moving along a first direction;

determine a second plurality of candidate directions by using a captured image of the object moving along a second direction; and

determine one candidate direction to be the imaging direction of the camera based on the first plurality of candidate directions and the second plurality of candidate directions.

7. The imaging direction estimation system according to claim 1, wherein the at least one processor is further configured to:

determine the imaging direction of the camera by the comparing at intervals; and

estimate the imaging direction based on change in direction of the camera in the intervals.

8. An imaging direction estimation method comprising:

estimating a real observed moving direction of an object in a captured image acquired by a camera whose, position is not fixed;

estimating a real position and a real moving direction of the object in a real world in a timing when the captured image was acquired, based on information other than the captured image;

selecting, from among a plurality of candidate directions which are candidates for an imaging direction of the camera, at least two candidate directions which are regarded as the imaging direction in the timing when the captured image was acquired, based on at least one of information acquired by an electronic compass and a background included in the captured image;

estimating, for each candidate direction in the at least two candidate directions, a candidate moving direction which is a direction regarded as an observed moving direction of the object under presumption that the object moving in the real moving direction is observed by the camera facing the candidate direction, based on a position of the camera and the estimated real moving direction; and

determining one candidate direction to be the imaging direction of the camera through comparing each estimated candidate moving direction and the real observed moving direction.

9. The imaging direction estimation method according to claim 8, further comprising:

excluding a predetermined number of directions based on difference between the plurality of directions and a direction indicated by the electronic compass; and

selecting directions which have not been excluded, as the at least two candidate directions.

10. The imaging direction estimation method according to claim 8, further comprising:

excluding a predetermined number of directions based on position relation between the position of the camera and the background; and

selecting directions which have not been excluded, as the at least two candidate directions.

11. The imaging direction estimation method according to claim 8, further comprising:

determining the imaging direction a plurality of times by using a plurality of captured images acquired by the camera and acquiring imaging directions;

specifying one imaging direction based on the acquired imaging directions; and

outputting the specified one imaging direction as a comprehensive estimation result.

12. The imaging direction estimation method according to claim 11, further comprising:

dividing the acquired imaging directions into a plurality of groups; and

specifying the one imaging direction by using only imaging directions included in one or more groups in which variance of the imaging directions in the one or more groups is equal to or less than a predetermined value.

13. The imaging direction estimation method according to claim 8, further comprising:

determining a first plurality of candidate directions by using a captured image of the object moving along a first direction;

determining a second plurality of candidate directions by using a captured image of the object moving along a second direction; and

determining one candidate direction to be the imaging direction of the camera based on the first plurality of candidate directions and the second plurality of candidate directions.

14. The imaging direction estimation method according to claim 8, further comprising:

determining the imaging direction of the camera by the comparing at intervals; and

estimating the imaging direction based on change in direction of the camera in the intervals.

15. A non-transitory computer-readable storage medium storing a program that causes a computer to perform:

estimating a real observed moving direction of an object in a captured image acquired by a camera whose, position is not fixed;

estimating a real position and a real moving direction of the object in a real world in a timing when the captured image was acquired, based on information other than the captured image;

selecting, from among a plurality of candidate directions which are candidates for an imaging direction of the camera, at least two candidate directions which are regarded as the imaging direction in the timing when the captured image was acquired, based on at least one of information acquired by an electronic compass and a background included in the captured image;

estimating, for each candidate direction in the at least two candidate directions, a candidate moving direction which is a direction regarded as an observed moving direction of the object under presumption that the object moving in the real moving direction is observed by the camera facing the candidate direction, based on a position of the camera and the estimated real moving direction; and

determining one candidate direction to be the imaging direction of the camera through comparing each estimated candidate moving direction and the real observed moving direction.

16. The storage medium according to claim 15, wherein the program further causes the computer to perform:

excluding a predetermined number of directions based on difference between the plurality of directions and a direction indicated by the electronic compass; and

selecting directions which have not been excluded, as the at least two candidate directions.

17. The storage medium according to claim 15, wherein the program further causes the computer to perform:

excluding a predetermined number of directions based on position relation between the position of the camera and the background; and

selecting directions which have not been excluded, as the at least two candidate directions.

18. The storage medium according to claim 15, wherein the program further causes the computer to perform:

determining the imaging direction a plurality of times by using a plurality of captured images acquired by the camera and acquiring imaging directions;

specifying one imaging direction based on the acquired imaging directions; and

outputting the specified one imaging direction as a comprehensive estimation result.

19. The storage medium according to claim 18, wherein the program further causes the computer to perform:

dividing the acquired imaging directions into a plurality of groups; and

specifying the one imaging direction by using only imaging directions included in one or more groups in which variance of the imaging directions in the one or more groups is equal to or less than a predetermined value.

20. The storage medium according to claim 15, wherein the program further causes the computer to perform:

determining a first plurality of candidate directions by using a captured image of the object moving along a first direction;

determining a second plurality of candidate directions by using a captured image of the object moving along a second direction; and

determining one candidate direction to be the imaging direction of the camera based on the first plurality of candidate directions and the second plurality of candidate directions.