Monitoring apparatus and system using 3D information of images and monitoring method using the same

• US 9,898,829 B2
• Filed: 09/17/2013
• Issued: 02/20/2018
• Est. Priority Date: 09/18/2012
• Status: Active Grant

• Current Assignee: Hanwha Vision Company Limited (Hanwha Corporation)
• Original Assignee: Hanwha Techwin Co., Ltd. (Hanwha Corporation)
• Inventors: Kim, Chang Soo
• Primary Examiner(s): Anderson, II, James M
• Application Number:
  US14/029,074
• Publication Number:
  US 20140078263A1
• Time in Patent Office: 1,617 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A monitoring apparatus using three-dimensional (3D) information of images includes: an image acquisition unit to acquire a two-dimensional (2D) image from a pan/tilt/zoom (PTZ) camera; an information extraction unit to extract 2D coordinate information of an object based on a pan/tilt angle of the PTZ camera and extract distance information between the PTZ camera and the object; an operation unit to calculate at least one of variation of the 2D coordinate information and the distance information by comparing a current frame and a previous frame of the 2D image, and variation of the 2D coordinate information and height information of the object by comparing a current frame and a previous frame of a 3D image of the object; and a position tracking unit to track a position of the object by controlling the PTZ camera based on the at least one of the two variations.

Claims

1. A monitoring apparatus comprising at least one processor to implement:

an image acquisition unit configured to acquire a two-dimensional (2D) image of an object which is captured by a pan/tilt/zoom (PTZ) camera;

an information extraction unit configured to extract distance information between the PTZ camera and the object based on a zoom scale of the PTZ camera;

an operation unit configured to calculate variation of the distance information between different frames of the 2D image, and variation of height information of the object between different frames of a 3D image of the object, the height information indicating a height of the object in a 3D space represented by the 3D image; and

a position tracking unit configured to determine a current position of the object based on the two variations.

2. The monitoring apparatus of claim 1, wherein a plurality of PTZ cameras including the PTZ camera are arranged to be fixed to predetermined positions.

3. The monitoring apparatus of claim 2, further comprising an interface unit configured to connect the plurality of PTZ cameras using an Internet protocol.

4. The monitoring apparatus of claim 3, further comprising an image conversion unit configured to synthesize the 2D image, which is captured by the PTZ camera and acquired by the image acquisition unit, and at least one another 2D image captured by at least one another PTZ camera among the plurality of PTZ cameras, and convert the synthesized 2D images into the 3D image.

5. The monitoring apparatus of claim 4, wherein the information extraction unit extracts the height information of the object from the 3D image.

6. The monitoring apparatus of claim 5, wherein the operation unit is configured to calculate both of the two variations, and

wherein the position tracking unit determines the current position of the object based on both of the two variations.

7. The monitoring apparatus of claim 1, wherein the information extraction unit extracts the distance information based on a focus length according to the zoom scale of the PTZ camera.

8. A monitoring system comprising a plurality of PTZ cameras, wherein each of the PTZ cameras comprises the monitoring apparatus of claim 1.

9. The monitoring system of claim 8, wherein each of the PTZ cameras comprises an image conversion unit configured to synthesize the 2D image, which is captured by the PTZ camera and acquired by the image acquisition unit, and at least one another 2D image captured by at least one another PTZ camera among the plurality of PTZ cameras, and convert the synthesized 2D images into the 3D image.

10. The monitoring system of claim 9, wherein the information extraction unit extracts the height information of the object from the 3D image.

11. The monitoring system of claim 10, wherein the operation unit is configured to calculate both of the two variations, and

wherein the position tracking unit determines the current position of the object by controlling of the plurality of PTZ cameras based on both of the variations.

12. The monitoring system of claim 9, wherein the information extraction unit extracts the distance information based on a focus length according to the zoom scale of the PTZ camera.

13. The monitoring system of claim 9, wherein each of the PTZ cameras further comprises a storage unit configured to store the height information and the distance information.

14. The monitoring system of claim 9, wherein each of the PTZ cameras transfers the 3D image converted by the image conversion unit, the height information and the distance information, which are extracted by the information extraction unit, to at least one another PTZ camera among the plurality of PTZ cameras through the network.

15. A monitoring method comprising:

acquiring a two-dimensional (2D) image captured by a pan/tilt/zoom (PTZ) camera;

extracting distance information between the PTZ camera and the object based on a zoom scale of the PTZ camera;

calculating variation of the distance information between different frames of the 2D image, and variation of height information of the object between different frames of a 3D image of the object, the height information indicating a height of the object in a 3D space represented by the 3D image; and

determining a current position of the object based on the two variations.

16. The monitoring method of claim 15, further comprising:

synthesizing the 2D image and at least one another 2D image captured by at least one another PTZ camera; and

converting the synthesized 2D images into the 3D image.