ROBOT VISION SYSTEM AND DETECTION METHOD

• US 20090141938A1
• Filed: 11/05/2008
• Published: 06/04/2009
• Est. Priority Date: 11/08/2007
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Hwang, Dae Hwan, Cho, Jae II, Lim, Eul Gyoon
• Granted Patent: US 8,331,617 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20052 Discrete cosine transform [DCT]
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A robot vision system for outputting a disparity map includes a stereo camera for receiving left and right images and outputting a disparity map between the two images; an encoder for encoding either the left image or the right image into a motion compensation-based video bit-stream; and a decoder for extracting an encoding type of an image block, a motion vector, and a DCT coefficient from the video bit-stream. Further, the system includes a person detector for detecting and labeling person blocks in the image using the disparity map between the left image and the right image, the block encoding type, and the motion vector, and detecting a distance from the labeled person to the camera; and an obstacle detector for detecting a closer obstacle than the person using the block encoding type, the motion vector, and the DCT coefficient extracted from the video bit-stream, and the disparity map.

Claims

1. A robot vision system for outputting a disparity map, the system comprising:

a stereo camera for receiving left and right images and outputting a disparity map between the two images;

an encoder for encoding either the left image or the right image into a motion compensation-based video bit-stream;

a decoder for extracting an encoding type of an image block, a motion vector, and a Discrete Cosine Transform (DCT) coefficient from the video bit-stream;

a person detector for detecting and labeling person blocks in the image using the disparity map between the left image and the right image, the block encoding type, and the motion vector, and detecting a distance from the labeled person to the camera;

an obstacle detector for detecting a closer obstacle than the person using the block encoding type, the motion vector, and the DCT coefficient extracted from the video bit-stream, and the disparity map.

2. The system of claim 1, further comprising:

a viewpoint motion detector for detecting a change in the motion vector corresponding to a change in an origin and a direction of the camera caused by a motion of the movable robot or camera pan tilt;

a tracking unit for estimating a location of a instantaneously stationary person, based on the detected change in the motion vector; and

a viewpoint motion pre-processor for performing viewpoint motion amount-based correction by analyzing the detected change in the motion vector, and for, when the change is a pan or tilting, averaging motion vectors in a direction determined by both boundary blocks and the location of the stationary person estimated by the tracking unit, and subtracting an average value from all the motion vectors.

3. The system of claim 2, wherein the viewpoint motion detector obtains a panning direction of the camera from the direction of the decoded motion vector, and obtains a tilt direction of the camera from the direction of the decoded motion vector and a type of blocks at upper and lower ends.

4. The system of claim 3, wherein the panning direction and the tilt direction are used for detecting the change in the motion vector corresponding to the change in the origin and direction of the camera.

5. The system of claim 2, wherein the viewpoint motion detector detects the change in the motion vector corresponding to the change in the origin and the direction of the camera using motion information and camera pan tilt information of the robot input through a variation of the origin and the viewpoint of the camera between a previous image and a current image from a main processor in the movable robot.

6. The system of claim 2, wherein the image reconfiguration unit receives one original image from the stereo camera, magnifies or reduces the image into different images in a predetermined magnification, and reconfigures the resultant image into an image having a resolution for encoding.

7. The system of claim 1, wherein the encoding uses any one of MPEG4, MPEG2, H.264, and H263.

8. The system of claim 1, further comprising an image reconfiguration unit for reconfiguring a range of the motion vector.

9. A detection method in a robot vision system, comprising:

reconfiguring left and right images;

receiving the reconfigured image and encoding the same into a motion compensation-based video bit-stream;

partially decoding the video bit-stream to extract an encoding type of an image block, a motion vector, and a DCT coefficient from the video bit-stream;

detecting and labeling person blocks in the image using a disparity map between the left image and the right image, the block encoding type, and the motion vector, and detecting a distance from the labeled person to the camera; and

detecting a closer obstacle than the person using the block encoding type, the motion vector, and the DCT coefficient extracted from the video bit-stream, and the disparity map.

10. The method of claim 9, further comprising

estimating a motion vector location of a stationary person;

detecting a change in the motion vector corresponding to a change in an origin and a direction of the camera caused by a motion of the movable robot or camera pan tilt; and

performing viewpoint motion amount-based correction by analyzing the detected change in the motion vector, and for, when the change is a rotation, averaging motion vectors in a direction determined by both boundary blocks, and subtracting an average value from all the motion vectors.

11. The method of claim 10, wherein the averaging of motion vectors comprises excluding the estimated motion vector of the stationary person.

12. A detection method in a robot vision system, comprising:

photographing left and right images and calculating a disparity map between left and right images using the two images;

reconfiguring one of the left and right images into an image having a resolution for encoding;

receiving the reconfigured image and encoding the same into a video bit-stream;

decoding to extract an encoding type, a motion vector, and a DCT coefficient of the image from the video bit-stream;

detecting and labeling blocks of a moving person within the image using the disparity map between the left image and the right image, and the motion vector, and detecting a distance from the labeled person to the camera; and

detecting a closer obstacle than the person using the extracted encoding type, motion vector, and DCT coefficient of the image, and the disparity map between the left image and the right image.

13. The method of claim 12, wherein the encoding uses any one of MPEG4, MPEG2, H.264, and H263.