Image depth perception device
First Claim
1. An image depth perception device, comprising:
- an image adaptive pre-processing sub-module configured;
to receive a standard speckle image having known depth and range information and an inputted speckle image sequence comprising inputted speckle images captured by an image sensor in a digital video format and having different characteristics of brightness, signal to noise ratio and size, wherein the standard speckle image is an image projected onto a plane perpendicular to the center axis of a speckle projector (z-axis) and having a known position distance, andto perform adaptive and uniform pre-processing on the standard speckle image and the inputted speckle image sequence, wherein the adaptive and uniform pre-processing comprises (a) converting format of the standard speckle image and the inputted speckle images from a Bayer format to a RGB format, (b) converting the RGB format to a YUV format by color space conversion, (c) processing adaptive de-noising on the YUV format, and (d) processing adaptive enhancing on the YUV format, so as to improve brightness uniformity of the inputted speckle images and the signal-to-noise ratio of both the standard speckle image and the inputted speckle images;
a memory for storing the pre-processed standard speckle image;
a block-matching motion estimation sub-module configured to compare each of the inputted speckle images in the pre-processed, inputted speckle image sequence with the pre-processed standard speckle image to calculate displacement of image blocks in the pre-processed inputted speckle image, wherein the block-matching motion estimation sub-module is configured to perform the comparison and the calculation by;
extracting an image block blockm×
n with a size of m×
n from the pre-processed inputted speckle image, andseeking, in the pre-processed standard speckle image, an optimum matching block of the image block blockm×
n within a search block search_blockM×
N having a size of M×
N and centered on a corresponding position of the image block blockm×
n, according to a search strategy and a similarity measurement indicator, such that the displacement of the image block blockm×
n is obtained, wherein M, N, m and n are integers, wherein M>
m and N>
n;
a depth calculating sub-module configured to calculate depth values of the image blocks in the pre-processed inputted speckle images, using a lookup table established according to different displacement values of the standard speckle image and corresponding depth values thereof, based on the displacement of the image blocks, to obtain a calculated depth image sequence, wherein one of the depth values of the image blocks is obtained from the lookup table when one of a horizontal displacement value and a vertical displacement value serves as an input value of the lookup table; and
a depth image post-processing sub-module configured to reduce noises of the calculated depth image sequence by noise-reducing filtering.
0 Assignments
0 Petitions
Accused Products
Abstract
In view of the active vision model based on structured light, a hardware structure of a depth perception device (a chip or an IP core) for high-precision images is disclosed. Simultaneously, the module is not only capable of serving as an independent chip, but also an embedded IP core in application. Main principle of the module is as follows. Speckle image sequence (obtained from an external image sensor and unknown depth information) is processed by adaptive and uniform pre-processing sub-module, then is inputted to the module to be compared with the standard speckle image (known depth information), then motion-vector information of the inputted speckle image is obtained by pattern matching of image blocks (similarity calculation) by the block-matching motion estimation sub-module, then depth image is obtained by depth calculation, and finally high-resolution sequence of depth image is outputted by post-processing the depth image.
22 Citations
12 Claims
-
1. An image depth perception device, comprising:
-
an image adaptive pre-processing sub-module configured; to receive a standard speckle image having known depth and range information and an inputted speckle image sequence comprising inputted speckle images captured by an image sensor in a digital video format and having different characteristics of brightness, signal to noise ratio and size, wherein the standard speckle image is an image projected onto a plane perpendicular to the center axis of a speckle projector (z-axis) and having a known position distance, and to perform adaptive and uniform pre-processing on the standard speckle image and the inputted speckle image sequence, wherein the adaptive and uniform pre-processing comprises (a) converting format of the standard speckle image and the inputted speckle images from a Bayer format to a RGB format, (b) converting the RGB format to a YUV format by color space conversion, (c) processing adaptive de-noising on the YUV format, and (d) processing adaptive enhancing on the YUV format, so as to improve brightness uniformity of the inputted speckle images and the signal-to-noise ratio of both the standard speckle image and the inputted speckle images; a memory for storing the pre-processed standard speckle image; a block-matching motion estimation sub-module configured to compare each of the inputted speckle images in the pre-processed, inputted speckle image sequence with the pre-processed standard speckle image to calculate displacement of image blocks in the pre-processed inputted speckle image, wherein the block-matching motion estimation sub-module is configured to perform the comparison and the calculation by; extracting an image block blockm×
n with a size of m×
n from the pre-processed inputted speckle image, andseeking, in the pre-processed standard speckle image, an optimum matching block of the image block blockm×
n within a search block search_blockM×
N having a size of M×
N and centered on a corresponding position of the image block blockm×
n, according to a search strategy and a similarity measurement indicator, such that the displacement of the image block blockm×
n is obtained, wherein M, N, m and n are integers, wherein M>
m and N>
n;a depth calculating sub-module configured to calculate depth values of the image blocks in the pre-processed inputted speckle images, using a lookup table established according to different displacement values of the standard speckle image and corresponding depth values thereof, based on the displacement of the image blocks, to obtain a calculated depth image sequence, wherein one of the depth values of the image blocks is obtained from the lookup table when one of a horizontal displacement value and a vertical displacement value serves as an input value of the lookup table; and a depth image post-processing sub-module configured to reduce noises of the calculated depth image sequence by noise-reducing filtering. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
Specification